Skip to main content
Proceedings of the National Academy of Sciences of the United States of America logoLink to Proceedings of the National Academy of Sciences of the United States of America
. 2022 Dec 13;119(51):e2214285119. doi: 10.1073/pnas.2214285119

A smartphone intervention that enhances real-world memory and promotes differentiation of hippocampal activity in older adults

Chris B Martin a,1, Bryan Hong b, Rachel N Newsome b, Katarina Savel b, Melissa E Meade b, Andrew Xia b, Christopher J Honey c, Morgan D Barense b,d,1
PMCID: PMC9907156  PMID: 36512503

Significance

The ability to vividly recollect our past declines with age, a trend that negatively impacts overall well-being. We show that using smartphone technologies to record and replay brief but rich memory cues from daily life can improve older adults’ ability to reexperience the past. This enhancement was associated with corresponding changes in the way memories were stored in the brain. Functional neuroimaging showed that repeatedly replaying memory cues drove memories apart from one another in the hippocampus, a brain region with well-established links to memory function. This increase in differentiation likely facilitated behavior by strengthening memory and minimizing competition among different memories at retrieval. This work reveals an easy-to-use intervention that helps older adults better remember their personal past.

Keywords: autobiographical memory, episodic memory, hippocampus, aging, fMRI

Abstract

The act of remembering an everyday experience influences how we interpret the world, how we think about the future, and how we perceive ourselves. It also enhances long-term retention of the recalled content, increasing the likelihood that it will be recalled again. Unfortunately, the ability to recollect event-specific details and reexperience the past tends to decline with age. This decline in recollection may reflect a corresponding decrease in the distinctiveness of hippocampal memory representations. Despite these well-established changes, there are few effective cognitive behavioral interventions that target real-world episodic memory. We addressed this gap by developing a smartphone-based application called HippoCamera that allows participants to record labeled videos of everyday events and subsequently replay, high-fidelity autobiographical memory cues. In two experiments, we found that older adults were able to easily integrate this noninvasive intervention into their daily lives. Using HippoCamera to repeatedly reactivate memories for real-world events improved episodic recollection and it evoked more positive autobiographical sentiment at the time of retrieval. In both experiments, these benefits were observed shortly after the intervention and again after a 3-mo delay. Moreover, more detailed recollection was associated with more differentiated memory signals in the hippocampus. Thus, using this smartphone application to systematically reactivate memories for recent real-world experiences can help to maintain a bridge between the present and past in older adults.


Autobiographical memory enables us to remember our personal past, and by extension contributes to our sense of identity (1, 2), the maintenance of social relationships (3, 4), and the ability to think about a self-relevant future (57). Retrieving an autobiographical memory is a layered process that involves recovery of general semantic knowledge (e.g., knowing what typically happens at a youth baseball game), personal semantic knowledge (e.g., knowing that you have a grandson who plays baseball), and recollection of episodic details that were unique to a specific event (e.g., remembering the look on your grandson’s face the first time he hit the ball). In neurologically healthy individuals, the ability to retrieve general and personal semantic knowledge typically remains constant across the lifespan, whereas recollection of event-specific details tends to decline with age, compromising our ability to vividly remember the past (8). The trajectory of this decline has been linked to corresponding reductions in the structural and functional integrity of the hippocampus (9, 10), which supports the encoding and retrieval of event-specific details from recent experiences (11, 12). Although memory decline is common in aging and significantly worsens quality of life, very few interventions specifically target autobiographical episodic memory. To fill this gap, we developed a smartphone application called HippoCamera, which is inspired by hippocampal function and designed to improve episodic recollection of real-world events in older adults.

HippoCamera is a digital memory aid that embodies principles from cognitive psychology and neuroscience known to improve memory, including distributed learning (13), deep encoding (14), the use of self-generated cues (15), the use of multimodal cues (16), and strengthening contextual associations for events (17). It provides users a tool to record and replay personalized, high-fidelity cues that capture the complex and dynamic nature of the real-world events they value most (Fig. 1 A and B). A HippoCamera cue consists of a self-generated 8-s verbal description (e.g., “Felix is playing baseball at Tom Brown Park”) that is played concurrently with an 8-s-speeded video (i.e., a 24-s video played at 3× speed, a design choice that enables efficient review and was inspired by the temporally compressed nature of endogenous hippocampal replay) (18). The application automatically curates replay sessions consisting of up to five distinct cues selected to balance distributed learning and prioritization of recent significant events over remote insignificant events. In the current study, cues were assigned to either a replay condition and viewed multiple times, or a within-subject baseline condition and never replayed. This design allowed us to target and characterize the effect of replaying HippoCamera cues while holding all other factors constant.

Fig. 1.

Fig. 1.

HippoCamera application and experimental timelines. (A) Cues are recorded in four steps: 1) Select a real-world event and initiate recording, 2) record an 8-s verbal description, 3) record a 24-s video, and 4) rate the event’s significance. (B) Replay sessions consist of up to 5 cues played sequentially. During replay, the 8-s audio is played concurrently with a speeded (3×) version of the video. Each cue is preceded by a text display that indicates approximately how much time has passed since the event as well as its exact date and time. (C) Experiment 1 timeline. (D) Experiment 2 timeline.

Hippocamera draws on the well-established phenomenon that a memory can be strengthened by retrieval, which alters how it is represented in the hippocampus (19, 20) and increases the probability of successfully retrieving it in the future (21). Evidence from computational modeling (22, 23), behavioral investigations (24, 25), and neuroimaging research (2628) suggests that strengthening is particularly likely when retrieval cues evoke strong neural reactivation of an episodic memory trace. Retrieval-related strengthening has variably been linked to hippocampal integration (i.e., increased neural similarity among memories; (2930) and hippocampal differentiation (i.e., decreased neural similarity among memories; (3132); both forms of representational change can reduce competition at retrieval (23). Importantly, integration tends to be observed when recall involves recovery of episodic gist, whereas differentiation is most apparent when engaging in elaborative recall of event-specific details (34). To the extent that repeatedly replaying high-fidelity HippoCamera cues evokes strong memory reactivation and elaborative recall, we hypothesized that its use would strengthen episodic autobiographical memory by promoting hippocampal differentiation, which would enable more detailed recall of event-specific information.

Memory strengthening may also be reflected in improved autobiographical sentiment. Mood disorders, including depression, are associated with impaired episodic memory for personally experienced events with intact semantic memory for general information and facts (35). Impairments in episodic memory negatively impact quality of life by reducing one’s sense of identity, confidence, and autonomy (36), and therefore further degrade emotional well-being. Moreover, episodic memory deficits are compounded in seniors with depression (37), highlighting the need for interventions that target episodic memory. As such, we also tested the hypothesis that replaying HippoCamera cues encourages repeated positive reminiscence and that this would be reflected in improved autobiographical sentiment.

Across two experiments, we found that older adults were able to easily incorporate HippoCamera into their daily lives with little supervision. Doing so enhanced episodic recollection of everyday experiences and produced more positive autobiographical sentiment at retrieval. Using fMRI, we revealed that replaying memory cues increased differentiation of activity patterns in the hippocampus, a measure that was positively correlated with the amount of event-specific episodic information that could be recalled. These findings indicate that repeated replay promoted differentiation of autobiographical memory representations in the hippocampus in a manner that facilitated detail-rich episodic retrieval.

Results

Cued Reactivation Improved Recollection of Event-Specific Detail.

Experiment 1.

Participants (N = 22, mean age 69.64 y ± 0.89 SEM, 16 women) used HippoCamera to record and replay episodic memory cues for events that took place over two consecutive weeks (Fig. 1C). During this time, they were instructed to record five events per day and view six replay sessions per day. Moreover, they were encouraged to distribute their replay sessions throughout each day, rather than view them in succession. This guidance was reinforced by smartphone notifications that intermittently reminded participants to record and replay. Compliance was generally high with an average of 4.8 ± 0.20 SEM cues recorded per day and 5.4 ± 0.27 SEM replay sessions viewed per day. Cues were randomly assigned to either the replayed or baseline condition on a per cue basis, meaning replayed and baseline events were interleaved both within and across days. Cues in the replayed condition were viewed an average of 8.7 ± 0.42 SEM times prior to memory testing. Cues in the baseline condition were never replayed and provided a within-subject comparison.

We assessed autobiographical memory performance using a cued-recall test administered at multiple time points–early and delayed (Fig. 1C). Early testing was completed immediately after the first and second weeks of HippoCamera use, with each assessment targeting memories for a subset of events that were recorded over the previous 7-d period. There were no significant differences in the number of details recalled between the first and second weeks of early testing (F(1, 21) = 0.72, P = 0.406), nor was there an interaction between week and replay condition (F(1, 21) = 0.11, P = 0.744). Delayed testing was completed 3.25 mo after early testing. Participants did not have access to their memory cues during this interval. We tested memory for the same set of events at early testing, delayed testing, and in the subsequent fMRI experiment (mean number of replayed trials = 17.1 ± 0.61 SEM, baseline trials = 17.2 ± 0.61 SEM). We selected cues for the purpose of testing in a manner that optimized matching between replayed and baseline memories in event significance, event frequency, and memory age (SI Appendix, section 1.1.3).

On each trial of our autobiographical memory tests, participants first viewed one of their 8-s self-generated cues, composed of the video and its associated verbal description. In all cases, this would be either a cue that had been recorded and then repeatedly replayed with HippoCamera (i.e., the replayed condition) or a cue that had been recorded but not previously replayed with HippoCamera (i.e., the baseline condition). Participants subsequently verbally recalled their memory for the corresponding event. They were encouraged to provide as many details as possible about the target event and were provided with an example response at the beginning of each test session (SI Appendix, section 1.1.1). Responses were scored to quantify retrieval of internal, external, and in-cue details (8, 38); SI Appendix, section 1.1.2). Internal details are event-specific and reflect episodic reexperiencing (e.g., recollecting the look on your grandson’s face when the first time he hit a baseball). External details reflect retrieval of general semantic information (e.g., knowing what tends to happen at a baseball game), personal semantics (e.g., knowing that your grandson is 4 y old), or details from a nontarget event (e.g., recollecting what happened in a different baseball game in which your grandson played). In-cue details reflect descriptions of information captured by the cues themselves, which may or may not involve memory retrieval. As such, in-cue details did not contribute to either internal or external detail counts (39, 40, 41).

Internal detail counts are shown in Fig. 2A. In line with our hypothesis, a Poisson generalized linear mixed model revealed a significant main effect of condition (replayed vs. baseline: b = 0.212, SE = 0.0183, z = 11.593, P < 0.001; SI Appendix, section 1.2 and Table S1), such that events in the replayed condition were recalled with significantly more Internal details than were those in the baseline condition. Proportionally, this reflected a 55.8% increase in internal details for replayed relative to baseline trials at early testing. Importantly, the benefit of replay on recall of internal details was observed in every participant. In addition to a main effect of condition, we found a main effect of test time point (early vs. delayed: b = −0.292, SE = 0.0336, z = −8.676, P < 0.001), indicating that participants recalled significantly more internal details at early testing than they did at delayed testing. Although the overall number of internal details declined over time, the relative benefit of cued reactivation was preserved: replayed events were recalled with 58.9% more event-specific details than baseline events at delayed testing. This pattern of results was also obtained after adjusting for verbal output by dividing detail count by word count on a trial-by-trial basis (SI Appendix, section 2.5 and Table S5) (42, 43). Moreover, repeated HippoCamera replay benefitted all subtypes of internal detail (internal event, time, place, perceptual, and thought/emotion; SI Appendix, section 2.3, Fig. S2, and Tables S2 and S3). Together, these data suggest that replay initially enhanced episodic autobiographical memory and that these enhancements persisted 3 mo after discontinuation of HippoCamera use.

Fig. 2.

Fig. 2.

Cued reactivation improved episodic recollection. (A) Mean number of Internal details for experiment 1. Early corresponds to behavioral performance measured during and immediately after HippoCamera use. Delayed corresponds to behavioral performance after a 3-mo delay, during which time participants did not have access to their memory cues. (B) Mean number of internal details for experiment 2. Open triangles reflect experiment 2 participants who failed the Montreal Cognitive Assessment. *** = P < .001.

We next examined whether repeatedly replaying HippoCamera memory cues influenced retrieval of external details. This set of exploratory analyses revealed a significant main effect of condition (b = −0.0335, SE = 0.0171, z = −1.964, P = 0.0495) and a significant interaction between condition and test time point (b = 0.0342, SE = 0.0172, z = 1.986, P = 0.0471). These effects reflect the fact that participants recalled significantly fewer external details for replayed as compared with baseline events at early testing (b = −0.135, SE = 0.0520, z = −2.603, P = 0.0092) but not delayed testing (b = 0.00139, SE = 0.0447, z = 0.031, P = 0.975; SI Appendix, sections 2.1 and 2.4, Figs. S1 and S3, and Tables S1, S3, and S4). Overall, these data indicate that using HippoCamera selectively enhanced recall of event-specific episodic information.

Lastly, we analyzed the number of in-cue details to determine whether HippoCamera replay made participants more reliant on cues at retrieval. We found a main effect of condition (b = −0.207, SE = 0.0295, z = −7.024, P < 0.001) that reflected fewer in-cue details in the replayed condition than the baseline condition. This result indicates that participants relied less on information reflected in their cues and more on elaborative retrieval processes following repeated replay with HippoCamera (SI Appendix, section 2.2, Fig. S1, and Table S1).

Experiment 2.

Experiment 2 was designed with two goals in mind. First, we wanted to replicate findings revealed in experiment 1 using a condition that reflected long-term autonomous HippoCamera use. To this end, an independent group of participants (n = 12, mean age 66.7 ± 0.81 SEM, 6 women) used HippoCamera for 10 consecutive weeks (Fig. 1D). They were encouraged to record one event per day and view one replay session per day. Compared with experiment 1, this protocol better approximated how older adults might use a digital memory aid outside of an experimental context. Compliance was high with an average of 0.95 ± 0.07 SEM cues recorded per day and 1.05 ± 0.03 SEM replay sessions viewed per day. Cues in the replayed condition were viewed an average of 7.8 ± 0.53 SEM times over the 70-d use period. By design, the mean number of replays per cue was comparable across experiments (Experiment 1 mean = 8.7) but distributed over a longer period in experiment 2.

Our second goal was to establish a purer baseline condition against which replayed memories could be compared. Anecdotal evidence from experiment 1 suggested that participants occasionally expected and/or hoped to replay cues that were in the baseline condition, which reflects some degree of unintended memory retrieval/reactivation. Moreover, randomly interleaving assignment of cues to the replayed and baseline conditions in experiment 1 may have led to an undesirable bleed over effect whereby replayed cues trigger retrieval of baseline memories if the events in question happened near together in time. These factors may have inflated the number of event-specific details that were recalled for baseline events in experiment 1. As such, HippoCamera cues in experiment 2 were assigned to either the replayed or baseline condition in a predictable, blocked manner, rather than the randomly interleaved assignment used in experiment 1. Blocked condition assignment alternated across weeks and was counterbalanced across participants. All participants were explicitly informed of these weekly condition switches to ensure that they never anticipated replay of cues in the baseline condition.

Early memory testing was completed 7 d after the 10-wk HippoCamera use period. Delayed testing was completed approximately 3.25 mo after early testing. We tested memory for the same subset of events at early and delayed testing (the mean number trials was identical across replayed and baseline conditions: M = 19.7 ± 0.33 SEM). Our behavioral results replicate and extend those from experiment 1 (41). The benefit of repeated replay on episodic recollection of real-world events was apparent in both the early and delayed testing periods (Fig. 2B). A Poisson generalized linear mixed model revealed that more Internal details were recalled in the replayed condition than in the baseline condition (main effect of condition: b = 0.250, SE = 0.0259, z = 9.660, P < 0.001), as well as at the early memory test relative to the delayed memory test (main effect of test time point: b = −0.1240, SE = 0.0324, z = −3.840, P < .001; SI Appendix, Table S1). There was also a significant interaction between condition and test time point (b = −0.0369, SE = 0.0134, z = −2.752, P = 0.006), driven by the fact that the difference in internal details between replayed and baseline events was larger at early testing (b = 0.574, SE = 0.0601, z = 9.549, P < 0.001) than it was at delayed testing (b = 0.427, SE = 0.0565, z = 7.560, P < 0.001). Proportionally, these differences can be quantified as an 83.8% increase in the number of event-specific details recalled for the replayed as compared with the baseline condition at early testing, and a 56.0% increase for the same comparison at delayed testing. As was the case in experiment 1, the benefit of replay on recall of internal details was observed in every participant. This pattern of results was also obtained after adjusting for verbal output (SI Appendix, section 2.5 and Table S5) (42, 43). Moreover, replay benefitted all subtypes of Internal details (Internal Event, Time, Place, Perceptual, and Thought/Emotion; SI Appendix, section 2.3, Fig. S2, and Tables S2 and S3). Notably, significantly more event-specific details were recalled in experiment 2 relative to experiment 1 (main effect of experiment: b = 0.376, SE = 0.0792, z = 4.751, P < 0.001), suggesting that cognitive behavioral interventions aimed at improving autobiographical memory may benefit from targeting a limited number of higher quality events per day and distributing review over a longer period.

Four of the 12 participants from experiment 2 failed the MoCA, suggesting that they may be showing early signs of cognitive decline. It is worth noting that replay enhanced episodic recollection of event-specific details in each of these individuals at early and delayed testing (open triangles in Fig. 2B). On average, 71% and 41% more internal details were recalled for replayed as compared with Baseline at early testing and delayed testing, respectively. At another level, one participant who passed the MoCA produced considerably more details than their peers. We note, however, that this individual did not show the largest proportional difference between baseline and replayed; the group average was 83.8%, the range was 33.7 to 170.45%, and the subject in question had an increase of 90.4%, which was 5th highest of the 12 participants. As such, excluding this participant from our statistical analysis did not eliminate or even weaken the effect.

We did not find any evidence for differences in recall of external details across conditions (SI Appendix, sections 2.1 and 2.4, Figs. S1 and S3, and Tables S1, S3, and S4), suggesting that replaying autobiographical memory cues does not influence subsequent retrieval of semantic information. This result is consistent with findings from experiment 1, and the broader notion that semantic knowledge tends to be relatively stable (8). We did find that the number of external details recalled differed significantly across early and delayed testing (main effect of test time point: b = −0.146, SE = 0.0355, z = −4.104, P < 0.001), reflecting the fact that participants recalled significantly more external details at early testing than they did at delayed testing. Although we did not predict this outcome, we speculatively suggest that there may be some dependency between internal and external details, such that a high number of internal details is accompanied by a relatively high number of external details that situate the target event in a broader context. Together, these findings converge with those from experiment 1 and suggest that HippoCamera specifically enhances recall of episodic autobiographical information.

As was the case in experiment 1, we also revealed a main effect of condition on in-cue detail counts (b = −0.194, SE = 0.0624, z = −3.111, P = 0.00186; SI Appendix, section 2.2 and Fig. S1), such that participants described fewer in-cue details for replayed memories than they did for baseline memories. These results indicate, again, that repeatedly replaying HippoCamera cues makes participants less reliant on these cues at retrieval and more reliant on elaborative episodic recollection.

Cued Reactivation Evoked More Positive Autobiographical Sentiment at Retrieval.

Having revealed that using HippoCamera to replay autobiographical memory cues selectively increased recall of event-specific episodic details, we next asked whether there were qualitative differences in the kind of language used to describe replayed and baseline memories (Fig. 3). To examine whether cued reactivation was associated with more positive memory-based event descriptions, we used a text-based sentiment analysis (Valence Aware Dictionary and sEntiment Reasoner; VADER) (44). This approach uses natural language processing to identify subjective states and quantify their polarity (i.e., positivity and negativity). As an example, the statement “We had an amazing time, and Felix was overjoyed when he hit the ball” will receive a more positive score than “We had a nice time, and Felix was pleased when he hit the ball”. Similarly, “We had a terrible time, and Felix was devastated when he struck out” will be scored more negatively than “We had a bad time, and Felix was disappointed when he struck out”.

Fig. 3.

Fig. 3.

Replay evoked more positive sentiment in autobiographical retrieval. (A) Composite sentiment scores from experiment 1 and (B) experiment 2. Composite values range from −1 (negative sentiment) to 1 (positive sentiment). Open triangles reflect experiment 2 participants who failed the Montreal Cognitive Assessment. * = P < 0.05, ** = P < 0.01.

We used paired-samples t tests to probe for differences between normalized composite sentiment scores that capture overall positivity and negativity for replayed and baseline memories. For experiment 1, this approach revealed that sentiment scores were significantly more positive for replayed memories than baseline memories at early testing (t(21) = 2.54, P < 0.01, d = 0.54) and delayed testing (t(19) = 2.11, P < 0.05, d = 0.47). A similar result was obtained for early testing in experiment 2 (t(11) = 2.42, P < 0.05, d = 0.70). This effect did not persist, however, at delayed testing in experiment 2 (t(11) = 0.28, P = 0.4, d = 0.08). Refer to SI Appendix, section 2.6 and Fig. S4) for sentiment frequency distributions. Comparing across experiments, we found that experiment 2 was generally associated with higher sentiment scores than experiment 1 (t(32) = 3.68, P < 0.001, d = 1.32). This difference may reflect an increase in the importance of the events recorded in experiment 2, for which we encouraged participants to record just one event per day.

Cued Reactivation Promoted Differentiation of Activity Patterns in the Hippocampus.

Having revealed that replaying autobiographical memory cues enhanced episodic recollection in older adults, we next sought to determine whether this effect was associated with increased differentiation of activity patterns in the hippocampus. To this end, we combined fMRI data (N = 25) obtained from participants in experiment 1 (N = 13 of 22 total participants) and experiment 2 (N = 12 of 12). For both experiments, cues were replayed a comparable number of times (Experiment 1 = 8.64; Experiment 2 = 7.77) and fMRI scanning was completed 7 d after the early autobiographical memory test (Fig. 2 A and C). In experiment 1, our recruitment efforts focused on finding older adults who were willing to use the HippoCamera application and visit our laboratory on a regular basis; willingness to be scanned was not a requirement for participation and several of our participants had medical implants (e.g., pacemaker), claustrophobia, arthritis, or other exclusions that made scanning not possible. For experiment 2, we completed extensive prescreening for all participants, including mock scanning sessions, to ensure that they would be able to complete the fMRI component of our experiment.

Each fMRI scanning session was designed to measure brain activity related to memory for the participant-specific events that had previously been probed in the early autobiographical memory test. We did this using three task components: Watch cue, mentally relive, and episodic probe (Fig. 4A). The mentally relive task component was unique to experiment 2. During the watch cue stage, participants watched and listened to one of their cues without having to make a behavioral response. After briefly fixating a centrally presented cross, they were then asked to mentally relive the event that was just cued in an unconstrained manner (45). They were instructed to do this by recollecting details from the cued event rather than simply visualizing the cue that was just played. No behavioral response was required during this stage. After another brief fixation, they completed an episodic probe task that involved making a “yes”/”no” judgment in response to a centrally presented probe word that referred to a person, place, thing, or action that the participant previously recalled during the early memory test. A given probe could be either a target or a lure. Targets reflected details that were true of the event in question. For example, we would use “KITE” as a target probe for a cue in which a boy is playing baseball if a participant previously recalled that her grandson was distracted by a kite flying in the park near the location of the game. Lures reflected details that were plausible but to our knowledge not true of the event in question. For example, we would use “LILLIAN” as a plausible lure for the baseball cue because the participant mentioned meeting this person when recalling a different event. This procedure (i.e., watch cue, mentally relive, and episodic probe) was repeated once for each tested memory, with repetitions appearing in separate runs. For the episodic probe task, one instance used a target probe word and the other used a lure. Participants correctly responded ‘yes’ to 86% ± 0.08 SEM of the target trials, and ‘no’ to 79% ± 0.11 SEM of the lure trials, indicating that our probes were successful in promoting recollection of event-specific details (82.5% overall accuracy).

Fig. 4.

Fig. 4.

Replay enhanced differentiation of activity in the hippocampus. (A) Each fMRI trial consisted of three phases. 1) Watch cue (8 s): view a HippoCamera audio–video cue, 2) mentally relive (8 s; Exp. 2): engage in elaborative recall of the event, and 3) episodic probe (4 s): judge whether a probe word captured an aspect of the event. (B) For each phase, we quantified differentiation (1 – Pearson’s r) between mean activity patterns obtained for all pairs of replayed memories and between all pairs of baseline memories. (C) Differentiation scores for each task phase of the fMRI task. Circle markers and solid lines depict data from experiment 1, whereas triangle markers and dashed lines depict data from experiment 2. Open triangles reflect experiment 2 participants who failed the Montreal Cognitive Assessment. * = P < 0.05, *** = P < 0.001.

We quantified differentiation of hippocampal activity patterns within each experimental condition separately using a representational similarity analysis (46) (Fig. 4B). Briefly, single-trial activity was estimated using a general linear model and extracted as spatially distributed patterns across the hippocampus. Activity patterns were averaged across two trials for each event cue, resulting in separate memory-specific estimates for the watch cue task component, the mentally relive task component, and the episodic Probe task component. For each of these tasks, we first quantified similarities between all pairs of replayed memories and between all pairs of baseline memories using Pearson’s r. These values were then subtracted from one and averaged to produce global measures of differentiation for replayed and baseline activity separately (Fig. 4C) (32, 33).

Our results indicated that using HippoCamera to replay autobiographical memory cues fundamentally altered the representational structure of episodic information in the hippocampus by promoting differentiation of memory-related activity patterns. A linear mixed model revealed a significant main effect of condition (replayed vs. baseline: b = 2.865 × 10−2, SE = 6.846 × 10−3, t(24) = 4.184, P < 0.001) and a significant interaction between condition and task (F(2, 2186) = 13.932, P < 0.001; SI Appendix, section 3.5 and Tables S6 and S7). This was primarily driven by increased differentiation in hippocampal activity patterns for replayed compared with baseline events during the episodic probe task (b = 0.0940, SE = 0.0157, t(45) = 5.981, P < 0.001) and the mentally relive task (b = 0.0641, SE = 0.0204, t(101) = 3.146, P = 0.0022). These results were also obtained after controlling for differences in retrieval-related BOLD effects across replayed and baseline trials (i.e., partial correlations using average voxel-wise activity within each condition as a covariate; episodic probe: t(24) = 1.972, P = 0.0301; Mentally Relive task: t(11) = 1.910, P = 0.0419). This finding indicates that replaying HippoCamera memory cues promotes differentiation in a manner that cannot be explained by differences in the magnitude of hippocampal activity across conditions. We found no evidence for a difference between replayed and baseline differentiation during the watch cue task (b = 0.0138, SE = 0.0157, t(45) = 0.875, P = 0.386). Nor did we find evidence for a significant main effect of task (watch cue vs. mentally relive vs. episodic probe: F(2, 11) = 1.532, P = 0.257). The condition by experiment interaction term was not significant (F(1, 23) = 2.42, P = 0.13), suggesting that methodological differences across experiments, such as the inclusion of the mentally relive task in experiment 2, did not influence the degree of differentiation across conditions. Moreover, restricting our analyses to activity from correct episodic probe trials did not change the overall pattern of results for this task component (mean baseline differentiation was 0.68 for all trials and 0.67 for correct trials; mean replayed differentiation was 0.77 for all trials and 0.78 for correct trials).

To provide a more comprehensive picture of the effect that replay had on activity patterns in the hippocampus, we probed for potential differences in differentiation across the hippocampal long axis (47, 48); SI Appendix, section 4.1, Fig. S6, and Tables S6 and S7). Briefly, this analysis revealed greater differentiation for replayed as compared with baseline trials in the anterior and posterior hippocampus for the mentally relive and episodic probe tasks. However, there were no significant interactions between ROI (anterior vs. posterior hippocampus) and either condition or task. In a second set of exploratory analyses, we asked whether the effect of replay on neural differentiation was unique to the hippocampus. To this end, we quantified differentiation in six structures that have been implicated in episodic memory processing (49, 50): ventromedial prefrontal cortex, anterior cingulate, posterior cingulate, angular gyrus, precuneus, and parahippocampal cortex. We also examined differentiation in a control region that has not been reliably linked to autobiographical memory (postcentral gyrus, i.e., primary somatosensory cortex). Briefly, we found marginal evidence for a main effect of condition (replayed > baseline) on differentiation scores in vmPFC (P = 0.02), posterior cingulate (P = 0.02), and precuneus (P = 0.05). However, none of these effects survived correction for multiple comparisons (SI Appendix, section 4.2, Fig. S6, and Tables S6 and S7). Thus, replaying cues with HippoCamera produced robust increases in neural differentiation in the hippocampus and marginal evidence for increased differentiation in a subset of cortical areas that have been linked to autobiographical memory retrieval.

Hippocampal Differentiation Was Positively Correlated with Episodic Recollection.

The degree of hippocampal differentiation of an event representation was positively correlated with the recollection of that event’s details on the unscanned early and delayed autobiographical memory tests (Fig. 5). Memory-specific differentiation scores were estimated by calculating mean pairwise pattern dissimilarities between a given memory and all other memories within the same condition. We then computed the correlation (Pearson’s r) between hippocampal differentiation and recall of internal details across trials for each participant separately. Because there is no principled reason to believe that the relationship between hippocampal differentiation and recall behavior differ qualitatively across our experimental conditions, we did not distinguish between replayed and baseline trials for the purpose of this analysis. Directional one-sample t tests against chance (i.e., correlation equal to zero) were performed using within-subject Fisher z-transformed correlation values. Using this approach, we found that degree of hippocampal differentiation measured during the episodic probe component of the fMRI task was positively correlated with the number of internal details recalled at early testing (t(24) = 3.97, P < 0.001, d = 0.795) and delayed testing (t(22) = 2.51, P < 0.01, d = 0.523). A similar result was obtained for the mentally relive component used in experiment 2 at early testing (t(11) = 3.45, P < 0.01, d = 0.997) and delayed testing (t(11) = 3.04, P < 0.05, d = 0.877). We also found above chance correlations between differentiation at the watch cue component and the number of internal details recalled at delayed testing (t(22) = 2.76, P < 0.05, d = 0.576). Conversely, we found limited evidence for any meaningful associations between degree of hippocampal differentiation and recall of external details (SI Appendix, section 4.3 and Fig. S7). Taken together, results from our pattern dissimilarity analyses revealed that replaying rich autobiographical memory cues promoted hippocampal differentiation, which in turn was positively correlated with recollection of event-specific episodic details on unscanned autobiographical memory tests administered 1 wk earlier and 3 mo later.

Fig. 5.

Fig. 5.

Degree of hippocampal differentiation is positively correlated with recollection of event-specific Internal details. Correlations (Pearson’s r) between hippocampal differentiation and recall of internal details during the early and delayed autobiographical memory tests. Each marker reflects the Fisher-z transformed r value obtained for one participant. Note that the mentally relive component was unique to experiment 2. Circles reflect participants from experiment 1. Triangles reflect participants from experiment 2. Open triangles denote experiment 2 participants who failed the Montreal Cognitive Assessment. Significance values indicate correlations greater than chance, i.e., correlation of zero, at the group level. * = P < 0.05, ** = P < 0.01, *** = P < 0.001.

Discussion

We have developed a smartphone-based intervention that uses self-generated, high-fidelity cues to improve memory for everyday events in older adults. Across two experiments, we found that repeatedly replaying rich autobiographical memory cues improved detail-rich recollection shortly after a 14-d (Experiment 1) and a 70-d (experiment 2) period during which participants used the application to record and repeatedly replay personally meaningful moments from their daily lives. This behavioral enhancement was also evident when memory was assessed a second time, 3 mo after participants stopped using the application. Moreover, replaying memory cues evoked more positive autobiographical sentiment at the time of retrieval. We used a pattern-based analysis approach with fMRI data to reveal changes in hippocampal activity related to our cued-reactivation protocol. This approach revealed increased differentiation of activity in the hippocampus related to memories for events that were previously replayed as compared with those that were recorded but never replayed. The extent of differentiation of memory-specific activity in the hippocampus was positively correlated with behavioral measures of episodic reexperiencing. Together, these findings support the conclusion that recording and replaying autobiographical memory cues can enhance episodic recollection by promoting differentiation of underlying representations in the hippocampus.

Replaying autobiographical memory cues enhanced recollection of event-specific details in older adults. Specifically, using HippoCamera to record and replay real-world memory cues generated a 56% increase in detailed episodic recollection after 14 d of use (experiment 1), and an 84% increase after 70 consecutive days of use (experiment 2). Importantly, the behavioral advantage of replay was apparent in every participant. This effect was selective in that our replay protocol did not consistently affect retrieval of semantic information (SI Appendix, Figs. S1 and S3). The significance of these results is apparent against a background of prior research that has revealed age-related declines in the episodic component of autobiographical memory (8, 51, 52). Our results indicate that replaying brief but detail-rich cues from daily life can combat the tendency for recall in older adults to be limited to high-level episodic details by helping to preserve the event-specific information that enables mnemonic discrimination among multiple related episodes. The examples “Felix played baseball” and “Felix played baseball and I was thrilled to see the joy on his face the first time he hit the ball” captures this difference between recall of episodic gist versus recall that is characterized by detail-rich mnemonic elaboration. Moreover, this replay-related enhancement persisted after a 3-mo delay between using HippoCamera and our second autobiographical memory assessment. Accordingly, by contributing to the preservation and accessibility of detail-rich memories, our cognitive behavioral intervention helps to bridge the present with the episodic past in older individuals. In particular, vivid, detail-rich retrieval establishes a more stable and reliable connection between the present self and past self than does retrieval that is deprived of event-specific details. To the extent these connections contribute to the maintenance of personal identity, we believe that the memory enhancements revealed here are meaningful (1, 2).

How does replaying real-world autobiographical memory cues improve episodic recollection? Remembering an event influences the probability and quality of future recollection (1921), and successful retrieval is thought to beget future retrieval success, while incomplete or failed retrievals reduce the likelihood of future retrieval success (2228). Our behavioral results are consistent with the idea that replaying high-fidelity, self-generated cues from everyday events evokes strong reactivation of event memories and therefore strengthens associations among episodic details. Moreover, building on prior laboratory-based work that examined episodic memory for pairs of object or video clips, they reveal a link between memory reactivation and enhanced recollection of complex and dynamic autobiographical events. We obtained this result despite using a cued-reactivation protocol that was not accompanied by explicit retrieval demands, which have been shown to improve subsequent memory relative to mere reexposure of previously encountered information (21, 53). Although this study was not designed to reveal potential differences between the effect of reexposure and retrieval demands on subsequent memory, we note that we did observe improved episodic recollection of event details even after excluding in-cue details (i.e., details that were captured by either the audio or video component of the memory cues; Fig. 2 and SI Appendix, Fig. S1). This observation suggests that reexposure to sufficiently rich cues can improve memory for uncued information about the cued event even in the absence of any retrieval demands.

Results from our pattern similarity analyses indicated that replaying detail-rich cues increased differentiation of memory-related activity in the hippocampus. In other words, repeated reactivation minimized similarities between episodic memory representations by reducing representational overlap in the hippocampus. Importantly, the degree to which a given memory representation was differentiated predicted the quality of episodic recollection, such that greater dissimilarity was associated with retrieval of more event-specific details. By linking the differentiation of activity in the hippocampus to enhanced episodic recollection, our results dovetail with evidence from computational modeling suggesting that greater differentiation among memory representations can reduce competition at retrieval and therefore facilitate detail-rich recollection (23, 54). The fact that increased differentiation was driven by HippoCamera replay after initial encoding suggests that our results cannot be fully explained by the notion that the hippocampus minimizes overlap among memory representations at the time of encoding (i.e., pattern-separation; (5556). Accordingly, we believe that using HippoCamera to replay memory cues differentiates underlying representations in a manner that protects against the postencoding loss of detail-based information.

The impact of episodic memory loss is far-ranging and includes reductions in one’s sense of identity, confidence, and autonomy (1, 2, 36). Because HippoCamera was designed to enhance episodic memory in older adults, we also asked whether replaying would evoke more positive autobiographical sentiment at the time of retrieval. Although we did not assess mood or screen for depressive disorders in this study, we reasoned that replay would encourage repeated positive reminiscence and that this would be reflected in improved autobiographical sentiment in our behavioral memory assessments (e.g., baseline retrieval: “We were at the park for almost the entire day” vs. replayed retrieval: “We were at the park for almost the entire day and it was just lovely”). Our results indicated that this was indeed the case, as sentiment scores were significantly more positive for replayed as compared with baseline memories at early and delayed testing in experiment 1 and at early testing in experiment 2. When considered in the context of a broader literature that links memory impairment to mood disorders, including depression (35, 37), these results suggest that using HippoCamera as a memory aid may contribute to emotional well-being.

Although the current set of experiments was designed to characterize the effect of HippoCamera replay on memory, it is worth noting that this memory aid also enriches encoding in several important ways. As such, because the replayed and baseline conditions had identical encoding protocols, events in both conditions were processed at encoding in a fundamentally different way from the incidental manner by which we typically learn about the world and encode new events. By incentivizing the creation of memory cues, HippoCamera encourages a shift from incidental encoding to intentional encoding, which is known to improve retrieval success in older adults (57). Anecdotally, many of our participants reported increased awareness of the world around them (e.g., one individual described their experience as a participant as follows, “… It was very motivational. I started to have more confidence in myself and started to be more aware of things around me. I think it made my memory go up a shot.”). Creating self-generated memory cues also promotes deep processing of event details and meaning, which HippoCamera achieves by requiring a brief verbal description of the event, a video that captures diagnostic perceptual information, and a rating of the personal significance of the event (Fig. 1A). Importantly, this deep processing ensures correspondence between the retrieval cue and the information encoded in memory (15). Lastly, numerous participants reported having varied their behavioral repertoire out of a desire to record and replay interesting content. For example, one participant noted “… Sometimes I would be sitting at home and realize that I needed to film something so I would go out to the library or the church just to have something to do”. Considering these points together, there was likely enriched encoding of events in both replayed and baseline conditions, and so we likely underestimated the beneficial effects of using HippoCamera when we contrasted replayed and baseline conditions. Ultimately, further research is required to isolate and quantify the contribution of enhanced encoding, perhaps through comparison of HippoCamera with a passive photograph system (i.e., wearable camera). At another level, we also note that it is not clear how best to define a baseline condition when conducting research outside of a controlled laboratory environment. We opted for a relatively conservative approach to ensure that we did not overestimate the effect of repeated replay.

This set of experiments, and HippoCamera more generally, builds on over a decade of research that has examined the efficacy of wearable cameras as memory aids in both healthy and cognitively impaired populations (39, 5859; for review, see refs. 40, 60, and 61). Wearable cameras, which passively capture photographs for later review, provide richer, more voluminous information about past events than more traditional memory aids such as written diaries. Indeed, reviewing such photographs improves episodic memory for everyday experiences relative to review of diary entries, and modulates activity in key memory-related brain areas, including the hippocampus (40, 60). Despite these promising outcomes, wearable cameras present practical barriers that make it difficult for older adults and individuals with memory impairment to easily integrate them into their daily life. First, they require companion hardware (e.g., a computer) and the technical proficiency to transfer still photographs for review. Second, the review process is time consuming as it can involve hundreds to thousands of indiscriminately captured photographs, creating an inefficient user experience and caregiver burden that compounds as cues accumulate over time. HippoCamera removes these practical barriers by enabling high-quality cues to be recorded and replayed on the same device, and by coupling automatically curated replayed sessions with periodic notifications reminding users to record and review their memory cues. Moreover, neither wearable camera technology nor popular social media platforms focus on enriching the encoding experience. HippoCamera is, to our knowledge, the only digital memory application that overtly promotes deep encoding strategies. At another level, few studies conducted with wearable cameras have deeply probed memory quality to determine whether the beneficial effects of photograph review extend beyond improved recognition of previously studied images. Here, we adapted Autobiographical Interview scoring by discounting in-cue details to demonstrate that repeatedly replaying HippoCamera memory cues improved memory for unstudied aspects of everyday experiences. Moreover, extant fMRI research that has used pictures from wearable cameras as stimuli has not asked the theoretically important question of how picture review improves memory (e.g., via pattern differentiation, integration of neural representations, or gain modulation). Using HippoCamera, we have revealed evidence that points to replay-related memory differentiation as a neural mechanism that can explain improvements in recall of episodic event details.

Is replaying with HippoCamera superior to reviewing non-HippoCamera content captured using a smartphone or camera? We think so. First, the guided recording process means that memory encoding and retrieval is intentional: participants are encouraged to pay greater attention to the events of their lives and generate cues that are specifically designed to trigger later retrieval. Second, our application curates replay sessions with powerful memory cues by guiding users to systematically reflect on their past in a highly structured and targeted way. Third, having a dedicated application with daily targets and notifications that remind participants to record and replay cues reinforces memory-oriented behavior and cognition in a manner that casual smartphone use does not. Compliance often presents a considerable barrier in the context of cognitive behavioral interventions in older adults (62). It is noteworthy that we obtained high compliance in every one of our older adult participants in terms of meeting daily record and replay expectations in experiment 1 and experiment 2.

A key finding revealed here is that repeated replay of HippoCamera cues strengthens memories in a manner that enhances detail-rich retrieval and increases differentiation among activity patterns in the hippocampus. A notable limitation of this study that impacts on interpretation of these results is the fact that these replay-related benefits were obtained using well-learned cues to prompt retrieval (i.e., previously replayed HippoCamera cues). It is thought that retrieval reflects an interaction between a cue and an established memory trace (i.e., ecphory; (63, 64). This raises the important question of whether the memory strengthening we observed reflects a narrow benefit that is obtained only when prompted by a previously viewed HippoCamera cue, or if this effect generalizes to other nonpracticed cues and/or retrieval contexts. At a minimum, we can make the strong claim that replaying HippoCamera memory cues increases ecphory when those precise cues are later used to trigger retrieval. However, we have not systematically asked whether the benefit of replay is attenuated when using test cues that are related to but different from those that were replayed with HippoCamera. Nevertheless, data from a single-case study conducted with a hippocampal amnesic patient speaks to this issue (SI Appendix, section 5 and Figs. S8 and S9). Notably, while she could not recall any episodic details for 3-wk-old events memorialized in her diary, she produced some details for 3-wk-old events recorded and repeatedly replayed with HippoCamera. In fact, she recalled some details spontaneously in conversation suggesting that replaying rich, self-generated autobiographical memory cues may have benefits that can be detected without cues.

Finally, we note that 65% of our participants were women, and that women usually outperform men in episodic memory performance (65). Therefore, to confirm the generality of these findings across genders and sexes, we will need to test these factors as moderators in a larger sample replay-based memory intervention.

In sum, we have developed a smartphone-based memory aid that uses cued reactivation of real-world events to improve episodic recollection in older adults. This beneficial outcome was linked to a corresponding increase in the differentiation of activity in the hippocampus and more positive autobiographical sentiment at the time of retrieval. By strengthening connections between the present and past self, this application presents a noninvasive approach to mitigating real-world age-related memory decline. More generally, to the extent that autobiographical memory makes important contributions to other aspects of cognition and the maintenance of meaningful interpersonal relationships, this intervention has potential to promote graceful aging.

Materials and Methods

Participants.

Experiment 1.

Twenty-two neurologically healthy older adults (mean age = 69.64 y ± 0.89 SEM, range = 62 to 76 y, 16 women) participated in experiment 1. Two participants were unable to complete a delayed autobiographical memory test. All participants obtained a passing score on the Montreal Cognitive Assessment (MoCA) (mean score = 27.10 ± 0.26 SEM, range = 26 to 30), suggesting that these individuals were cognitively healthy at the time of testing (66). Of this sample, 13 individuals also participated in the fMRI experiment (mean age = 68.84 y ± 1.19 SEM, range = 62 to 76 y, 10 women, mean MoCA = 27.62 ± 0.35 SEM).

Experiment 2.

Twelve neurologically healthy older adults participated in experiment 2 (mean age = 66.77 y ± 0.81 SEM, range = 61 to 71, 6 women, mean MoCA score = 26.58 ± 0.85 SEM). Four of these individuals failed the MoCA (scores of 20, 24, 25, and 25) but had no documented history of neurological or cognitive disorder. The research protocol for both experiments was reviewed and approved by the Research Ethics Board at The University of Toronto. Written informed consent was obtained from all participants prior to HippoCamera use and again before fMRI data collection.

HippoCamera Smartphone Application.

We developed the HippoCamera application using a participatory design approach aimed at optimizing ease of use and enjoyability in older adults. Cues can be created in approximately 40 s and replayed in approximately 60 s. All participants were able to easily incorporate the application into their daily routines and most indicated that they enjoyed doing so. As evidence, we note that nearly all participants met our expectations in terms of how many cues should be recorded per day and how many replay sessions should be viewed per day in both experiments.

This research was conducted using a beta version of the application developed by our research team. The software was subsequently updated by Tactica Interactive (Winnipeg, Manitoba) to improve stability, enhance visualizations, enable cloud-based storage, and allow researchers to remotely access content recorded by participants. This version of the application can be obtained for scientific purposes from Apple’s App Store and Google Play. As of the time of writing, this is a research-dedicated application that requires an access code that can be obtained from a corresponding author.

Recording Cues with HippoCamera: Cues are recorded in a 4-step process, with automated transitions between steps (Fig. 1A). First, participants intentionally make the decision to record an ongoing event. Second, they capture an 8-s audio recording that is a self-generated verbal description of the target event. Third, they record a 24-s high-definition video. Fourth, they rate the significance of the event using a 1 to 5 scale. Meta-data are also automatically recorded, including time, date, and GPS co-ordinates, when available. Upon completing a recording, the application generates integrated cues that combine the verbal description with a speeded version of the video. Audio is stripped from the 24-s video and the resulting file is accelerated by a factor of three, resulting in an 8-s-speeded video. The speeded video is then coupled with the audio file containing the 8-s verbal description. A notification system can be toggled on or off to encourage participant compliance. For the current study, these notifications reminded participants to record five cues per day in experiment 1 and one cue per day in experiment 2.

Replaying Cues with HippoCamera: Replay takes place within sessions that consist of up to five sequentially presented cues that are automatically selected (Fig. 1B). Cues are separated by a lead-in screen that displays the date and approximate age of the event. Within a session, cues are replayed in reverse chronological order, meaning newer cues are played before older cues. The application is designed to replay cues in a manner that achieves balance between distributed learning and the prioritization of recent, highly significant events. For the current research, notifications encouraged participants to replay 6 times per day in experiment 1 and once per day in experiment 2.

Replay Instructions: At intake, participants were given guidance regarding when they should initiate replay sessions and how they should engage with replayed content. Specific instructions were developed to standardize our replay protocol as much as is possible in the context of unsupervised, real-world research. We instructed participants to initiate replay sessions at relatively quiet moments to minimize distraction and optimize effortful recollection. Although we did not communicate firm expectations, we suggested that they anchor some of their replay sessions to routine events that often present distraction-free opportunities for replay, such as while drinking coffee in the morning. We also instructed participants to engage in deep, contemplative processing of memories during replay by asking them to actively recollect a specific aspect of each cued event. This approach was motivated by the large body of research that has revealed a clear benefit for active retrieval over mere reexposure to a stimulus (i.e., the test effect; (67). Lastly, participants were encouraged to engage in a moment of reflection using in-app instructions that were presented on screen at the conclusion of each replay session: “Take a moment to reflect on the memories reviewed”. This prompt was designed to give participants an opportunity to contemplate their memories at a more leisurely pace than is possible during the preceding replay session. Moreover, the open-ended nature of the guidance allowed them to flexibly focus on their personal past in a self-directed manner.

Autobiographical Memory Test.

Behavioral performance was assessed using an adapted version of the Autobiographical Interview (8) (SI Appendix, section 1.1.1). Whereas the Autobiographical Interview allows participants to choose which memories to recall, we prompted retrieval by presenting participants with their self-generated HippoCamera cues. Trial order was randomized, and experimenters were blinded to the condition of each cue. Verbal responses were recorded, transcribed, and then scored using an adapted version of the Autobiographical Interview protocol (38) (SI Appendix, section 1.1.2). Specifically, each recalled detail was scored as being either Internal, External, or In-Cue. Internal and external detail counts excluded in-cue details, meaning descriptions of information that was readily apparent in the cue (either the verbal description or video) did not count toward recall. In other words, we conservatively discounted details that a naïve observer could also describe based on having seen only the cue, but not having experienced the event. See SI Appendix, section 1.1.3 for details regarding stimulus selection.

Sentiment Analysis.

Cued-recall transcripts were processed using VADER, a lexicon and rule-based sentiment analysis tool that systematically identifies and quantifies affective states communicated in natural language (44). The polarity and intensity scores of the 9,000 words comprising the VADER lexicon reflect sentiment ratings from human observers. VADER generates scores that reflect positive sentiment, negative sentiment, neutral sentiment, and a normalized weighted composite score that ranges between −1 (extremely negative) and 1 (extremely positive). We used composite scores as dependent measures, as they provide a single unidimensional sentiment value. Although in-cue details were excluded from internal and external detail counts, they were not excluded for the purpose of performing our sentiment analyses because there is no principled reason to predict that in-cue details will be described with more positive or more negative valence across conditions.

fMRI Task.

Data for experiment 1 were collected across 4 functional runs. Data from one run could not be collected for one participant due to a technical error. Data for experiment 2 were collected across eight functional runs. Three participants were unable to complete all eight runs due to physical discomfort: one completed four runs, two completed six runs. Each experiment 1 trial was 16 s in duration and consisted of two subtasks (Watch Cue and Episode Probe) separated by baseline fixation (Fig. 4A). More specifically, participants first watched a HippoCamera cue (8 s), then fixated on a centrally presented cross (4 s), and finally responded to an episodic verification probe (4 s). Each experiment 2 trial was 28 s in duration and consisted of three sub-tasks (Watch Cue, Mentally Relive, and Episode Probe) separated by baseline fixation (Fig. 4A). Participants first watched a HippoCamera cue (8 s), then fixated on a centrally presented cross (4 s), then actively recollected the cued event (8 s), then fixated on a centrally presented cross (4 s), and finally responded to an episodic verification probe (4 s). Trials in each experiment were separated by a jittered baseline fixation that spanned 4 to 8 s in duration. A brief audio tone was played 500 ms prior to the onset of a new trial. Moreover, the onset of each subtask within a trial was signaled by changing the color of the baseline fixation cross from white to red. All timing parameters were inspired by prior research in which participants viewed and later mentally relived audio–video stimuli that were similar in duration to the HippoCamera cues used here (68).

See SI Appendix, sections 3.1–3.4 for details regarding fMRI data acquisition, preprocessing procedures, ROI definition, and analytical procedures.

Supplementary Material

Appendix 01 (PDF)

Acknowledgments

This work was supported by Project Grants from the Canadian Institutes for Health Research to M.D.B. (PJT-173336 and PJT-126003), a Scholar Award from the James S McDonnell Foundation to M.D.B., a Connaught Innovation Award to M.D.B., and a Centre for Aging & Brain Health Innovation Researcher Clinician Partnership Program Grant to M.D.B. We are grateful to Miranda Chang, Alana Changoor, Haleema Khan, Grace Manalili, Arya Rahbarnia, Ashmita Singh, My An Tran, and Shamini Vijaya Kumar for their contributions to recruitment, data collection, and transcription. We thank Celia Fidalgo for assistance with statistical analyses.

Author contributions

C.B.M., C.J.H., and M.D.B. designed research; C.B.M., B.H., R.N.N., and K.S. performed research; C.B.M., B.H., and A.X. contributed new reagents/analytic tools; C.B.M., B.H., R.N.N., K.S., and M.E.M. analyzed data; and C.B.M., C.J.H., and M.D.B. wrote the paper.

Competing interests

The authors declare a competing interest. The authors have organizational affiliations to disclose, C.B.M, B.H., R.N.N., A.X., C.J.H., and M.D.B. own shares in Dynamic Memory Solutions Inc., a company focused on developing digital tools to improve memory. The University of Toronto holds the ownership rights to the HippoCamera technology used to conduct the research described herein, but has given Dynamic Memory Solutions the rights to commercialize. No person, nor organization received any financial remuneration for the use of the HippoCamera application in these studies. At the time of publication, this is a research-dedicated application that we will make available to other memory scientists at no charge., Yes, the authors have stock ownership to disclose, C.B.M, B.H., R.N.N., A.X., C.J.H., and M.D.B. own shares in Dynamic Memory Solutions Inc., a company focused on developing digital tools to improve memory., Yes, the authors have patent filings to disclose, Patent No.: 11,397,774.

Footnotes

This article is a PNAS Direct Submission.

Contributor Information

Chris B. Martin, Email: cmartin@psy.fsu.edu.

Morgan D. Barense, Email: morgan.barense@utoronto.ca.

Data, Materials, and Software Availability

Anonymized behavioral data have been deposited on the OSF (https://osf.io/7g2rx/; doi:10.17605/OSF.IO/7G2RX). fMRI data have been deposited to OpenNeuro (doi:10.18112/openneuro.ds004336.v1.0.0).

Supporting Information

References

  • 1.Conway M. A., Pleydell-Pearce C. W., The construction of autobiographical memories in the self-memory system. Psychol. Rev. 107, 261–288 (2000). [DOI] [PubMed] [Google Scholar]
  • 2.Addis D. R., Tippett L., Memory of myself: Autobiographical memory and identity in Alzheimer’s disease. Memory 12, 56–74 (2004). [DOI] [PubMed] [Google Scholar]
  • 3.Gaesser B., Schacter D. L., Episodic simulation and episodic memory can increase intentions to help others. Proc. Natl. Acad. Sci. U.S.A. 111, 4415–4420 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Harris C. B., Rasmussen A. S., Berntsen D., The functions of autobiographical memory: An integrative approach. Memory 22, 559–581 (2014). [DOI] [PubMed] [Google Scholar]
  • 5.Pillemer D., Directive functions of autobiographical memory: The guiding power of the specific episode. Memory 11, 193–202 (2003). [DOI] [PubMed] [Google Scholar]
  • 6.Schacter D. L., Addis D. R., Buckner R. L., Remembering the past to imagine the future: The prospective brain. Nat. Rev. Neurosci. 8, 657–661 (2007). [DOI] [PubMed] [Google Scholar]
  • 7.Bluck S., Alea N., Habermas T., Rubin D. C., A tale of three functions: The self-reported uses of autobiographical memory. Social Cogn. 23, 91–117 (2005). [Google Scholar]
  • 8.Levine B., Svoboda E., Hay J. F., Winocur G., Moscovitch M., Aging and autobiographical memory: Dissociating episodic from semantic retrieval. Psychol. Aging 17, 677–689 (2002). [PubMed] [Google Scholar]
  • 9.Gorbach T., et al. , Longitudinal association between hippocampus atrophy and episodic-memory decline. Neurobiol. Aging 51, 167–176 (2017). [DOI] [PubMed] [Google Scholar]
  • 10.Cabeza R., et al. , Maintenance, reserve and compensation: The cognitive neuroscience of healthy ageing. Nat. Rev. Neurosci. 19, 701–710 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Scoville W. B., Milner B., Loss of recent memory after bilateral hippocampal lesions. J. Neurol. Neurosurg. Psychiatry 20, 11–21 (1957). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Tulving E., Markowitsch H. J., Episodic and declarative memory: Role of the hippocampus. Hippocampus 8, 198–204 (1998). [DOI] [PubMed] [Google Scholar]
  • 13.Ebbinghaus H., Memory: A contribution to experimental psychology. Ann Neurosci. 20, 155–156 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Craik F. I., Lockhart R. S., Levels of processing: A framework for memory research. J. Verbal Learn. Verbal Beh. 11, 671–684 (1972). [Google Scholar]
  • 15.Mäntylä T., Nilsson L. G., Cue distinctiveness and forgetting: Effectiveness of self-generated retrieval cues in delayed recall. J. Exp. Psychol. Learn. Mem. Cogn. 14, 502–509 (1988). [Google Scholar]
  • 16.Tan D. S., Stefanucci J. K., Proffitt D. R., Pausch R., “The Infocockpit: Providing location and place to aid human memory” in Proceedings of the 2001 Workshop on Perceptive User Interfaces (ACM, 2001), pp. 1–4, 10.1145/971478.971526. [DOI] [Google Scholar]
  • 17.Robin J., Wynn J., Moscovitch M., The spatial scaffold: The effects of spatial context on memory for events. J. Exp. Psychol. Learn. Mem. Cogn. 42, 308–315 (2016). [DOI] [PubMed] [Google Scholar]
  • 18.Skaggs W. E., McNaughton B. L., Wilson M. A., Barnes C. A., Theta phase precession in hippocampal neuronal populations and the compression of temporal sequences. Hippocampus 6, 149–172 (1996). [DOI] [PubMed] [Google Scholar]
  • 19.McGaugh J. L., Memory–A century of consolidation. Science 287, 248–251 (2000). [DOI] [PubMed] [Google Scholar]
  • 20.Dudai Y., Karni A., Born J., The consolidation and transformation of memory. Neuron 88, 20–32 (2015). [DOI] [PubMed] [Google Scholar]
  • 21.Roediger H. L., Karpicke J. D., Test-enhanced learning: Taking memory tests improves long-term retention. Psychol. Sci. 17, 249–255 (2006). [DOI] [PubMed] [Google Scholar]
  • 22.Norman K. A., Newman E. L., Detre G., A neural network model of retrieval-induced forgetting. Psychol. Rev. 114, 887–953 (2007). [DOI] [PubMed] [Google Scholar]
  • 23.Ritvo V. J., Turk-Browne N. B., Norman K. A., Nonmonotonic plasticity: How memory retrieval drives learning. Trends Cogn. Sci. 23, 726–742 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.St. Jacques P. L., Schacter D. L., Modifying memory: Selectively enhancing and updating personal memories for a museum tour by reactivating them. Psychol. Sci. 24, 537–543 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Sinclair A. H., Barense M. D., Surprise and destabilize: Prediction error influences episodic memory reconsolidation. Learn. Mem. 25, 369–381 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Newman E. L., Norman K. A., Moderate excitation leads to weakening of perceptual representations. Cereb. Cortex 20, 2760–2770 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Detre G. J., Natarajan A., Gershman S. J., Norman K. A., Moderate levels of activation lead to forgetting in the think/no-think paradigm. Neuropsychologia 51, 2371–2388 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Wang T. H., Placek K., Lewis-Peacock J. A., More is less: Increased processing of unwanted memories facilitates forgetting. J. Neurosci. 39, 3551–3560 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Schlichting M. L., Mumford J. A., Preston A. R., Learning-related representational changes reveal dissociable integration and separation signatures in the hippocampus and prefrontal cortex. Nat. Commun. 6, 1–10 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Robin J., Buchsbaum B. R., Moscovitch M., The primacy of spatial context in the neural representation of events. J. Neurosci. 38, 2755–2765 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.LaRocque K. F., et al. , Global similarity and pattern separation in the human medial temporal lobe predict subsequent memory. J. Neurosci. 33, 5466–5474 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Ezzyat Y., Inhoff M. C., Davachi L., Differentiation of human medial prefrontal cortex activity underlies long-term resistance to forgetting in memory. J. Neurosci. 38, 10244–10254 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Martin CB, Hong B, Newsome RN, Savel K, Meade ME, Xia A, Honey CJ, Barense MD, Dataset: A smartphone intervention that enhances real-world memory and promotes differentiation of hippocampal activity in older adults. Open Science Framework https://osf.io/7g2rx/ Deposited 16 November 2022 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Brunec I. K., Robin J., Olsen R. K., Moscovitch M., Barense M. D., Integration and differentiation of hippocampal memory traces. Neurosci. Biobehav. Rev. 118, 196–208 (2020). [DOI] [PubMed] [Google Scholar]
  • 35.Söderlund H., et al. ,Autobiographical episodic memory in major depressive disorder. J. Abnorm. Psychol., 51–60 (2014). [DOI] [PubMed] [Google Scholar]
  • 36.Waters T. E. A., Relations between the functions of autobiographical memory and psychological wellbeing. Memory 22, 265–275 (2014). [DOI] [PubMed] [Google Scholar]
  • 37.Christman S., et al. , Accelerated brain aging predicts impaired cognitive performance and greater disability in geriatric but not midlife adult depression. Transl. Psychiatry 10, 1–11 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Renoult L., et al. , Classification of general and personal semantic details in the Autobiographical Interview. Neuropsychologia 144, 107501 (2020). [DOI] [PubMed] [Google Scholar]
  • 39.Loveday C., Conway M. A., Using SenseCam with an amnesic patient: Accessing inaccessible everyday memories. Memory 19, 697–704 (2011). [DOI] [PubMed] [Google Scholar]
  • 40.Silva A. R., Pinho M. S., Macedo L., Moulin C. J. A., A critical review of the effects of wearable cameras on memory. Neuropsychol. Rehabil. 28, 117–141 (2018). [DOI] [PubMed] [Google Scholar]
  • 41.Martin CB, Hong B, Newsome RN, Savel K, Meade ME, Xia A, Honey CJ, Barense MD, A smartphone intervention that enhances real-world memory and promotes differentiation of hippocampal activity in older adults. OpenNeuro https://openneuro.org/datasets/ds004336 Deposited 16 November 2022 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Lockrow A. W., et al. , Taking stock of the past: A psychometric evaluation of the Autobiographical Interview. bioRxiv [Preprint] (2021). 10.1101/2021.12.22.473803 (Accessed 16 November 2022). [DOI] [PubMed]
  • 43.Setton R., Sheldon S., Turner G. R., Spreng R. N., Temporal pole volume is associated with episodic autobiographical memory in healthy older adults. Hippocampus 32, 373–385 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Hutto C., Gilbert E., “Vader: A parsimonious rule-based model for sentiment analysis of social media text” in Eighth International AAAI Conference on Weblogs and Social Media (2014). [Google Scholar]
  • 45.Tulving E., Memory and consciousness. Can. Psychol. 26, 1–12 (1985). [Google Scholar]
  • 46.Kriegeskorte N., Mur M., Bandettini P. A., Representational similarity analysis-connecting the branches of systems neuroscience. Front. Syst. Neurosci. 2, 4 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Poppenk J., Evensmoen H. R., Moscovitch M., Nadel L., Long-axis specialization of the human hippocampus. Trends Cogn. Sci. 17, 230–240 (2013). [DOI] [PubMed] [Google Scholar]
  • 48.Brunec I. K., et al. , Multiple scales of representation along the hippocampal anteroposterior axis in humans. Curr. Biol. 28, 2129–2135 (2018). [DOI] [PubMed] [Google Scholar]
  • 49.Svoboda E., McKinnon M. C., Levine B., The functional neuroanatomy of autobiographical memory: A meta-analysis. Neuropsychologia 44, 2189–2208 (2006). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Schacter D. L., Addis D. R., Buckner R. L., Remembering the past to imagine the future: The prospective brain. Nat. Rev. Neurosci. 8, 657–661 (2007). [DOI] [PubMed] [Google Scholar]
  • 51.Nyberg L., et al. , Selective adult age differences in an age-invariant multifactor model of declarative memory. Psychol. Aging. 18, 149–160 (2003). [DOI] [PubMed] [Google Scholar]
  • 52.Hutto C.J., & Gilbert E.E., (2014). VADER: AParsimonious Rule-based Model for Sentiment Analysis of Social Media Text. Eighth International Conference on Weblogs and Social Media (ICWSM-14). Ann Arbor, MI, June 2014. [Google Scholar]
  • 53.Karpicke J. D., Roediger H. L., The critical importance of retrieval for learning. Science 319, 966–968 (2008). [DOI] [PubMed] [Google Scholar]
  • 54.Norman K. A., Newman E., Detre G., Polyn S., How inhibitory oscillations can train neural networks and punish competitors. Neural. Comput. 18, 1577–1610 (2006). [DOI] [PubMed] [Google Scholar]
  • 55.Marr D., Simple memory: A theory for archicortex. Philos. Trans. R. Soc. Lond. B Biol. Sci. 262, 23–81 (1971). [DOI] [PubMed] [Google Scholar]
  • 56.Yassa M. A., Stark C. E., Pattern separation in the hippocampus. Trends Neurosci. 34, 515–525 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Light L. L., Singh A., Implicit and explicit memory in young and older adults. J. Exp. Psychol. Learn. Mem. Cogn. 13, 531–541 (1987). [DOI] [PubMed] [Google Scholar]
  • 58.Hodges S., et al. , “SenseCam: A retrospective memory aid” in International Conference on Ubiquitous Computing (Springer, Berlin, Heidelberg, 2006), pp. 177–193. [Google Scholar]
  • 59.Woodberry E., et al. , The use of a wearable camera improves autobiographical memory in patients with Alzheimer’s disease. Memory 23, 340–349 (2015). [DOI] [PubMed] [Google Scholar]
  • 60.Chow T. E., Rissman J., Neurocognitive mechanisms of real-world autobiographical memory retrieval: Insights from studies using wearable camera technology. Ann. N. Y. Acad. Sci. 1396, 202–221 (2017). [DOI] [PubMed] [Google Scholar]
  • 61.Allé M., et al. , Wearable cameras are useful tools to investigate and remediate autobiographical memory impairment: A systematic PRISMA review. Neuropsychol. Rev. 27, 81–99 (2017). [DOI] [PubMed] [Google Scholar]
  • 62.Boot W. R., et al. , Video games as a means to reduce age-related cognitive decline: Attitudes, compliance, and effectiveness. Front. Psychol. 4 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Semon R., The Mneme (G. Allen & Unwin, 1921). [Google Scholar]
  • 64.Josselyn S., Köhler S., Frankland P. W., Finding the engram. Nat. Rev. Neurosci. 16, 521–534 (2015). [DOI] [PubMed] [Google Scholar]
  • 65.Herlitz A., Rehnman J., Sex differences in episodic memory. Curr. Dir. Psychol. Sci. 17, 52–56 (2008). [Google Scholar]
  • 66.Nasreddine Z. S., et al. , The montreal cognitive assessment, MoCA: A brief screening tool for mild cognitive impairment. J. Am. Geriatr. Soc. 53, 695–699 (2005). [DOI] [PubMed] [Google Scholar]
  • 67.Roediger H. L. III, Butler A. C., The critical role of retrieval practice in long-term retention. Trends Cogn. Sci. 15, 20–27 (2011). [DOI] [PubMed] [Google Scholar]
  • 68.Buchsbaum B. R., Lemire-Rodger S., Fang C., Hervé A., The neural basis of vivid memory is patterned on perception. J. Cogn. Neurosci. 24, 1867–1883 (2012). [DOI] [PubMed] [Google Scholar]
  • 69.Schapiro A. C., Kustner L. V., Turk-Browne N. B., Shaping of object representations in the human medial temporal lobe based on temporal regularities. Curr. Biol. Seoul, South Korea: Association of Computing Machinery; 22, 1622–1627 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Kyle C. T., Stokes J. D., Lieberman J. S., Hassan A. S., Ekstrom A. D., Successful retrieval of competing spatial environments in humans involves hippocampal pattern separation mechanisms. Elife 4, e10499 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Ritchey M., Montchal M. E., Yonelinas A. P., Ranganath C., Delay-dependent contributions of medial temporal lobe regions to episodic memory retrieval. Elife 4, e05025 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72.Kim M., Jeffery K. J., Maguire E. A., Multivoxel pattern analysis reveals 3D place information in the human hippocampus. J. Neurosci. 37, 4270–4279 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.Tompary A., Davachi L., Consolidation promotes the emergence of representational overlap in the hippocampus and medial prefrontal cortex. Neuron 96, 228–241 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Hulbert J. C., Norman K. A., Neural differentiation tracks improved recall of competing memories following interleaved study and retrieval practice. Cereb. Cortex 25, 3994–4008 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 75.Chanales A. J., Oza A., Favila S. E., Kuhl B. A., Overlap among spatial memories triggers repulsion of hippocampal representations. Curr. Biol. 27, 2307–2317 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 76.Dimsdale-Zucker H. R., Ritchey M., Ekstrom A. D., Yonelinas A. P., Ranganath C., CA1 and CA3 differentially support spontaneous retrieval of episodic contexts within human hippocampal subfields. Nat. Commun. 9, 1–8 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77.O’Reilly R. C., McClelland J. L., Hippocampal conjunctive encoding, storage, and recall: Avoiding a trade-off. Hippocampus 4, 661–682 (1994). [DOI] [PubMed] [Google Scholar]
  • 78.Hodges S., Berry E., Wood K., SenseCam: A wearable camera that stimulates and rehabilitates autobiographical memory. Memory 19, 685–696 (2011). [DOI] [PubMed] [Google Scholar]
  • 79.Lee M. L., Dey A. K., “Lifelogging memory appliance for people with episodic memory impairment” in Proceedings of the 10th International Conference on Ubiquitous Computing (2008), pp. 44–53.. [Google Scholar]
  • 80.Berry E., et al. , The use of a wearable camera, SenseCam, as a pictorial diary to improve autobiographical memory in a patient with limbic encephalitis: A preliminary report. Neuropsychol. Rehabil. 17, 582–601 (2007). [DOI] [PubMed] [Google Scholar]
  • 81.Berry E., et al. , The neural basis of effective memory therapy in a patient with limbic encephalitis. J. Neurol. Neurosurg. Psychiatry 80, 1202–1205 (2009). [DOI] [PubMed] [Google Scholar]
  • 82.Browne G., et al. , SenseCam improves memory for recent events and quality of life in a patient with memory retrieval difficulties. Memory 19, 713–722 (2011). [DOI] [PubMed] [Google Scholar]
  • 83.Pauly-Takacs K., Moulin C. J. A., Estlin E. J., SenseCam as a rehabilitation tool in a child with anterograde amnesia. Memory 19, 705–712 (2011). [DOI] [PubMed] [Google Scholar]
  • 84.St. Jacques P. L., Conway M. A., Cabeza R., Gender differences in autobiographical memory for everyday events: Retrieval elicited by SenseCam images versus verbal cues. Memory 19, 723–732 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 85.Crete-Nishihata M., et al. , Reconstructing the past: Personal memory technologies are not just personal and not just of rmemory. Hum. Comput. Interact. 27, 92–123 (2012). [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix 01 (PDF)

Data Availability Statement

Anonymized behavioral data have been deposited on the OSF (https://osf.io/7g2rx/; doi:10.17605/OSF.IO/7G2RX). fMRI data have been deposited to OpenNeuro (doi:10.18112/openneuro.ds004336.v1.0.0).


Articles from Proceedings of the National Academy of Sciences of the United States of America are provided here courtesy of National Academy of Sciences

RESOURCES