Linking the Rapid Cascade of Visuo-Attentional Processes to Successful Memory Encoding

Abstract

While it is broadly accepted that attention modulates memory, the contribution of specific rapid attentional processes to successful encoding is largely unknown. To investigate this issue, we leveraged the high temporal resolution of electroencephalographic recordings to directly link a cascade of visuo-attentional neural processes to successful encoding: namely (1) the N2pc (peaking ~200 ms), which reflects stimulus-specific attentional orienting and allocation, (2) the sustained posterior-contralateral negativity (post-N2pc), which has been associated with sustained visual processing, (3) the contralateral reduction in oscillatory alpha power (contralateral reduction in alpha > 200 ms), which has also been independently related to attentionally sustained visual processing. Each of these visuo-attentional processes was robustly predictive of successful encoding, and, moreover, each enhanced memory independently of the classic, longer-latency, conceptually related, difference-due-to memory (Dm) effect. Early latency midfrontal theta power also promoted successful encoding, with at least part of this influence being mediated by the later latency Dm effect. These findings markedly expand current knowledge by helping to elucidate the intimate relationship between attentional modulations of perceptual processing and effective encoding for later memory retrieval.

Introduction

Every day we are exposed to a wide array of stimuli in our sensory environment, but we successfully encode into memory relatively few of them. The stimuli we later remember are usually among the ones we attended to during encoding, but attention is not a single process. Rather, it is a complex set of operations, and we know very little about the neural mechanisms whereby each of these attentional operations influences successful encoding. The most powerful method to examine the neural mechanisms of successful long-term memory encoding is the “subsequent memory paradigm” (Paller et al. 1987; reviewed: Paller and Wagner 2002), in which stimulus responses at the study phase are classified as being later remembered or forgotten based on a subsequent memory test. In subsequent memory electroencephalographic (EEG) studies, the most investigated activity marker is a broadly distributed, long-latency (400–800 ms), positive-polarity ERP wave known as the “difference-due-to-memory” or Dm (Paller et al. 1987). Although the Dm can be modulated by some types of attention (e.g., attention to meaning), other attentional processes onset well before 400 ms, and we know little about how they contribute to later memory. The current study takes advantage of the high temporal resolution of EEG in a modified visual search task to examine the impact these rapid visuo-attentional effects have on encoding processes and later memory retrieval.

Visual search tasks require identifying a target item amongst distractors. These types of tasks evoke a cascade of EEG components that are modulated by attentional focus and demands, including the N2pc and the contralateral reduction in alpha power (CRA, 8–12 Hz). (1) The N2pc (negative 200 ms posterior-contralateral; Luck and Hillyard 1994) has been related to stimulus-specific attentional allocation (reviewed: Kiss et al. 2008) and capture (Qi et al. 2013). (2) Following the N2pc, the sustained posterior-contralateral negativity (SPCN; >300 ms) has been linked to sustained attentional processing for item-feature discrimination (Jolicoeur et al. 2008), and is generally larger when the target-discrimination task is more difficult. (3) Finally, temporally co-occurring with the SPCN (>300 ms), there is a poststimulus reduction in oscillatory alpha power contralateral to the target, which we will refer to here as the CRA. As with most other decreases of alpha, this effect is indicative of increased cortical activity (e.g., Laufs et al. 2003). The CRA has also been linked to longer-latency item-analysis processes (van Diepen et al. 2016), which are cognitively distinct from the SPCN (Fukuda et al. 2015; Bae and Luck 2017; De Vries et al. 2017). The onsets of these three EEG components, the N2pc, the SPCN, and the CRA, occur well before the Dm effect; how they modulate subsequent memory, however, is unknown.

Given that these three components have been associated with enhanced perceptual processes, we hypothesized that the early attentional processes these components index would predict subsequent memory (Hypothesis 1). The relation between these components and the Dm, however, is less clear. The Dm is typically found during deep encoding tasks and an abundance of evidence suggests that this component is related to conceptual processing of the stimuli (reviewed: Paller and Wagner 2002). On the one hand, enhanced early perceptual processes may lead to enhanced subsequent memory by boosting later conceptual-level processing. If so, the early attentional processes indexed by the N2pc, SPCN, and CRA should contribute to the Dm (Hypothesis 2a). On the other hand, perceptual processes could enhance later memory success in parallel, relatively independently of conceptual processing. Accordingly, the early attentional processes indexed by N2pc, SPCN, and CRA may not contribute to the Dm (Hypothesis 2b). Finally, recent evidence has linked successful encoding to an increase in midfrontal theta power, both pre- and post-stimulus (reviewed: Hsieh and Ranganath 2014), potentially reflecting interactions between prefrontal cortex and the hippocampus that promote later memory (Backus et al. 2016). We hypothesized that these interactions might enhance later conceptual processing such that midfrontal theta would be associated with the longer latency Dm effect, and thereby to memory encoding (Hypothesis 3).

Materials and Methods

Participants

Forty-two subjects participated in the study. Six of these were excluded due to low memory performance (d′ < 0.35 at memory retrieval for both all responses and high-confidence responses), five for excessive lateral eye-movements (see preprocessing), three for poor data quality, and one for postexperimental disclosure of Tourette’s syndrome. Thus, there were 27 subjects included in the final analysis (age = 22.1 (±2.7) years old [15 males, 12 females]). All subjects were right handed and had normal or corrected-to-normal vision, and all gave written informed consent. The study was conducted in accordance with a protocol approved by the Duke Institutional Review Board. Participants were paid $15/h or received class credit from the Duke Psychology Department for their participation in the study.

Paradigm

Participants engaged in a modified visual-search style paradigm in a memory encoding phase, followed by a retrieval test phase (Fig. 1). During the encoding and retrieval phases, participants were seated approximately 60 cm from a 24-inch LCD monitor and the stimuli were presented using the Presentation software package (Neurobehavioral Systems).

Figure 1.

Examples of encoding and retrieval trials. (A) For the encoding phase of the experimental session, subjects were instructed to covertly shift their attention to the lateralized object circumscribed by a ring of a designated color (e.g., pink, counterbalanced across subjects) and to determine if this target object was known or unknown. Known objects could be identified and named, whereas unknown objects could not be. (B) During the retrieval phase of the session, subjects were presented with an old or new object in the midline (just below fixation) and asked to determine if the object was definitely old, probably old, probably new, or definitely new. All the electrophysiological data reported here are from the encoding phase (A).

During the encoding phase (Fig. 1A), participants were briefly presented with two ring-circumscribed objects (each object’s size ~3.2°; each ring size ~4.5°), one to the left and one to the right of fixation (object center 2.6° to each side). One of the objects was circumscribed in a gold ring (218, 165, 32) and the other in a pink ring (255, 51, 255), with these colors being chosen due to their minimal overlap with the object colors. Participants were instructed to covertly shift their attention (i.e., while maintaining fixation) to the object (“target object”) that was indicated by the target-designating color of the surrounding ring (e.g., circumscribed in pink), and to indicate—as quickly as possible—whether the object was known or unknown. The target color was fixed for each subject for the entire encoding phase, counterbalanced across subjects. “Known objects” (80% of items) were everyday objects participants could identify and name (e.g., chair), whereas “unknown objects” (20% of items) were rare objects participants could not generally identify or name (e.g., a glass insulator). Before encoding, participants practiced the encoding task to ensure they understood the instructions and were able to minimize eye movements, which were tracked with a bipolar horizontal electrooculogram (HEOG). The objects were presented for a duration of 400 ms and then followed by an intertrial interval (ITI) of 1600–2000 ms (uniform distribution), during which the fixation cross remained on the screen. In total, 640 image-pair trials were presented across 16 blocks, consisting of 512 (80%) familiar object targets and 128 (20%) unknown object targets.

The encoding trials and the retrieval trials were presented in completed separated phases of the session, with all the retrieval trials (Fig. 1B) occurring after all of the encoding trials had been completed (Fig. 1A). In each retrieval trial, a single object was presented in the midline just beneath fixation for 1000 ms, and participants indicated whether the object was definitely old, probably old, probably new, or definitely new. The ITI during retrieval was 2100–2500 ms (uniform distribution) and 768 trials were presented across 16 blocks, consisting of 512 (67%) old and 256 (33%) new items.

This visual-search task was designed to be a deep-encoding task that forced subjects to conceptually identify the target stimulus (e.g., dog), with unknown items being included to serve as “catch” items to ensure that the subjects were engaged in the task. As such, during the retrieval phase that followed, when memory was tested (Fig. 1B), we only accessed memory for known items that were targets and were correctly identified as being known during the encoding phase, to be sure that these items were attentionally attended to, as well as accurately identified conceptually. All electrophysiological data reported here are from the encoding phase.

E‌EG Processing

EEG was DC-recorded (500 Hz sampling rate—three-stage cascaded integrator low-pass comb filter with a corner frequency of 130 Hz), using a 64-channel, custom-layout, equidistant, extended-coverage electrode cap (Woldorff et al. 2002), and a Brainproducts Actichamp amplifier with active electrodes (Acticap). The data were recorded referenced to the right mastoid. Data were also filtered offline using a 0.05 Hz highpass causal FIR filter and a 70 Hz low-pass filter. Data were then downsampled to 250 Hz in conjunction with application of a corresponding antialiasing filter. Independent component analysis (ICA) was used to correct for eyeblinks, which were extracted using the extended infomax algorithm as implemented in EEGlab (using pop_runica, EEGlab13.4.4b; Delorme and Makeig 2004). Independent components (ICs) that reflected eyeblinks (1 or 2 ICs per participant) were removed from the data. Finally, trials were epoched [−1000, 1800 ms], rereferenced to the algebraically averaged mastoids, and baselined from −200 to 0 ms. Residual eye blinks were detected [−500, 1000 ms] using a step response test [−100, 100 uV], with trials that surpassed this threshold being rejected.

Lateral eye movements were detected with a unit step function convolutional filter (full length = 200 ms) applied to the HEOG lateral eye electrodes left-minus-right difference wave, from 0 to 500 ms. Trials with eye movements greater than 1.5° (corresponding to ~24 uV; Luck 2014) during object presentation were flagged for rejection due to concern that they might contaminate the N2pc analysis. Subjects with fewer than 200 remaining trials (total across all conditions) after artifact rejection were removed from the analysis (N = 5).

To optimally examine the posteriorly distributed, attention-related ERPs, a current-source-density (CSD) analysis was applied, which entails calculating the second-spatial derivative of the EEG for each trial at each time point (ft_scalpcurrentdensity (default usage → spherical spline); Oostenveld et al. 2011). This processing, which derives the current sources and sinks at the scalp/skull, emphasizes the nearer sources, which has an advantage when examining components such as the posteriorly distributed N2pc when there could be some residual eye-movement artifact, as the CSD derivation makes it much less likely for activity from such frontal sources to propagate to the posterior channels measuring the N2pc.

Frequency information was computed from the CSD-transformed data, after the average ERP for each condition was subtracted from each trial. Frequency decomposition was performed on the EEG data using multitaper methods as implemented in the analysis software package Fieldtrip (Oostenveld et al. 2011), in which discrete prolate slepian sequences were used to estimate the power in logarithmically spaced frequencies from 2 to 30 Hz. The window widths for the tapers were 2 cycles for 2–4 Hz, 3 cycles for 4–7 Hz, 5 cycles for 8–14 Hz, 7 cycles for 15–20 Hz, and 10 cycles for 21–30 Hz. Smoothing by means of multitapers was specified as 5 × log₁₀ of each frequency. Log₁₀-transformed, time-locked, power spectra for each subject were subsequently binned and averaged according to the various conditions. No baseline normalization was performed.

The EEG cap utilized a slightly modified 10–10, equidistant-electrode montage with extended inferior occipital coverage (Woldorff et al. 2002). When reporting electrode sites, we report standard 10–10 electrode names if the electrode in our montage is within 5 mm of the corresponding 10–10 site. For those electrodes further than this (5–10 mm), the name had a prime appended to it (e.g., O1′). No electrodes were further than about 1 cm from their closest standard 10–10 site.

E‌EG Statistical Analyses

Most of the ERP and oscillatory components under investigation in this study have been well characterized in the attention and memory literature. This prior knowledge was used to guide our statistical analysis. For all statistical tests of functional contrasts, except for the Dm, electrode clusters were chosen via visual inspection of waveforms generated from the trials collapsed across all conditions, within the context of previous literature. For the Dm, we selected a cluster centered on electrode Cz (Paller and Wagner 2002). When determining temporal windows to test for significance, we generally relied upon the waveforms generated from all trials. For the N2pc, we identified the peak deflection of the grand average across all conditions and centered a 100 ms window to test for modulation based upon subsequent memory. For the CRA, we did the same, but utilized a window of 500 ms to account for its longer duration and latency. For the SPCN, we tested a window following the N2pc and lasting until the SPCN returned to baseline (i.e., until there was no difference between activity over the contralateral and ipsilateral hemispheres, again collapsed across all conditions). For the Dm, we relied upon prior literature, selecting an epoch between 400 and 800 ms. Mnemonic theta-related effects have been reported both pre- and post-stimulus (reviewed: Hsieh and Ranganath 2014), and thus we examined three temporal epochs [prestimulus: −500 to 0 ms; early: 0–500 ms; and late 500–1000 ms] while using a Bonferonni correction for multiple comparisons P < 0.017. When plotting ERP timecourses, a 20 Hz, butterworth fourth-order filter was applied to the grand average; no statistics, however, were run on this smoothed data. When reporting numerical results, we provide the mean and standard deviation (in parentheses), in addition to t-values (from paired t-tests), P-values, temporal epochs, and Cohen d values.

In addition to these a priori defined tests, we also performed running t-tests, for all memory contrasts, in order to identify significant time epochs. For this purpose, the ERP data were segmented into 50-ms bins, and t-tests were run on each bin with a critical threshold of P < 0.05. For both the ERP and oscillatory data, statistically significant sequences of at least three of such 50-ms time bins were considered meaningful and are marked in gray on all plots. As expected, these analyses corroborated our a priori planned statistical test. In addition to this, all plots of difference waves include a shaded margin around the trace to indicate the standard error of the difference wave. This was included partiularly for the difference waves, as all the statistical tests were within-subject repeated-measures, and thus the variance of the difference wave was the relevant variance measure for the statistical test conducted.

A mediation model was run with the toolbox from Tor Wager and colleagues (Wager et al. 2008; Wager et al. 2009). This trial-to-trial, subject-wise multilevel model included all remembered and forgotten trials. In general, mediation models rely upon a regression framework to determine how two correlated variables (e.g., A, B) contribute to a common outcome (C), when they both predict that outcome. If there is no mediation, then A and B are said to independently contribute to C. If there is a mediation, it suggests that A predicts B (or B predicts A), which then predicts C. In our study, A occurred temporally prior to B and thus testing the alternative path (B predicts A) was nonsensical. This mediation link is known as an indirect path and suggests that part of the way in which A predicts C is determined by its impact on B.

The main analyses run focused on how specific EEG markers related to subsequent memory. However, it is possible that our EEG markers of interest (N2pc, SPCN, CRA, and midfrontal alpha) could have been influenced by other factors as well. Potential factors of noninterest include reaction time during encoding (RT), perceived luminance (L^*) of the target stimulus based upon the CIELAB standard (where L^* = 0 is black and L^* = 100 is white), trial number, and animacy (living vs. nonliving). There are multiple highly similar ways to compute L^* based upon the CIELAB standard, code pertaining to the exact transformation used here is available on Github (https://github.com/brg015/image_pro_v1/blob/master/image_pro_v1/GetLumValues.m).

To ensure that these potential confounds (RT, L^*, trial number, animacy) were not driving apparent subsequent memory-related results, additional trial-wise regression models were run with reference to the EEG effects of interest (e.g., N2pc amplitude ~ subsequent memory [0,1] + 1/RT + L^*[0:100] + trial number [1:640] + nonliving [0,1]). These models were run within participant, with the resulting beta-values being entered into subsequent t-tests. If the subsequent memory parameter were still significant, it would suggest that subsequent memory was still related to the EEG parameter of interest, even after accounting for these factors of noninterest.

Results

Behavioral

Subjects’ behavioral performance is summarized in Table 1. Retrieval performance had a mean d′ of 0.82 (±0.30) across all trials (Cohen’s d = 2.76). Response times (RTs) during encoding did not differ for subsequently remembered and forgotten items (P = 0.67), but during retrieval, they were slower for forgotten than remembered items [t(26) = −3.29, P = 0029, Cohen’s d = −0.63]. To maximize trial counts for the electrophysiological results at encoding, the data were collapsed across high and low confidence for the later memory test responses. This resulted in subjects having at encoding an average of 221 (±55) later remembered trials and 120 (±36) later forgotten trials. Subjects also had significantly slower RTs for known versus unknown items at encoding [t(26) = −10.52, P = 7.3e-11, Cohen’s d = −2.02] and worse performance [t(26) = −3.99, P = 0.00047, Cohen’s d = −0.77] when responding to catch trials (unknown objects) during encoding (i.e., when they incorrectly responded that unknown items were known), as would be expected given the rarity of the catch items. These behavioral differences between known and unknown objects did not affect the EEG results, which were analyzed only for known objects.

Table 1.

Summary of behavioral results

	Remembered	Forgotten
All trials (%)	65 (8)	35 (8)
Encoding RT	690 (106)	690 (110)
Retrieval RT	905 (123)	980 (164)

Notes: Note that remembered and forgotten items encompassed all responses to known objects only. Reported values are means with standard deviations in parentheses.

Long-latency Subsequent Memory ERP effect: Dm

As noted in the Introduction, the Dm effect is a long-latency (400–800 ms) positivity over central scalp regions elicited at encoding that is larger for subsequently remembered than for subsequently forgotten items (Paller and Wagner 2002). Consistent with this literature, during a 300–850 ms window, we found a robust Dm [t(26) = 3.05, P = 0.0052, {400–800 ms}, Cohen’s d = 0.59] at the vertex and surrounding electrodes (Cz, FCz, CPz, FC1, FC2, C1a, C2a) (Fig. 2).

Figure 2.

Dm effect in central electrodes during encoding for later remembered minus later forgotten items. Running t-tests identified an epoch from 300 to 850 ms (shaded gray), where subsequently remembered items had a greater positivity than subsequently forgotten items.

Lateralized ERP Attention Effects at Encoding: N2pc and SPCN

Lateralized ERP attention effects at encoding were computed by directly subtracting the ERPs at the contralateral channels (relative to the relevant target object) minus the ERPs at the corresponding ipsilateral channels (contralateral minus ipsilateral, i.e., c − i). This operation effectively provides twice the number of trial counts, as left target responses (right-hemisphere electrodes minus left-hemisphere responses) are combined with right target responses (left hemisphere electrodes minus right-hemisphere electrodes) and is very commonly utilized when computing the lateralized attention-related N2pc and SPCN components. A visual analysis of all trials indicated that the N2pc peaked at approximately 260 ms (Fig. 3; black electrodes: O1′, P3i). This activity peak at encoding, analyzed from 210 to 310 ms (±50 ms around the peak), showed a significantly larger amplitude for later remembered than for later forgotten trials [t(26) = −3.39, P = 0.0022, Cohen’s d = −0.65].

Figure 3.

Both the N2pc and the SPCN were larger for subsequently remembered than forgotten items. Running t-tests identified an epoch from 200 to 800 ms (shaded gray), where subsequently remembered items had a greater contralateral negativity than subsequently forgotten items.

Following the N2pc, we identified a longer-latency sustained negativity from 400 to 800 ms reflecting an SPCN-like component, which was also significantly modulated as a function of later remembered minus later forgotten [t(26) = −3.25, P = 0.0032, Cohen’s d = 0.63]. We set the end duration of this post-N2pc lateralized negativity to be at the time point at which this activity collapsed across all known trial types all known-object trials no longer showed a contralateral versus ipsilateral difference. It is also important to note that this negativity was not clearly separated from the N2pc in time, with only a small dip in magnitude between the N2pc and this sustained negativity (i.e., the N2pc negativity here seemed to progress directly into the SPCN negativity). However, this sustained negativity extended far beyond what would be considered an N2pc, which is thought to reflect the initial orienting of attention toward a visual target in a visual search paradigm. Rather, its extended duration seems likely to reflect the continued processing of the lateralized target stimulus, similar to the classic SPCN. Accordingly, we will refer to this long-latency, sustained negativity effect as the SPCN, while also providing alternative perspectives in the discussion.

Oscillatory Contralateral Reduction in Alpha Power [8–12 Hz]

A robust CRA effect at encoding was identified for all known-object items (Fig. 4; black electrodes: O1′, P3i, TO1), peaking at approximately 500 ms. As predicted, the magnitude of this effect varied as a function of subsequent memory [t(26) = −2.76, P = 0.011, {250–750 ms}, Cohen’s d = 0.53]. Running t-tests identified an early epoch (200–500 ms) and a later epoch (700–1000 ms) where decreased contralateral alpha power was associated with better memory.

Figure 4.

Contralateral reduction in alpha (CRA), reflecting increased cortical activity, was prominent both early in the trial (200–500 ms) and later in the trial (700–1000 ms). The electrodes denoted by black diamonds were chosen based upon contralateral-minus-ipsilateral (c − i) contrast for all real items—later remembered minus later forgotten contrast were derived from this region of interest.

Oscillatory Theta Power [4–7 Hz]

As illustrated by Figure 5, theta power over midfrontal (FCz, CPz, FC1, FC2, C1a, C2a) electrodes (midfrontal theta) was significantly higher for later remembered than for later forgotten items in all three analyzed time periods: prestimulus [t(26) = 3.47, P = 0.0018, {−500 to 0 ms}, Cohen’s d = 0.67], early [t(26) = 3.45, P = 0.0020, {0–500 ms}, Cohen’s d = 0.66], and late [t(26) = 3.35, P = 0.0025, {500–1000 ms}, Cohen’s d = 0.64].

Figure 5.

Midfrontal theta was greater for subsequently remembered than forgotten items (all P < 0.05). The electrodes denoted by black diamonds were chosen based upon theta variations for all real items—later remembered minus later forgotten contrast were derived from this region of interest.

Regression Analysis

Even after accounting for parameters of noninterest, the memory results were qualitatively the same and still statistically significant (Table 2). A few of the parameters of noninterest also showed relationships to EEG markers under examination. Of ancillary note, the N2pc was larger for faster reaction times (1/RT), for nonliving items, and for items with larger perceived luminance (L^*).

Table 2.

Demonstrates the results of a multiple regression analysis in which variables of noninterest were additionally accounted for

Measure	Time	Memory			1/RT			L *			Trial#			Nonliving
		t-Value		d	t-Value		d	t-Value		d	t-Value		d	t-Value		d
N2pc	[210–310 ms]	−2.66	*	−0.5	−4.64	* * *	−0.9	−4.23	* * *	−0.8	0.76		0.1	−2.64	*	−0.5
SPCN	[400–800 ms]	−3.25	* *	−0.6	−0.57		−0.1	−1.19		−0.2	0.67		0.1	−1.68		−0.3
Dm	[600–800 ms]	2.23	*	0.4	1.51		0.3	0.78		0.1	3.00	* *	0.6	−1.09		−0.2
CRA	[250–750 ms]	−2.18	*	−0.4	−0.63		−0.1	−1.09		−0.2	0.90		0.2	1.36		0.3
Fr. Theta	[−500 to 0 ms]	3.30	* *	0.6	−1.25		−0.2	−1.22		−0.2	3.03	* *	0.6	−0.36		−0.1
Fr. Theta	[0–500 ms]	3.47	* *	0.7	1.60		0.3	−1.93	#	−0.4	1.59		0.3	−0.44		−0.1
Fr. Theta	[500–1000 ms]	3.25	* *	0.6	−5.12	* * *	−1.0	0.03		0.0	2.61	*	0.5	0.97		0.2

Notes: Of note, all the memory related results were significant, even after accounting for these potential factors. The table reports both t-values and Cohen’s d values (d). #P < 0.10, ^*P < 0.05, ^**P < 0.01, ^***P < 0.001.

Independent and Dependent Contributions to Memory

In sum, our results linked subsequent memory not only to the traditional Dm effect, but also to several rapid-onset, more spatially specific, attention-related EEG components, including the N2pc and CRA. We also identified a sustained negativity following the N2pc that also predicted subsequent memory and appears temporally and spatially similar to an SPCN. Finally, we also found that midfrontal theta was greater for subsequently remembered than forgotten trials, even before the trial began. Having successfully identified this set of effects, we next examined interactions among them, based upon what we directly observed (i.e., not a priori hypothesized), in order to determine if their contributions to memory were dependent or independent of each other. Dependent processes should be correlated with each other, whereas independent processes would be expected to be uncorrelated. We computed, for each subject, the Spearman correlation across trials, between the following eight EEG measures: N2pc (210–310 ms), the SPCN (400–800 ms), the early (200–500 ms) and late (700–1000 ms) CRA, the superior-central positivity underlying the Dm (300–850 ms), and the midfrontal theta in the three analyzed epochs of prestimulus (−500 to 0 ms), early (0–500 ms), and late (500–1000 ms). For each of these eight EEG measures, the trial-wise value included in the correlation analysis was based upon the average magnitude within the indicated temporal epoch. For these analyses we utilized a Bonferonni correction for multiple comparisons (N = (8²—8)/2 = 28, critical P < 0.05/28 = 0.0018).

The correlations yielded three key findings. First, neither the N2pc, the subsequent sustained SPCN-like negativity, nor the CRA was correlated with the-central positivity of the Dm, suggesting they contribute independently to successful encoding. Second, in marked contrast to the N2pc, the SPCN and the CRA, early theta [0–500 ms] was strongly correlated with the Dm [atanh(r) = 0.074 (0.056), t(26) = 6.83, P = 2.97e−7, Cohen’s d = 1.31], suggesting the former contributes to the latter. Furthermore, the magnitude of this effect (Dm to early theta) was far stronger than any possible relationships between other conceptually related processing activity (Dm, theta) and attention-related processing activity (SPCN, N2pc, CRA; all P < 0.0001). Third, there was no significant correlation between the CRA and SPCN, further suggesting that although these processes overlap in time and both seem to support sustained visual processing, they do so in different ways. Additionally, we found correlations between the similarly distributed N2pc and SPCN, as well as between the three theta epochs (all P < 0.0018); these correlations would seem unsurprising, however, given the spatial correspondence and temporal adjacency of these effects.

To clarify the significant correlation between the midfrontal theta and the superior-central positivity of the Dm, we ran an exploratory multilevel mediation model (Wager et al. 2008; Wager et al. 2009). In this analysis, we tested whether the magnitude of the superior-central positivity underlying the Dm (600–800 ms) mediated the relationship between early theta power (125–225 ms) and subsequent memory. These time windows were chosen such that they were completely nonoverlapping (given that the theta kernel is 750 ms) and within the bounds in which a significant correlation was identified: theta (0–500 ms) and superior-central positivity associated with the Dm (300–850 ms). The results of this model showed a partial mediation (a^*b; P = 0.0031, Cohen’s d = 0.22, proportion mediated = 7%; Fig. 6), suggesting that the influence of the theta-indexed process on subsequent memory is driven in part by its relationship with the superior-central positivity underlying the Dm.

Figure 6.

A multilevel mediation model showed that superior-central positivity of the Dm effect partially mediated the relationship between pre-Dm theta power and subsequent memory. Reported values are means, with standard errors in parentheses. The variables {a, b, c} and their associated significance indicate that all components in the model {memory, theta, Dm} are pairwise related - as would be expected given the prior correlation analysis. The italized variables {a*b, c'} and their associated significance provide evidence for a mediation effect.

Discussion

The contribution of rapid visuo-attentional processes to successful memory encoding is largely unknown, as most paradigms and analyses have focused solely on longer-latency mechanisms, such as the role of conceptual processing as indexed by the Dm. By utilizing a novel spatial-focusing paradigm, we investigated the mnemonic contributions of a rapid cascade of visuo-attentional processes indexed by the N2pc (attentional orienting and allocation), SPCN (item-feature identification), and CRA (item-analysis processes), as well as their relationship to longer-latency conceptual processing as indexed by the Dm. We additionally examined the role of the level of midfrontal theta for subsequent memory and, in an intermediating way, the Dm effect. The study yielded three main findings: (1) Each of the EEG markers of the rapid-onset attentional modulations of visual processing—the N2pc, SPCN, and CRA—predicted better subsequent memory; (2) the N2pc, SPCN, and CRA visual-attention effects led to enhanced memory independently of the Dm; and (3) finally, midfrontal theta power predicted both the Dm effect and subsequent memory. These findings are summarized in Figure 7.

Figure 7.

EEG effects enhancing subsequent memory. The schematic figure displays a set of parallel processes that all serve to independently contribute to successful memory encoding including attentional orienting and allocation as indexed by the N2pc ERP (green), sustained item-specific visual processing as indexed by the SPCN ERP (orange), sustained visual processing as indexed by oscillatory alpha-band CRA (yellow; CRA), and preparatory cognitive-control processing followed by conceptual processing, indexed by theta-band EEG and the Dm (purple).

Rapid-Onset Attentional Processes as Indexed by the N2pc, SPCN, and CRA Predict Subsequent Memory

Consistent with our first hypothesis, we found that markers of early attention processes, including the N2pc, SPCN, and CRA, all predicted subsequent memory, and all did so independently of the longer latency Dm. We discuss our interpretation of each of these effects below.

N2pc

The cascade of attentional processes that are being tracked here and facilitate memory encoding are initiated with attentional allocation toward the target item that is both spatially and stimulus specific. This shift is indexed by the attention-sensitive N2pc (starting ~210 ms), which was larger for subsequently remembered items. One potential explanation for why the magnitude of the N2pc predicted subsequent memory is that the strength of the focusing of attention toward the relevant target stimulus will naturally fluctuate from trial to trial, due to factors such as randomly varying levels of task focus. That is, trials with strong attentional orienting and focusing, as indexed by the elicitation of larger N2pc’s, lead to stronger perceptual processing and thus stronger encoding of the stimulus, and thereby to better subsequent memory. Another potential explanation relies upon attentional bias, as larger N2pcs have also been found for more rewarding stimuli (Donohue et al. 2013) and more fear-provoking stimuli (Eimer and Kiss 2007) as compared with neutral stimuli. In the current experiment, in which random objects were presented to the participants, it is possible that they more strongly oriented to stimuli that were both targets and personally meaningful or salient—leading to an increased N2pc and enhanced subsequent memory.

SPCN

The SPCN sustained negativity follows the N2pc, typically showing a similar scalp distribution, and tends to be larger for more difficult-to-discriminate stimuli (Jolicoeur et al. 2008). As hypothesized, we observed a sustained-negativity attentional modulation following the N2pc, which also predicted subsequent memory, suggesting that stimuli that were more difficult to discriminate, or might otherwise invoke more extended discrimination processes, might be more strongly encoded. Another potential explanation, however, might relate to a spatially similar slow-wave component, the contralateral delay activity (CDA). This component has been associated with increased working-memory load (Luria et al. 2016) in that it gradually increases in amplitude (with load) and is sustained until the information in working-memory is discarded. As such, another interpretation is that the SPCN effect reflects extended manipulation of the item in working-memory, which is also known to increase long-term memory retrieval (Khader et al. 2007).

CRA

Another marker of increased, spatially specific, visual processing that has been reported more recently is a contralateral reduction in alpha power (CRA), which we also found to be linked to improved subsequent memory. In general, reductions in alpha activity have been associated with increased cortical activity (Laufs et al. 2003). While proactive lateralized alpha in a spatial-attention cueing task has been well characterized (Worden et al. 2000), and even dissociated from lateralized slow-wave ERPs (Grent-‘T-Jong et al. 2011), the function of lateralized reactively triggered alpha, and its relationship to ERPs, is less clear. Analogous to proactive lateralized processes, preliminary evidence suggests that reactively triggered alpha decreases (CRA) and lateralized slow-wave ERPs (SPCN), although both are related to sustained visual processing, serve independent functions (Fukuda et al. 2015; Bae and Luck 2017; De Vries et al. 2017), consistent with our finding that these measures were uncorrelated. With respect to visual search, one recent interpretation is that the CRA serves to “protect the sensory processing of stimuli” (van Diepen et al., 2016) from other distracting stimuli, and thus the early latency CRA observed here might reflect this type of protection, which then contributes to better encoding. The CRA might also reflect working-memory-related processes, as the CRA magnitude has also been found to be modulated as a function of working-memory load (Fukuda et al. 2015) and thus the enhanced CRA observed here might reflect working-memory-related processes that contribute to better encoding.

Early Attention Effects and Working-Memory

Our modified visual-search task was designed to examine the interactions between rapid attentional processes and long-term memory. As such, our interpretation of our results was guided by the premise of examining modulations of well-established visual-search-related markers, in particular the N2pc, SPCN, and CRA. However, as we also note, modulations in the SPCN and CRA can also be interpreted as reflecting working-memory-related processes. Indeed, various researchers tend to view the SPCN and CDA as reflecting the same process, or at least closely related ones (Cosman et al. 2016; van den Berg et al. 2016), as sustained attentional processing for item-feature discrimination (marked by the SPCN) can be viewed as corresponding to extended working-memory processes. Moreover, links between working-memory and long-term memory have been discussed for decades (reviewed: Hartshorne and Makovski 2019), and it is possible that the observed modulations of the SPCN and CRA are indicative of working-memory-related processes, which then proceed to facilitate long-term memory formation. Nevertheless, although our results could also be interpreted in a working-memory framework, we believe they are best interpreted in an attention-related framework. The primary reason for this view is that the visual-search task was specifically designed to manipulate attentional demands, rather than in manipulating working-memory processes. Moreover, probably the best marker we have of working-memory-related processes in this task is the reaction time, given that it serves as a surrogate measure of how long individuals considered an item in working-memory prior to responding. However, reaction time (see Table 2) was not found to correlate with the amplitude of either the SPCN or the CRA, as would be expected if these signals were reflective of working-memory related processes. Reaction times also did not differ between subsequently remembered and subsequently forgotten items, again emphasizing that the time participants held the item in WM did not have an impact on later long-term memory success. While we do not rule out a role of working-memory with respect to our results, we believe the evidence accrued mainly reflects a more attention-related account.

The Early Attentional Processes Indexed by N2pc, SPCN, and CRA Did Not Contribute to the Dm

Having found that the rapid cascade of early attentional effects indexed by the N2pc, SPCN, and CRA predicted subsequent memory, we examined whether any of these early attention effects correlated with the hallmark long-latency Dm effect. Consistent with Hypothesis 2b, none of these early attention effects correlated with the Dm, suggesting that their contributions to subsequent memory occur in parallel to or addition to the Dm, and, moreover, such contributions are not mediated by the Dm. Given that the Dm has been linked to conceptual processing, whereas these early attention effects have been associated primarily with effects on perceptual processes, the most parsimonious account is that perceptual and conceptual processes, including their modulation by attention, can make independent contributions to subsequent memory performance in a recognition memory test.

The Magnitude of the Midfrontal Theta was Associated with both the Dm Effect and Subsequent Memory

Finally, consistent with our third hypothesis, we found that the magnitude of early-latency poststimulus midfrontal theta (125–225 ms) predicted not only subsequent memory performance, but that its positive relationship with subsequent memory was partially mediated by the long-latency Dm effect (600–800 ms). In addition, we found that elevated theta power, both pre- and post-stimulus was predictive of better subsequent memory. The finding that elevated theta power pre- and post-stimulus predicted subsequent memory suggests that theta’s contribution to subsequent memory might arise via multiple mechanisms.

The enhancement of prestimulus theta power has been previously linked to subsequent memory (Guderian et al. 2009), particularly during intentional encoding (Schneider and Rose 2016), with intracranial recordings suggesting this type of prestimulus activity may be driven by the hippocampus (Fell et al. 2011). Prestimulus theta power has also been linked to preparatory top-down control (Min and Park 2010; van Noordt et al. 2017), which might contribute to successful encoding as well. Accordingly, we suggest that elevated prestimulus theta power reflects preparatory cognitive control processes that aid the item encoding and/or is reflective of the facilitation of more basic mnemonic processes.

The relationship between poststimulus frontal midline theta and subsequent memory was found to be partially mediated by the longer-latency Dm effect. Poststimulus theta power has previously been linked to working-memory and memory-encoding processes (reviewed: Hsieh and Ranganath 2014), and it has also been linked to cognitive control in tasks where there is need for top-down control (reviewed: Cavanagh and Frank 2014).

If elevated midfrontal theta power (poststimulus) is related to working-memory processes that mediate long-term memory formation, we had expected that midfrontal theta power would be higher when reaction times were longer; however, this was not the case in the present study. In fact, it appears as if longer reaction times (see Table 2) resulted in less midfrontal theta power poststimulus [500–1000 ms]—arguing against a working-memory interpretation.

With regards to cognitive control processes, our task was rather difficult; objects were only presented for 300 ms, and unknown items were, on average, successfully identified only 62% of the time, with known items being correctly identified 84% of the time. Thus, early-latency top-down cognitive control was likely essential to effectively performing the object-discrimination task. Moreover, the fact that faster reaction times at encoding were linked to greater theta power poststimulus (see Table 2), suggests that this elevated theta power was associated with more efficient item-related processing, as might be expected if it was reflective of enhanced control-related processes. As such, our interpretation of poststimulus theta effect, in relation to the partial mediation of the Dm, is that poststimulus cognitive control mechanisms causally contribute to longer-latency conceptual processing, which in turn leads to stronger/better encoding.

With regards to memory, theta also had a direct effect (c′) in the mediation model, suggesting a primary role as well. While the neuroanatomical geometry of the hippocampus renders its activity largely undetectable with scalp EEG (Nunez and Srinivasan 2006), a few studies have reported that increased midfrontal theta power is associated with prefrontal cortex and hippocampal connectivity, which is known to facilitate encoding (Garrido et al. 2015; Backus et al. 2016). Accordingly, the direct effect might be reflective of these types of neural interactions following stimulus presentation.

Conclusions

In sum, our results indicate that, when processing a stimulus or event in our environment, a cascade of rapid-onset attentional processes operate in concert that can facilitate successful memory encoding of those stimuli, mostly independently of, and on-setting earlier than, conceptual-level processing that has been indexed by the longer-latency Dm effect. This attentional cascade is initiated by increased stimulus-specific attentional orienting and allocation (N2pc) and is followed by sustained visual processing (SPCN and CRA) likely reflecting item-specific discrimination processes and/or working-memory processes that also contribute to successful memory encoding. In parallel to these visuo-attentional processes, increased preparatory cognitive control processes, as indexed by early midline frontal theta, appears to lead to enhanced longer-latency conceptual processing (Dm). These parallel item-focused visuo-attentional processes and enhanced conceptual processes all appear to then serve to strengthen memory encoding and lead to better subsequent memory retrieval.

Funding

National Institutes of Health (F31-MH114454 to B.R.G. and R01/R56-NS051048 to M.G.W.).

Notes

None.

Disclaimers

None.