Does rehearsal matter? Left anterior temporal alpha and theta band changes correlate with the beneficial effects of rehearsal on working memory.

Abstract

Rehearsal during working memory (WM) maintenance is assumed to facilitate retrieval. Less is known about how rehearsal modulates WM delay activity. In the present study, 44 participants completed a Sternberg Task with either intact novel scenes or phase-scrambled scenes, which had similar color and spatial frequency but lacked semantic content. During the rehearsal condition participants generated a descriptive label during encoding and covertly rehearsed during the delay period. During the suppression condition participants did not generate a label during encoding and suppressed (repeated “the”) during the delay period. This was easy in the former (novel scenes) but more difficult in the later condition (phase-scrambled scenes) where scenes lacked semantic content. Behavioral performance and EEG delay activity was analyzed as a function of maintenance strategy. Performance during WM revealed a benefit of rehearsal for phase-scrambled but not intact scenes. Examination of the absolute amplitude revealed three underlying sources of activity for rehearsal, including the left anterior temporal (ATL) and left and midline parietal regions. Increases in alpha and theta activity in ATL were correlated with improvement in performance on WM with rehearsal only when labeling was not automatic (e.g., phase-scrambled scenes), which may reflect differences in labeling and rehearsal (i.e., semantic associations vs. shallow labels). We conclude that rehearsal only benefits memory for visual stimuli that lack semantic information, and that this is correlated with changes in alpha and theta rhythms.

1. Introduction

It is well documented that maintenance via rehearsal benefits memory for verbal stimuli, such as words and numbers, typically through cumulative rehearsal when confronted with a list of verbal stimuli to remember. Baddeley (A. Baddeley, Logie, Bressi, Sala, & Spinnler, 1986) established the idea that rehearsal benefits memory, suggesting that the repetition of the to-be-remembered item will refresh the memory trace via the articulatory process in the phonological loop. Studies support that blocking rehearsal with articulatory suppression, repeating a word such as “the” over and over again, decreases performance compared with when someone rehearses (A. Baddeley, 2012; A. Baddeley, Lewis, & Vallar, 1984). Visual stimuli may also benefit from rehearsal when the stimulus is recoded into a verbal representation (Brown, Forbes, & McConnell, 2006; Souza & Skóra, 2017; Verhaeghen, Palfai, & Johnson, 2006). However, the benefit of labeling and rehearsal may depend on the working memory task and types of stimuli (Dent & Smyth, 2005; Sense, Morey, Prince, Heathcote, & Morey, 2017; Souza & Oberauer, 2018). Recently, studies using verbal stimuli in both serial recall and complex span tasks have suggested that rehearsal does not benefit working memory (Souza & Oberauer, 2018, 2020).

Most visual stimuli, no matter how simple or complex, have an affiliated word representation (e.g., a picture of an ocean with sand and trees has a label of “beach”) which automatically links the stimulus with stored semantic representations (Logie, 2011) that contains many features (e.g., sand and ocean) and associated knowledge (e.g., beaches are located in warm places and have palm trees). Thus, combining visual information with a semantic verbal representation may result in a deeper level of encoding (Craik & Lockhart, 1972; Ensor, Surprenant, & Neath, 2019) because the stimulus is encoded in both the visual and verbal domains (Paivio, 1969). The novel features of the visual stimuli may also help to guide attention to salient features which facilitates encoding (Nelson & Reed, 1976; Paivio, 1969). Few studies have examined whether complex visual stimuli such as scenes benefit from recoding. The effect of recoding has been studied with complex stimuli such as abstract shapes (Mayer, et al., 2007; Song & Jiang, 2006) and complex patterns (Brown, et al., 2006; Orme, Brown, & Riby, 2017; Riby & Orme, 2013). While some stimuli may benefit from recoding, especially if they contain high semantic content (Orme, et al., 2017; Riby & Orme, 2013), not all stimuli benefit from recoding (Chin & Schooler, 2008). Complex stimuli such as scenes may not benefit from recoding or rehearsal, especially if features lack concreteness such as nameable colors or line orientation (Verhaeghen, et al., 2006).

In order to understand the mechanisms that support maintenance of visual information, it is critical to examine neural activity during the delay period (Sreenivasan & D’Esposito, 2019) as it supports successful retrieval from working memory. Delay activity has been defined as a period of increased and sustained activation throughout the delay period (Sreenivasan, Curtis, & D’Esposito, 2014; Sreenivasan & D’Esposito, 2019) and has been identified in prefrontal and posterior parietal regions. Similar to the behavioral rehearsal literature, the delay activity literature has identified these patterns of maintenance with verbal and simple visual stimuli, with few studies measuring delay activity during maintenance of complex visual stimuli. Additionally, few studies of delay activity have controlled the maintenance strategy used by participants during the maintenance of information which is surprising because maintenance strategies can differentially engage brain regions (Fig2 - Sreenivasan and D’Esposito (2019)).

This traditional view of delay activity suggests that it is supported by persistent neuronal firing, which maintains information online until a response is made (Constantinidis, et al., 2018). Different types of maintenance strategies may be used by the participant especially without explicit instructions. Rehearsal engages language areas (Henson, Burgess, & Frith, 2000), especially for verbal stimuli (A. Baddeley, 2003). More specifically, studies that control for maintenance have identified left temporal activation that reflects the active use of the phonological loop during rehearsal (A. Baddeley, 2003; A. Baddeley, et al., 1984). Activation in the parietal region has been suggested as the store for visual information during maintenance and may serve as the buffer in which information lives until it is needed for retrieval, analogous to the verbal information store (A. Baddeley, 2003). In fact, it has been proposed that the visual cache, which holds non-spatial visual information, may overlap with the functions of the episodic buffer proposed in the multicomponent model (Logie, 2011). The episodic buffer is believed to serve as the interface between the information in the visual and verbal stores with stored semantic knowledge (A. Baddeley, 2000). Thus, the lateral posterior parietal cortex could represent the brain region in which the recoded verbal label is associated with the visually stored picture and the long-term semantic associations, consistent with the output buffer hypothesis (A. Baddeley, 2000; Hutchinson, Uncapher, & Wagner, 2009). Although, recoding has been proposed to automatically occur with verbalizable stimuli (Postle, D’Esposito, & Corkin, 2005) and rehearsal (Brown & Wesley, 2013), other mechanisms may support maintenance such as attentional mechanisms that engage a different set of brain regions. Attentional refreshing, which involves directing attention inward to selectively keep information active and prevent interference from other brain regions, engages attentional mechanisms (Cowan, et al., 2005).

If maintenance strategies are not explicitly controlled, delay period activity is difficult to interpret and may explain recent challenges to the established patterns (Miller, Lundqvist, & Bastos, 2018; Sreenivasan & D’Esposito, 2019). Controlling for these components is critical to identifying brain regions that support memory for visual information and how activity in these regions changes when words and semantic meaning are associated with visual information to support memory. In the present study two experiments were completed to understand how articulatory rehearsal, namely repeating a generated label throughout the delay period, influences short- and long-term memory for complex visual stimuli and the underlying delay activity.

Our study recorded continuous EEG during a visual working memory task in which maintenance strategy was controlled. Participants used either covert articulatory rehearsal or covert articulatory suppression while maintaining two complex visual images. In Experiment 1, the complex visual images were naturalistic outdoor scenes. In Experiment 2, the complex visual images were phase-scrambled versions of the scenes presented in Experiment 1. Naturalistic outdoor scenes are easier to label than the phase-scrambled versions, which lack semantic content but contain similar features (e.g., nameable colors). Therefore, we increased task difficulty in Experiment 2. It was predicted that both types of stimuli would rely on the phonological loop for recoding and rehearsal and the visual store for the visual component of the stimulus. Additionally, labeling both types of stimuli would involve associations with semantic long-term memory, although the associations would be more effortful in Experiment 2 as labeling was likely not automatic. Hence, greater attentional resources would be necessary to support a more effortful labeling and rehearsal.

The neural patterns observed during maintenance were examined in order to understand how maintenance strategy modulates delay activity for complex visual stimuli. We focused on the oscillatory correlates associated with working memory maintenance, theta and alpha band activity (Klimesch, 1996, 1999; Pavlov & Kotchoubey, 2020), and beta activity, which is associated with language-based maintenance techniques (Weiss & Mueller, 2012; Weiss & Rappelsberger, 1998). Gamma band activity has also been examined in WM (Jokisch & Jensen, 2007; Kopp, Schröger, & Lipka, 2001, 2006) and has been associated with attentional selection (Green, et al., 2017). However, the role of gamma activity during maintenance is less convincing (Pavlov & Kotchoubey, 2020), so the current study focused on the established theta, alpha and beta oscillations. It was hypothesized that controlling for the maintenance strategy by comparing articulatory rehearsal to articulatory suppression would result in differences in behavioral performance and delay period activity. Regardless of the type of complex visual stimulus, it was predicted that articulatory rehearsal would provide a behavioral advantage over suppression both in tests of working memory and immediate long-term memory. Delay activity during articulatory rehearsal would be sustained throughout the delay period for both types of complex stimuli, in the alpha and beta bands in parietal regions, and correlated with performance. Additionally, it was predicted that there would be increased and sustained delay activity, in the alpha and theta bands in frontal regions, with increasing task difficulty.

2. Methods

2.1. Participants

Experiments 1 and 2 were conducted at The City College of New York (CCNY), a large institution located in a large urban city. CCNY is a non-traditional institution that consists of diverse population of students from many backgrounds and ages. The study was approved by the Institutional Review Board of The City University of New York Human Research Protection Program (CUNY IRB). A total of 54 participants provided informed consent and completed the study. Participants were adults between the ages of 18 and 56, with normal or corrected vision, and with no self-reported cognitive or psychiatric illnesses. Our study includes a representative sample of CCNY students, with an age limit below the current age for age-related impairment (Alzheimer’s Association, 2019). The study was carried out following the CUNY IRB guidelines and regulations. Participants were compensated with either 15 dollars or one extra course credit per hour of participation. The behavioral task with EEG recordings took approximately 2 hours to complete.

2.1.1. Experiment 1

One participant was excluded from Experiment 1 of the study for failure to follow instructions resulting in a final sample in Experiment 1 of 29 participants (age = 25.4 (8.1) years, 14 females). For EEG analysis, a total of 6 participants were excluded: 5 participants for noisy or unusable EEG recordings and 1 participant for failing to follow instructions resulting in a final sample for Experiment 1 consisting of 24 participants (age = 25.8 (8.6) years, range 18–56, 11 females).

2.1.2. Experiment 2

Four participants were excluded because of computer malfunction preventing recording of behavioral responses resulting in a final sample of 20 participants (age = 24.8 (9.5) years, range 18–56, 12 females).

2.2. Experimental Design and Statistical Analyses

2.2.1. Task

Participants completed a modified version of a Sternberg WM Task (Sternberg, 1966), with separable phase (e.g. encoding, delay period, probe choice). The overall study design consisted of two working memory (WM) tasks (see Figure 1; 100 trials per task) and a delayed old-new recognition task (150 trials) approximately 10-minutes later. During this short 10-minute break, participants remained in the lab and did not engage in any tasks.

Figure 1. Example Working Memory Trial in Experiment 1 (Intact Scenes) and Experiment 2 (Scrambled Scenes).

The task consisted of a low cognitive load (2 images). Participants were presented with a fixation cross (1 sec) that indicates the start of the trial, followed by 2 images in succession (2 sec each), a blank screen during the delay period (6 sec), a probe choice (2 sec), which is either one of the earlier presented images or a new image, and a phase-scrambled image (1 sec) that indicates the end of the trial. An example of a rehearsal trial in which participants generate the label for each image and rehearse during the delay period (a) and an example of a suppression trial in which participants suppress during the delay period (b). Experiment 2 used the same task used in Experiment 1 except with phase-scrambled scenes. An example of a rehearsal trial in which participants generate the label for each phase-scrambled image and rehearse during the delay period (c) and an example of a suppression trial in which participants suppress during the delay period (d).

For the WM tasks, participants were presented with a fixation cross (1 sec) that indicates the start of the trial, followed by 2 images in succession (2 sec each), a blank screen during the delay period (6 sec), a probe choice (2 sec), which is either one of the earlier presented images or a new image, and a phase-scrambled image (1 sec) that indicates the end of the trial. The long 6-sec delay-period is indicative of real-world scenarios for maintaining information and has been used in similar studies of memory (Ellmore, Ng, & Reichert, 2017). The participant made probe choices using a RB-530 response pad (Cedrus Inc). If a probe matched one of the previously presented images from the encoding set, the participant pressed the green (right) button on the response pad. If the probe did not match the encoding set, the participant pressed the red (left) button.

Before completing the WM tasks a short 10-trial practice set was completed with the experimenter and feedback was provided after each trial. For the first 3 rehearsal practice trials, the experimenter provided example labels for each image and rehearsed the labels out loud for the participant. During the remaining rehearsal practice trials, participants were encouraged to generate the verbal labels during encoding and rehearse out loud during the delay. For the first 3 suppression practice trials, the participant was reminded not to generate a label and the experimenter repeated “the..the..the” out loud during the delay period at a pace of 1 word per second. During the remaining suppression practice trials participants were reminded not label during encoding and repeat “the” out loud during the delay, to confirm task compliance. Before beginning the task, participants were reminded to rehearse or suppress covertly (silently) and compliance was confirmed throughout the test session via intercom.

For the articulatory rehearsal condition (termed: rehearsal condition), during presentation of the images (i.e., the encoding phase), participants were instructed to generate a verbal label (i.e., descriptive label like “beach”). During the delay period participants were instructed to rehearse covertly (i.e., using their inner voice) the label they generated; they were instructed to rehearse throughout the entire delay period. For the articulatory suppression condition (termed: suppression condition), participants were discouraged from generating a verbal label during the encoding phase and instructed to repeat the word “the” covertly throughout the delay period (A. D. Baddeley, Thomson, & Buchanan, 1975; Landry & Bartling, 2011). Suppression was used to block verbal rehearsal by engaging the articulatory process of the phonological loop with an unrelated word. This suppression task uses repetition of a non-task related word such as “the” over and over again (A. D. Baddeley, et al., 1975; Landry & Bartling, 2011; Ricker, Cowan, & Morey, 2010). This classic suppression task was chosen over an alternative task such as random number generation or a math problem because these alternative tasks engage attentional processes. The goal of suppression is to block rehearsal without a demand on attention, which has been demonstrated to have a longer term effect on memory tasks (Norris, Hall, Butterfield, & Page, 2019; Ricker, et al., 2010). Participants completed both the rehearsal and suppression conditions in a randomized order.

The recognition task was a mix of any encoding image from either the rehearsal and suppression conditions (40 images from each condition), as well as new images (70 images). Participants indicated if the images were old or new. During the recognition task, participants indicated if the image was presented in either of the working memory conditions or if it was a new image. If they indicated that they saw the image in one of the earlier working memory conditions, they were asked to indicate if they remembered labeling the image and verbally stated the label that was used. During the recognition task participants were required to respond (i.e., indicate if the image was old or new) before providing the label verbally. Participants were given unlimited time to provide the label overtly. This confrontation of participants during the recognition task by the experimenter served as task-check to see if participants followed instructions. All participants, except one who was excluded from the analysis, confirmed that they followed instructions and indicated the condition in which they saw the old images (i.e., rehearsal vs suppression). Although the generated labels were reviewed during the recognition task to confirm task compliance, they were not systematically analyzed.

Experiment 2 was a replication of Experiment 1 with phase-scrambled stimuli to increase task difficulty by removing semantic content. A complex visual stimulus that has not been evaluated to our knowledge is a phase-scrambled scene which is difficult to recode, specifically with one-word representations, and is often described based on basic features (e.g., colors or line orientation). The difficult-to-describe nature of these stimuli make them suitable for studying visual memory because it is not likely to be automatically recoded nor to trigger a deeper semantic representation (Brown, et al., 2006; Brown & Wesley, 2013). Therefore, these stimuli are well suited to assess benefits of maintenance strategies (i.e., articulatory rehearsal) for storing and remembering complex visual information.

2.2.2. Stimuli

The stimuli were presented as 800 by 600 pixels on a gray background. Precise stimulus timings were recorded using a photosensor that was located on the computer monitor in the upper right corner. The photosensor detected stimulus onset and offset using a small black box presented on the gray background. The small box was present whenever a stimulus was present and was not visible to participants. The timings were then coded to represent the separate events (e.g., delay period, probe, etc.). In Experiment 1, stimuli consisted of high-resolution, color outdoor scenes, which did not contain any faces or words. Scenes were randomly selected from the SUN database (Xiao, Hays, Ehinger, Oliva, & Torralba, 2010) and were resized to 800 by 600 pixels. Experiment 2 employed the same design as Experiment 1 with Fourier phase-scrambled (Matlab, v2016) versions of the scenes used in Experiment 1 (Figure 1c and d). Importantly, the scenes contained the same colors and spatial frequencies as the scenes used in Experiment 1 but lacked semantic content. It was more challenging to generate labels because phase-scrambling removes all semantic content.

2.2. Behavioral Analysis

Behavioral data were processed in Python 3.0, and plots were created using Seaborn (v0.9.0) with custom scripts. Statistical analysis was conducted in JASP (v0.9.0.1) using paired-samples t-tests to compare behavioral accuracy and reaction time between conditions (rehearsal vs. suppression) on the WM and recognition tasks and Pearson correlations between behavior and delay activity.

2.3. EEG Acquisition, Processing and Plots

2.3.1. EEG Acquisition and Processing

Continuous 64-channel EEG was collected at a sampling rate of 1000 Hz using an active electrode system with actiCHamp system (Brain Products). Electrodes were reduced to an impedance of 25 kOhms or lower, per the manufacturer’s specifications. Channels with an impedance above 25 were interpolated. The reference electrode was the left mastoid (TP9) and was re-referenced offline to the common average reference. Data were visually inspected and muscle artifacts were identified and marked for removal. Eye blinks were corrected with a pattern-matching algorithm using principle component analysis (Ille, Berg, & Scherg, 2002; Picton, et al., 2000). The method removes the variance associated with a blink, from each channel, using LOC and ROC channels (opposite deflection pattern > 100 μV). Artifact-corrected data was used for all analyses.

Participants were included in the EEG analysis if they had more than 50 delay periods that survived the artifact scan in both the rehearsal and suppression conditions. Trials were excluded based on BESA Research criteria for artifact rejection: trials were excluded for amplitude > 148 μV*, gradient > 75 μV, low-signal criteria > 0.1. To maximize the number of trials included, amplitudes (*) above 148 were manually reviewed and may have been included on an individual trial basis. In Exp 1, the average number of delay period was 77 for suppression and 78 for rehearsal and in Exp 2, it was 80 for each condition, with no significant differences between the number of delay periods included (p > .05).

2.3.2. EEG Time Frequency Analysis

Time-frequency analyses (TFA; absolute amplitude and temporal spectral analysis) were conducted on artifact-corrected delay periods (0 to 6000 ms) and bandpass-filtered between 4 and 30 Hz. Complex demodulation of the recorded EEG signals for each trial was carried out in BESA Research v6.1 (Hoechstetter, et al., 2004; Papp & Ktonas, 1977). A detailed description of the demodulation can be found in Ellmore, et al. (2017). Briefly, the timeframe (t) was set as the entire delay period (0 to 6000 msecs) and the baseline was set to the same period (Ellmore, et al., 2017). The finite impulse response (FIR) filter was applied in latencies of 100 msecs and 0.5 Hz steps. First, the raw amplitude of brain activity within each frequency and latency bin was generated for condition (termed Time Frequency Analysis or TFA). Then, the amplitude within frequency bin across time was compared to the baseline amplitude and averaged over trials (termed Temporal Spectral Analysis or TSA). The TSA is:

$$TSA=\frac{A\left(t,f\right)-\mathit{\text{Abaseline}}\left(f\right)}{\mathit{\text{Abaseline}}\left(f\right)}*100\%$$

A(t,f) = amplitude during the timeframe of interest and frequency A_baseline (f) = mean amplitude in the frequency band during the baseline period Temporal Spectral Analysis is expressed as a percent or change in amplitude, the resulting TSA is a value that is either + or −. A positive change represents enhancement of activity (or synchronization) and a negative change represents a suppression of activity (or desynchronization) relative to the event of interest, the delay period (Pfurtscheller, 2001; Pfurtscheller & Da Silva, 1999).

A brain source montage was applied to TFA to distinguish between broad, potentially overlapping sources (e.g., frontal, central, parietal, etc.). The brain region montage consists of 15 discrete regions and source-level analysis calculates weighted combinations of the recorded scalp electrodes in order to reduce the amount of overlap (Hoechstetter, et al., 2004; Scherg, 1992). This source-level analysis provides a powerful way to reduce the spatial overlap of the sensor-level activity (Ellmore, et al., 2017).

2.3.3. EEG Plots

A Brain Vision Matlab script for reading BESA-generated files into other packages (https://www.besa.de/downloads/matlab/) was used to read in the TFA files. BESA’s TFPlot was used to generate the full-trial TSA plot. The TFA and TSA delay period average plots were recreated using custom scripts and generated in Python. The significant clusters from permutation tests that were generated in BESA Statistics, were retraced in Adobe Illustrator v2019, and overlaid on the python-generated TFA average plots. Average TFA by frequency was calculated using custom Python scripts. The frequency band differences were calculated by averaging the absolute amplitude within a given frequency band across the entire delay period (0–6000 msec) for rehearsal and subtracting the corresponding value for the suppression condition. The frequency (f) ranges under consideration were 4 to 30 Hz: theta (4–8 Hz), alpha (8–13 Hz), and lower and upper beta (13–20 and 20–30 Hz, respectively).

2.3.4. EEG Statistical Analysis

Scalp-level TFA and source-level TFA were compared using paired-samples t-tests with corrections for multiple comparisons using non-parametric cluster permutation testing (N = 1,000 permutations) (Maris & Oostenveld, 2007) in BESA Statistics v2.0. In order to account for multiple comparisons across frequency, time, and sensor space, non-parametric permutation t-tests or correlations were run. The analyses result in a cluster value. For the t-test, the cluster value is a sum of t- values and for a correlation, the cluster value is a sum of the r-values for the data points in a group of adjacent bins (sensor (<4 cm distance), time (100 msec), frequency (0.5 Hz) bins). A null distribution of summed t-values for the t-test analysis or r-values for the correlation analysis are generated from random clusters across subjects and across time-frequency bins. Significant clusters are summed t- or r-values within a time-frequency domain that exceed a specific threshold and are then compared to the random null distribution (Maris & Oostenveld, 2007). Significant clusters were defined as p-value less than or equal to 0.05 and marginally significant was defined as less than or equal to 0.1. Cluster permutation tests involve sensors that are <4 cm apart, which suggests that if a significant finding occurs across neighboring electrodes it is unlikely that the effect is due to chance. This type of non-parametric permutation test overcomes the Type I error that can occur with multiple comparison testing. Significant clusters are indicated with a mask, either blue or orange. An orange cluster represents greater amplitude for one condition versus another (always rehearsal compared to suppression) for the TFA or event-related synchronization for the TSA. Blue represents a smaller amplitude (for rehearsal versus suppression) for the TFA or event-related desynchronization for the TSA.

3. Results

3.1. Experiment 1

3.1.1. Behavioral.

Performance on the working memory task revealed that there was no significant difference between rehearsal and suppression (0.95 vs. 0.95 proportion correct, t₍₂₈₎ = .70, p = 0.49, d = 0.13), suggesting that rehearsal did not provide a short-term behavioral advantage (Figure 2a and b). There was no significant difference in reaction time between rehearsal and suppression (951.50 vs. 970.28 msec, t₍₂₈₎ = −1.19, p = 0.24, d = −0.22) for correct responses on the on the working memory task. Similarly, there was no long-term behavioral advantage on the delayed recognition task for rehearsal vs. suppression (0.80 vs. 0.78 proportion correct, t₍₂₈₎ = 1.38, p = 0.18, d = 0.23).

Figure 2. Working Memory and Recognition Performance in Experiment 1 (Intact Scenes) and Experiment 2 (Phase-Scrambled Scenes) Suggests Rehearsal Only Benefits Memory for Scenes that Lack Semantic Content.

Performance is shown with four plots for each condition and experiment. 1) The violin plot shows the distribution of performance across each condition; the cluster of similar scores is reflected at the wider (top) portion of the violin and the thinner (bottom) portion with fewer scores. The dashed lines reflect the quartiles of the overall distribution similar to the box plot. 2) The boxplot also shows the distribution of performance for each condition (i.e., the solid lines inside the box align with the dashed lines of the violin plot, representing the quartiles). In addition, the boxplot contains whiskers extend to the minimum and maximum score (1.5 times the median) for each condition and the diamonds reflect outliers (greater than 1.5 times the median). Outliers were included in all analyses. 3) In between the violin plot and boxplot is a point plot, which reflects the mean of performance accuracy, with error bars reflecting the 95% confidence intervals; the confidence intervals were generated with bootstrapping using 1,000 iterations. 4) Finally, the individual dots on the inner-most portion represents single participants score for a condition. The gray line connects the dots that represents performance for the corresponding condition (i.e., performance on suppression (left) with performance on rehearsal (right)). a) Comparison of performance on WM task for Experiment 1 (n = 29), as measured by proportion correct, shows that rehearsal (light orange) provided no benefit for short-term memory as compared with suppression (dark orange on the working memory task (p = 0.49). b) Comparison of performance on the recognition task for Experiment 1, for images from the rehearsal condition (light orange) versus images from the suppression condition (dark orange), suggested that there was no long-term benefit of rehearsal (p = 0.18). c) Examination of performance on the working memory task for Experiment 2 (n = 20) shows that rehearsal (light blue) provided a short-term advantage as compared with suppression (dark blue) on the working memory task (p < 001). d) Comparison of performance on the recognition task for Experiment 2 also revealed a long-term advantage for images from the rehearsal condition (light blue) versus images from the suppression condition (dark blue), (p < .001).

3.1.2. EEG.

Change in sensor-level amplitude over time (temporal spectral analysis) reveals a transient pattern of delay activity during maintenance throughout the 6 sec delay period (Supplementary Figure 1a). It is transient because activity was increased and synchronous early in the delay period in the upper alpha and lower beta range (500 msec to 3000 msec) but became desynchronous later in the delay period (4000 msec to 6000 msec). Temporal spectral analysis revealed no significant difference between rehearsal versus suppression (p = 0.08), nor was rehearsal significantly correlated with working memory (p = 0.46) or recognition performance (p = 0.28). This suggests that the early pattern of event-related synchronization or enhancement (ERS) in alpha and beta and the late pattern of event-related synchronization or suppression (ERD) in the same frequency bands was the same regardless of the maintenance mechanism. The full trial time frequency analysis revealed similar patterns of ERD in alpha and beta range (Supplementary Figure 2a). This transient pattern of delay activity was found across all sensors. The strongest increases and decreases in activity were observed in the bilateral parietal and parieto-occipital sensors.

Sensor-level changes in absolute amplitudes between the two conditions (n=24 subjects) with corrections for multiple comparisons revealed 100 significant clusters (Supplementary Table 1, p < .05). Sensor-level analysis revealed two distinct patterns of activity for the rehearsal and suppression conditions (Supplementary Figure 3). For the rehearsal condition (Figure 3 and c – P8 electrode - orange clusters), amplitude was greater in the theta and beta range for the left frontal, bilateral fronto-temporal, and central regions, and in the beta range for the right parietal region, throughout the delay period. For the suppression condition (Figure 3 a and b – F1 electrode - blue clusters), the amplitude was greater in the upper alpha and lower beta range in the mid-frontal regions early in the delay, and in the theta and upper alpha range in the midline and centro-frontal, right parietal, and occipital regions later in the delay.

Figure 3. Comparison of Absolute Amplitude Delay Period Activity in Experiment 1 (Intact Scenes) Reveals Different Activity Patterns Between Rehearsal and Suppression.

Select absolute amplitude plots in the left frontal and right parietal regions of the 6-sec delay period revealed 106 clusters of significant differences in activity (p < 0.05). The y-axis shows frequency (Hz); x-axis shows the time in sec. a) Montage of 61-scalp electrodes on head plot with two electrodes highlighted (colored black) F1 and P8. b) Selected electrode shown from the right parietal region (P8 electrode) displays orange clusters which represents the bins in frequency-time that are greater in amplitude for the rehearsal condition as compared with the suppression condition. Electrode selected based on results from previous research in our lab highlighting the right parieto-occipital region (Ellmore, et al., 2017). c) Selected electrode from the left frontal region (F1 electrode) shows blue clusters which represents the bins in frequency-time that are greater in amplitude for the suppression condition as compared with the rehearsal condition.

Source-level analysis of the absolute amplitudes found 57 significant clusters across 13 of the 15 brain regions. The main source of activity for the rehearsal condition was central midline (CM_BR; 12 clusters, Time 100 to 6000 msec, Frequency 12–29 Hz), but overall changes were apparent over numerous brain regions. In the suppression condition, three regions were identified as sources of activity including the right temporal parietal region (TPR_BR; 8 clusters, Time 800 to 5400 msec, Frequency 10.5–17.5 Hz), the left temporal parietal region (TPL_BR; 8 clusters, Time 0 to 5900 msec, Frequency 7–12.5 Hz), and the right anterior temporal region (TAR_BR; 11 clusters, Time 100 to 5800 msec, Frequency 15–30 Hz) regions. The TPL_BR and TPR_BR amplitude differences are more concentrated in the upper alpha and lower beta ranges, whereas the TAR_BR activity was mainly in the beta range.

3.2. Experiment 2

3.2.1. Behavioral.

When task difficulty increased, there was both a significant short-term advantage of rehearsal (Figure 2c and d) compared to suppression (0.85 vs. 0.78 proportion correct, t₍₁₉₎ = 7.93, p < .001, d = 1.77) as well as a long-term advantage during rehearsal compared to suppression (0.71 vs. 0.62 proportion correct, t₍₁₉₎ = 4.58, p < .001, d = 1.02). There was no significant difference in reaction time between rehearsal and suppression (1080.31 vs. 1060.47 msec, t₍₁₉₎ =1.143, p = 0.27, d = 0.26) for correct responses on the WM task.

3.2.2. EEG.

It was predicted that rehearsal and suppression would produce similar EEG delay period activity as in Experiment 1 since participants would be engaging in the same maintenance strategy, which should engage similar brain regions. Change in sensor-level amplitude over time (temporal spectral analysis) confirmed a transient pattern of delay activity during maintenance (Delay Period: Supplementary Figure 1b and Whole Trial: Supplementary Figure 2b).

Sensor-level temporal spectral analysis showed transient changes, similar to the delay activity pattern observed in Experiment 1, across all sensors. Activity was increased and synchronous in the early delay period and became desynchronous later in the delay with the greatest change observed in bilateral parietal and parieto-occipital sensors. Comparison between the rehearsal and suppression conditions using temporal spectral analysis revealed three clusters of significantly different activity (Supplementary Table 3). An early (0–1400 msec) and a late (3900–6000 msec) cluster represents alpha and beta ERD greater for the rehearsal condition as compared with suppression. The opposite pattern of TSA was identified in the middle of the delay period (1100–4600 msec) in which ERS was greater in the alpha and beta regions for the rehearsal condition as compared with suppression. Activity during rehearsal was also significantly correlated with WM performance (Figure 4a and b; Cluster 1: blue-negative correlation, cluster value = −38801.2, p = 0.005, Cluster 2: orange-positive correlation, Cluster value = 20445.5, p = 0.065), but not performance on the recognition task (Figure 4c; p = 0.62). Early ERS of alpha and beta activity was associated with better performance, and later ERD of the same frequency bands was associated with better performance on the WM task.

Figure 4. Delay Period Activity Time Frequency Analysis in Experiment 2 (Phase-Scrambled Scenes) Reveals Correlations with Performance.

a) The y-axis is frequency (Hz); x-axis is the time in sec. Time Frequency Analysis correlation plot for the PO8 electrode during the 6-sec delay period in a whole window analysis. Significant clusters are represented by a mask of orange and blue. The first cluster (b-left plot, orange, latency: 0 to 2400 msec and frequency: 4 to 30 Hz) represents a positive correlation between activity during the delay period and performance on the working memory task from the rehearsal condition (Cluster value = 20445.5, p = 0.065) and the second cluster (b-right plot, blue, latency: 1400 to 4600 msec and frequency: 4 to 30 Hz) represents a negative correlation with performance (cluster value = −38801.2, p = 0.005). Performance on the recognition task was not significantly correlated with delay activity (c; p = 0.62).

Sensor-level analysis of the absolute amplitude between the two conditions (n = 20 subjects) with corrections for multiple comparisons revealed 15 significant clusters (Supplementary Table 2, p < 0.05). Greater amplitude was observed in the upper alpha and beta ranges across a majority of the sensors for the rehearsal condition as compared with the suppression condition, specifically in the bilateral temporal, occipital and parietal regions, and left frontal region (Supplementary Figure 4). The pattern of delay activity appeared to be sustained until about 3000 msec and then decreased in amplitude until the end of the delay period, specifically in right parietal and occipital regions (Figure 5a and b – P08 – orange cluster). There was also a brief early period (Time 0 to 1800 msec) in which greater amplitude was observed in the suppression condition in the theta, alpha, and beta range (blue cluster).

Figure 5. Comparison of Absolute Amplitude Delay Period Activity in Experiment 2 (Phase-Scrambled Scenes) Reveals Greater Activity for Rehearsal.

The y-axis is frequency (Hz); x-axis is the time in sec. a) Montage of 61-scalp electrodes on head plot with one electrode highlighted (colored black) P08. b) Absolute amplitude plot for the PO8 electrode during the 6-sec delay period. PO8 was selected from all electrodes in the head map because it has the broadest clusters present of the other electrodes. The orange represents the clusters in frequency-time that are greater in amplitude for the rehearsal condition as compared with the suppression condition (15 significant clusters across all electrodes, p < 0.05). These cluster details are described in Supplementary Table 3.

Source-level analysis of absolute amplitudes revealed 67 significant clusters across 11 of the 15 brain regions (Supplementary Figure 5). Of the significant clusters, a focus of activity in the rehearsal condition was observed in the left anterior temporal region (TAL_BR; Figure 6a and b, 10 clusters, Time 0 to 6000 msec, Frequency 4–30 Hz), left parietal (PL_BR; 10 clusters, Time 0 to 6000 msec, Frequency 6–30 Hz), and midline parietal region (PM_BR; 6 clusters, Time 0 to 6000 msec, Frequency 20–30 Hz) regions. For the suppression condition, an early focus of activity (Time 0 to 14000 msec) in the alpha range (Frequency 7–13.5 Hz) was found in the right and left parietal, parietal midline, and occipito-polar midline regions; a later focus in the theta range (4–8 Hz) was found in the same brain regions as well as in the right frontal region (Time 3800 to 6000 msec).

Figure 6. Delay Period Activity Brain Region Analysis in Experiment 2 (Scrambled Scenes) Reveals a Left Anterior Temporal Region during Maintenance Rehearsal.

a) 3-D head with the left anterior temporal region (TAL_BR) highlighted. b) Absolute amplitude plot for the TAL_BR region during the 6-sec delay period (10 clusters, Time 0 to 6000 msec, Frequency 4–30 Hz). The y-axis is frequency (Hz); x-axis is the time in sec. Orange clusters represent rehearsal delay activity greater than suppression and blue clusters represent suppression delay activity greater than rehearsal. If focal activity is not visible, it suggests that the underlying sources of brain activity are more widespread. A clear focal point of activity for the rehearsal condition is found in the left anterior temporal region (TAL_BR) and the left- and parietal midline regions (PL_BR, and PM_BR respectively see Supplementary Figure 5) which suggests that these regions are the source of delay activity for articulatory rehearsal.

3.3. Cross-Study Comparisons

Correlations between absolute amplitude and performance were run to see how changes in amplitude relate to changes in performance for Experiment 1 and Experiment 2. For Experiment 1, performance was not significantly different between conditions (Supplementary Figure 6). The differences in performance were not significantly correlated to differences in amplitude for any region or frequency range. In Experiment 2 however, significant correlations between absolute amplitude and performance were found exclusively in the TAL_BR region. Figure 7 shows the correlations for each region identified as a potential source of rehearsal by frequency. Increases in both theta (r = 0.48, p = 0.034) and alpha (r = 0.52, p = 0.019) oscillations are related to improved performance when rehearsal is employed. Importantly, the significant correlations for the TAL_BR region suggest that increases in theta and alpha band activity are related to improved short-term performance, when rehearsal is employed with images that lack semantic content.

Figure 7. Correlations of Absolute Amplitude with Performance in Experiment 2 (Phase-Scrambled Scenes).

The y-axis frequency difference (Amplitude for Rehearsal condition minus Amplitude for Suppression Condition). The x-axis performance difference (Proportion Correction for Rehearsal condition minus Proportion Correction for Suppression Condition). The frequency difference was calculated by averaging the absolute amplitude within a given frequency range (i.e., theta (4–7 Hz), alpha (8–13 Hz), lower beta (13–20 Hz), and higher beta (20–30 Hz) across the entire delay period. The correlations between difference in frequency and difference in performance are presented as a matrix for Experiment 2, with the TAL_BR region on the left (pink dots represent individual subjects), PL_BR region in the middle (red dots represent individual subjects), and PM_BR region on the right (orange dots represent individual subjects). These regions reflect the regions described in Supplementary Figures 7–9. These correlations reveal significant relationships in the alpha and theta frequency ranges for the TAL_BR region. Increases in both theta (r = 0.48, p = 0.034) and alpha (r = 0.52, p = 0.019) oscillations are related to improved performance when rehearsal is employed.

An exploratory comparison between results of Experiment 1 and Experiment 2 is included in the Supplementary Results.

4. Discussion

The role that rehearsal plays in modulating neural delay activity remains an open research question. Experiments 1 and 2 examined how controlling for maintenance strategy impacted the delay activity observed during a visual WM task. A behavioral advantage of rehearsal was observed only when the stimuli lacked semantic content (i.e., phase-scrambled scenes). Examination of changes in delay activity over time during rehearsal with intact scenes (Experiment 1) and phase-scrambled scenes (Experiment 2) failed to reveal a sustained pattern of delay activity. Rather a transient pattern of delay activity with an early synchronous pattern in the alpha and beta ranges during the first half of the delay period followed by a desynchronous pattern of activity in the same ranges until the probe choice were observed. Additionally, early synchronous alpha and beta range activity and greater desynchronous activity in the same ranges was related to better short-term performance for the rehearsal condition with phase-scrambled scenes.

Examination of absolute amplitude during rehearsal with intact scenes (Experiment 1) and separately with phase-scrambled scenes (Experiment 2) revealed differences in all observed frequencies ranges when compared with suppression. Additionally, the TAL_BR, PL_BR, and PM_BR regions were identified as a source of delay activity during rehearsal. In fact, improved performance on rehearsal (as compared with suppression of rehearsal) in the former region (TAL_BR) was related to increases in the alpha and theta frequency bands. Interestingly, this was only the case when the stimuli lack semantic content, suggesting that increases and alpha and theta are important when maintenance requires increased effort (i.e., automatic vs. non-automatic labeling).

In Experiment 1, generating a meaningful label for novel scenes and rehearsing them during the delay period provided neither a short- or long-term behavioral advantage. Comparison of sensor-level delay activity revealed a distinct pattern of greater amplitude in the beta range in the right parietal and centromedial regions for rehearsal compared to suppression. This activity was not correlated with either short- or long-term memory. Source-level analysis of EEG activity during rehearsal did not reveal any clear sources of activity. Additionally, comparison of change in amplitude over time did not reveal differences between the conditions. It did, however, reveal a transient pattern of change in amplitude in the alpha and beta ranges throughout the delay, with an early synchronous pattern followed by a late desynchronous pattern of activity. The lack of neural difference between change in amplitude over time between the suppression and the rehearsal condition, suggests that the same transient pattern of activity supported the near-ceiling performance in each condition.

When task difficulty increased in Experiment 2 using phase-scrambled scenes there was both a significant short-term as well as a long-term advantage with labeling and rehearsal. Sensor-level EEG activity during rehearsal showed greater amplitude in the upper alpha and beta ranges, specifically in the bilateral temporal, occipital and parietal regions and a left frontal region. Source-level analysis revealed a focus of activity during the rehearsal condition in the left anterior temporal, left parietal and parietal midline regions. Examination of the change in amplitude over time confirmed the transient pattern of delay activity that was observed in Experiment 1. Additionally, sensor-level temporal spectral analysis for rehearsal was significantly different from suppression and was also correlated with short-term memory, with better performance associated with greater enhanced alpha and beta activity early in the delay period and greater suppression of these oscillations later in the delay period. The significant difference in neural activity between the suppression and the rehearsal condition suggests that increases in the alpha and beta ranges (early in the delay) and decreases in the same ranges (later in the delay) are modulated by the use of rehearsal and the amount of effort associated with recoding and rehearsing.

4.1. Role of Rehearsal in Visual Memory

The role of rehearsal in supporting visual memory remains unclear, especially whether or not rehearsal benefits complex visual stimuli. Experiment 1 used intact, novel outdoor scenes that contained semantic information and were easy to generate a descriptive label for (e.g., a beach). In the rehearsal condition, label generation provided a dual means of encoding (Brown, et al., 2006; Nelson & Reed, 1976; Paivio, 1969), one in the visual and one in the verbal domains, which was intended to benefit memory. However, there was no difference in performance on the short- or long-term memory task with intact scenes, which suggests that complex scenes do not benefit from this type of maintenance strategy. Complex scenes automatically trigger stored semantic associations (Ensor, et al., 2019); as a result, the association provided automatic deeper encoding (Craik & Lockhart, 1972) for both conditions. It has also been established that humans can remember thousands of images after only seeing the images for a brief time (Brady, Konkle, Alvarez, & Oliva, 2008; Standing, 1973; Standing, Conezio, & Haber, 1970). This ability is termed the picture superiority effect (Stenberg, 2006) and may also account for the fact that performance for images was near ceiling regardless of maintenance strategy.

The benefit of a recoding a visually presented stimulus may depend on whether semantic associations are automatically accessed without labeling (Santangelo, 2015; Souza & Skóra, 2017). In Experiment 1 automatic semantic associations occurred; thus, the addition of rehearsing with a generated label offered no more benefit than accessing those stored associations (Brown & Wesley, 2013; Ensor, et al., 2019; Nelson & Reed, 1976). Although some stimuli such as spatial layouts may not benefit from recoding (Postle, et al., 2005), some complex stimuli may benefit from this processes. In Experiment 2, phase-scrambled images that lacked semantic content were used and these images benefited from recoding and rehearsal. This suggests that the benefit of rehearsal depends on the ability to associate semantic meaning; therefore, when participants generated a label and rehearsed throughout the delay period, they engaged in deeper encoding and elaborative rehearsal (Brown, et al., 2006; Cermak, 1971; Craik & Lockhart, 1972; Ensor, et al., 2019; Phaf & Wolters, 1993) improving short-term memory for these stimuli as well as long-term memory. These findings are consistent with the idea that generating a label and rehearsing is only beneficial to visual stimuli when semantic information is not automatically accessed (Nelson & Reed, 1976) because it may direct the subjects’ attention to salient features (Souza & Skóra, 2017) to support memory.

The effect of recoding on WM has been studied behaviorally for decades with some studies suggesting that the addition of verbal processing may interfere with memory for the visual stimulus, also referred to as verbal overshadowing (Brandimonte, Hitch, & Bishop, 1992; Chin & Schooler, 2008; Schooler & Engstler-Schooler, 1990). A benefit of recoding may only occur with specific types of visual stimuli (Schooler & Engstler-Schooler, 1990), such as in the case of when the label can help to distinguish the encoded stimulus from the probe. The creation of the label can also lead to a distortion of the original memory which results in confusion during the probe response (Chin & Schooler, 2008). The negative impact may be related to the amount of time available for the generation for the label (Souza & Skóra, 2017) or may depend largely on the probe task and the specific instructions given (Chin & Schooler, 2008). For example, verbal overshadowing is evident with forced recall tasks (Chin & Schooler, 2008), whereas with a simple recognition task recoding may beneficial, such as in the current study.

4.2. Delay Activity and Rehearsal

Delay activity during a working memory task is often associated with the engagement of either the prefrontal cortex or the posterior parietal regions but has been reported in studies that often fail to control for maintenance strategy. When intact scenes served as stimuli in Experiment 1, we observed greater activity in the left temporal and bilateral central regions during rehearsal, which suggests the engagement of the phonological loop (A. Baddeley, 2003; Hwang, et al., 2005). Suppression of rehearsal resulted in the engagement of more frontal regions suggesting reliance on attentional mechanisms (Camos, Mora, & Oberauer, 2011) as well as greater mental effort (Kopp, et al., 2006). Source analysis revealed diffuse activity for both rehearsal and suppression conditions, which suggests no underlying focus of delay activity.

In Experiment 2, activity was greater in the upper alpha and beta ranges for the rehearsal condition throughout the delay period, specifically in the bilateral temporal, occipital and parietal regions and left frontal region, as compared with suppression. The brain regions source analysis also revealed a focal source for the rehearsal condition in the left anterior temporal and parietal regions. Greater activation in the left temporal and frontal regions suggests the engagement of the phonological loop (Hwang, et al., 2005; Munk, et al., 2002; Scherg, 1992) and has also been implicated in the maintenance of non-spatial visual memory (Munk, et al., 2002). Importantly, the anterior temporal lobe is associated with learning and memory as well as language. Bilateral temporal cortex resection in patients with epilepsy has been associated with reduction in short-term memory abilities, especially those involving semantic content (Boucher, Dagenais, Bouthillier, Nguyen, & Rouleau, 2015). Left lateralized resection, as compared with right, has been shown to have greater impacts on learning of verbal information, short-term recall, and a greater rate of forgetting (Helmstaedter, Elger, Hufnagel, Zentner, & Schramm, 1996; Hermann, Wyler, Bush, & Tabatabai, 1992).

Additionally, the increased delay activity in frontal regions could represent the engagement of the central executive to support non-automatic recoding of difficult-to-label images, requiring increased attentional demand during rehearsal (Logie, 2011). Although, the phase-scrambled scenes contain the same visual features as regular scenes (e.g., color and spatial frequency), they lack the automatic semantic associations. The easy-to-label images used in Experiment 1, on the other hand, had an automatic semantic association and a verbal label (Ensor, et al., 2019; Logie, 2011). Thus, the generation of a label in Experiment 2 was more effortful (e.g., reliance on features (Verhaeghen, et al., 2006)) both during the recoding process and rehearsal and did not automatically produce a verbal label.

Engagement of occipito-temporal and parietal regions are associated with both visual working memory and attentional selection (Mayer, et al., 2007; Mayer, Kim, & Park, 2011). Parietal lobe involvement in maintenance has been linked to attentional selection (Munk, et al., 2002; Nelson & Reed, 1976) specifically with regards to novelty (Mayer, et al., 2011) and may also play a role in the integration of features in complex objects (Mayer, et al., 2007). Although, activation in posterior parietal cortex has also been associated with attention, it likely does not exclusively reflect attentional processes (Hutchinson, et al., 2009). If recoding with semantic associations requires increased attention, as in the case of a more effortful association between a stimulus and stored semantic knowledge, the central executive is likely to play an important role as reflected by increased activation of frontal networks. If attentional resources are not required, as in the case of an automatic association such as in Experiment 1, the central executive is less likely to play a role (Brown & Wesley, 2013). The results of these experiments confirm that controlling for maintenance strategy differentially recruits the prefrontal and parietal regions and modulates delay activity (Sreenivasan & D’Esposito, 2019).

4.3. Delay Activity and Complex Visual Stimuli

Elucidating how different patterns of delay activity contribute to WM is a current research focus (Sreenivasan & D’Esposito, 2019). While it has long been established that sustained activity observed during maintenance represents both maintenance of encoded information and focusing of attention inward, recent research has suggested that delay activity is more complex (Miller, et al., 2018; Rose, et al., 2016; Sreenivasan & D’Esposito, 2019). For example, only information in the focus of attention may be reflected in delay activity, while items outside the focus of attention may actually be represented by activity silent mechanisms (Rose, et al., 2016; Stokes, 2015).

Examination of the change in amplitude over time in both Experiments 1 and 2 suggests that when controlling for maintenance strategy, the pattern of delay activity is more transient. There is an early period of increased, synchronous activity (until approximately 3000 msec) with the most robust changes observed in the parietal and parieto-occipital regions, as has been previously reported in the literature (Berger, Omer, Minarik, Sterr, & Sauseng, 2014; Jensen, Gelfand, Kounios, & Lisman, 2002; Khader, Jost, Ranganath, & Rösler, 2010; Tuladhar, et al., 2007). The observed activity patterns were in the alpha and beta ranges, frequency ranges implicated in maintenance of visual information. Although most EEG studies have established synchronous alpha activity during the delay period in adults, this pattern has also been found in children (Sato, et al., 2018). Alpha enhancement in parietal regions likely reflects inhibition of information that could interfere with maintenance (Jensen & Mazaheri, 2010). The inhibitory role of alpha was established in WM studies that compare target stimuli with distractors (Waldhauser, Johansson, & Hanslmayr, 2012) and find large alpha activity associated with the distractor stimulus. Although a distractor stimulus was not present during the delay period, recruitment of parietal regions is critical for the maintenance of visual information to prevent interference of other information or brain regions. Increased beta activity is related to controlling the information that is being actively maintained (de Vries, Slagter, & Olivers, 2019), but may also represent similar underlying processes to alpha activity as they are often correlated (Hanslmayr, Staudigl, & Fellner, 2012; Pavlov & Kotchoubey, 2020). The early pattern of synchronous delay activity was followed by a period of desynchronous activity in the same frequency bands for the remainder of the delay period. This pattern of activity is often associated with visual stimulation during the encoding period of WM tasks (Heinrichs-Graham & Wilson, 2015; Jokisch & Jensen, 2007) as well as retrieval during the probe presentation (Heinrichs-Graham & Wilson, 2015; Klimesch, 1999). Desynchronous alpha and beta activity during the delay has been observed in other studies with complex visual stimuli (Ellmore, et al., 2017) and may suggest planning for the upcoming response during the probe presentation (Heinrichs‐Graham & Wilson, 2015).

The observed pattern of transient delay activity is consistent with recent reports that maintenance is not necessarily supported by persistent delay activity in prefrontal and parietal regions (Miller, et al., 2018; Sreenivasan & D’Esposito, 2019); instead, delay activity may reflect more complex processes in cortex and subcortical regions. Alternatively, previous reports of sustained delay activity could reflect a maintenance period in which participants did not utilize a particular strategy. Rather, they focused their attention inward until they were required to produce a response (Cowan, et al., 2005). We argue that this explanation is more likely for studies that do not instruct participants on what strategy to use. In consideration of the fact that many studies, which have established the hallmark delay activity signatures, used very short delay periods (~1–3 sec) it is plausible that the observed patterns of delay activity emerge with longer delay periods. While the early pattern of synchronous alpha activity in parietal regions reflects attentional processes critical for preventing interference (Jensen & Mazaheri, 2010), the extended maintenance period and later pattern of desynchronous alpha and beta activity may reflect a period of semantic elaboration and even reactivation of the encoded stimulus (Hanslmayr, et al., 2012).

We argue that greater effort is likely required to maintain difficult-to-name complex visual stimuli (e.g., phase-scrambled scenes), as compared with intact scenes, because the rehearsal process is not automatic as it is with intact scenes. Although this was not reflected in the experiment-level comparison between absolute amplitude, it can be argued that it is evident in the correlations between absolute amplitude and performance. In Experiment 2, increases in theta and alpha band activity in the left anterior temporal region were associated with increased performance with the application of rehearsal. Interestingly, performance also improved for some subjects that showed a decrease in theta and alpha band activity. The theta and alpha rhythms have long been implicated in working memory maintenance (Jensen, et al., 2002; Jensen & Tesche, 2002; Khader, et al., 2010). Theta has been found to play an important role in working memory maintenance, specifically in frontal regions (Jensen & Tesche, 2002), and may be related to focusing attentional resources on the to-be-remembered stimuli (Riddle, Scimeca, Cellier, Dhanani, & D’Esposito, 2020). More recently, theta has been associated with language processing (Covington & Duff, 2016), including language comprehension (Piai, et al., 2016), language generation (Hermes, et al., 2014), and semantic associations (Pu, Cheyne, Sun, & Johnson, 2020). More specifically, studies of semantic processing using sentence violation paradigms have found increases in theta power in the hippocampus, parahippocampus as well as the language-associated neocortical areas (Pu, et al., 2020) specifically with processing semantic meaning. An opposite pattern of decreases in theta power have also been reported in the hippocampus, specifically during generation of language (Hermes, et al., 2014). The findings in Experiment 2 are consistent with the recruitment of language-based regions to support articulatory rehearsal.

Although the Experiment 2 findings that both increases in theta and decreases in theta in the left anterior temporal region are related to improved performance may seem contradictory, the improvement in performance for both groups of participants (i.e., those with increases theta and those with decreases theta) may reflect individual application of the same technique. The cognitive operations involved generating a label to describe the picture followed by rehearsal of the label. Participants were encouraged to use meaningful labels, as in Experiment 1, but required more effort in the generation as the association was not automatic. For some participants, drawing on long-term semantic associations provided a behavioral advantage when rehearsal was applied, which was evident in increased theta band activity (Pu, et al., 2020). Whereas, for others more shallow, less meaningful labels were applied; therefore, the rehearsal period may simply reflect the generation of words that are rehearsed, as evidenced by a decrease in theta band activity (Hermes, et al., 2014). This method was equally as effective in the short-term, likely because of the low working memory load and short time period over which maintenance occurred.

4.4. Limitations and Future Directions

One limitation of the current study is that participants engaged in covert rather than overt rehearsal and suppression. Covert rehearsal was used to reduce the amount of noise introduced into the EEG signals. Task compliance was therefore based on participant confirmation during the recognition task (i.e., they reported their generated labels). It is possible that participants did not engage in suppression; consequently, the failure to produce the intended behavioral outcome in Experiment 1 could be explained by this. While the intended behavioral outcome was found in Experiment 2, it is again possible that participants did not engage in suppression. Future experiments that rely on covert maintenance mechanisms should include a response-based behavioral confirmation such as an immediate reporting of generated labels or the number of times the suppression word is repeated. Future studies should also examine the type of label generated to see if the depth of label (e.g., deeper descriptive label like “sunset” vs. shallower label like “white and black lines”) or the verbalizability of the stimulus (Brown & Wesley, 2013) impacts subsequent memory.

Another possible limitation to these findings is that the differences in performance (for Experiment 2) is the result of generating a label during the encoding phase of the suppression conditions in addition to rehearsing, as opposed to the rehearsal alone. Consequently, it can be argued that it is difficult to make conclusions about rehearsal versus suppression and the associated delay activity without adding a third condition in which participants generated a label and then suppressed during the delay period. We argue that our conclusions about rehearsal can be made with the current conditions because, even though during the suppression condition participants were discouraged from generating a label, they likely labeled the image automatically. Research on verbalizability of images has found that the automatic recoding of visual stimuli will occur even without explicit instructions to do so and even if the stimulus is difficult to verbalize or produce a label for (Postle, et al., 2005; Wright, et al., 1990). Thus, we argue that even for the difficult-to-label images used in Experiment 2, participants likely generated verbal labels automatically. Future research studies should include a condition in which labeling and suppression are combined to confirm that generating a label during encoding, especially with explicit instructions to do so, has no additional influence on delay activity and performance.

Building on the previous limitation, the fact that participants may have automatically labeled images regardless of the task instructions may explain the behavioral results of Experiment 1 as well. Participants may have been prevented from automatically labeling the images during the suppression condition if suppression was not restricted to the delay period of the WM task. Some studies that examine rehearsal with suppression of rehearsal instruct participants to carry-out the suppression during encoding as well as the delay period (Chein & Fiez, 2010; Souza & Oberauer, 2020) in order to prevent the ability to verbally recode. Therefore, if labeling was prevented during the encoding period, especially in Experiment 1, a clear advantage of rehearsal may have emerged. Future studies should examine if carrying-out suppression during encoding, particularly with easy-to-label visual stimuli, would reveal a benefit of rehearsal.

Another limitation of these experiments is that there was no verbal activity control condition. Participants either rehearsed or suppressed during the delay period for the working memory task, but there was no condition in which participants were not involved in a verbal activity to serve as a comparison delay period. A no-instruction control task was not included in the experimental design because of studies that suggest rehearsal will occur automatically. In fact, studies that have included control conditions in which instructions to rehearse were omitted (see Exp2 - Souza and Oberauer (2020)) have noted that participants reported spontaneous rehearsal even without instruction to do so. Unlike behavioral studies of rehearsal that have included control conditions (see Exp1 - (Thalmann, Souza, & Oberauer, 2019)), compliance was not able to be confirmed through assessment of session recordings as the maintenance strategies were covert in order to reduce introduction of noise into the EEG signal. Future studies that examine covert rehearsal should include a no-verbal activity control condition in which participants are confronted afterwards about which maintenance strategy was used, if any.

5. Conclusions

The results of these experiments provide behavioral evidence that rehearsal impacts short-term memory for some, but not all visual stimuli. Rehearsal also modulates the neural pattern of delay period activity as represented by the absolute amplitude of the delay period signal, especially in the left anterior temporal region. Changes in theta and alpha bands during the delay period correlate with working memory performance but only when maintaining labeled associations for stimuli that lack semantic content.

Highlights.

• Rehearsal may only benefit memory for visual stimuli lacking semantic information.

• Rehearsal modulates delay period activity for complex visual stimuli.

• Delay activity during a 6 sec maintenance period is transient.

• Increases in alpha and theta are related to improvement with rehearsal.