Neural Correlates of Sequence Encoding in Visuomotor Learning

Abstract

To examine the neural basis of sequence learning, a fundamental but poorly understood human ability, we recorded event-related potentials (ERPs) while subjects viewed and memorized randomly directed sequences of motions for later imitation. Previously, we found that the amplitude of ERPs elicited by successive motion segments decreased as a function of each segment's serial position. This happened when subjects were required to remember the sequence, but not when they were performing a perceptual task. Here, to study the functional significance of this amplitude gradient in sequence learning, we presented each sequence several times in succession and examined changes in ERP amplitude as subjects learned the sequence through repeated observation and imitation. Behaviorally, with each repetition subjects grew more accurate in reproducing what they had seen. At the same time, ERPs grew smaller with each successive presentation, replicating and extending previous demonstrations of repetition suppression. Importantly, a comparison of ERPs to segments occupying different serial positions within a sequence revealed a decreasing amplitude gradient that grew steeper with sequence repetition. This sharpening of the amplitude gradient may reflect an explicit encoding process that relies on a magnitude code for serial order.

INTRODUCTION

A half century ago Lashley (1951) called attention to the importance of complex sequences of behaviors and to the challenges associated with trying to understand their underlying mechanisms. One particularly elusive question has been how the components of a learned sequence are organized in the right order, so as to avoid ambiguity between sequences that contain similar components (Keele et al. 2003). This problem is especially acute in the early stages of learning—the first exposures to the sequential information—before higher-level associations between items and their positions are formed (e.g., Marshuetz 2005). Many studies have used the serial reaction time (SRT) paradigm, in which subjects respond as quickly as possible to long sequences of signals, gradually improving their reaction times with repetition of the sequence. Several brain regions, including the basal ganglia, cerebellum, hippocampus, and prefrontal and motor regions, have been implicated in sequence learning by neuroimaging studies using the SRT task (Albouy et al. 2008; Grafton et al. 1995; Hikosaka et al. 1998). The low temporal resolution of the techniques those studies used to track brain activity, however, precluded an examination of differential processing of distinct items within the sequence being learned. Additionally, the implicit nature of the SRT task (subjects are often unaware of the repetition) makes it inappropriate for studying more explicit forms of learning, such as learning a sequence of body movements by imitation.

To track processing of sequential components on a finer timescale, we have carried out coordinated behavioral and electrophysiological studies in which subjects performed an imitation task, viewing and later attempting to reproduce randomly generated motion sequences (Agam and Sekuler 2007; Agam et al. 2005, 2007; Maryott and Sekuler 2009). In a recent study, we derived event-related potentials (ERPs) from scalp electroencephalographic recordings from subjects who were performing this imitation task (Agam and Sekuler 2007). ERPs were recorded during the task's observation period. Each ERP was time-locked to the onset of an individual segment in a sequence of motions made by a disc that moved on a display screen. As successive motion segments were seen, the amplitude of the ERP evoked by each segment declined as a function of the segment's serial position, producing what can be described as an amplitude gradient. This gradient occurred only when subjects were required to remember what they had seen; when subjects performed a perceptual task with no memory demands, the ERPs evoked by the same visual stimuli did not vary with segment serial order. In our earlier study each unique motion sequence was presented only once, which limited our ability to elucidate the functional significance of the changes in amplitude. The present study aims to shed light on the role of the ERP amplitude gradient in sequence encoding. To that end, we extended our electrophysiological investigations into the domain of learning, giving subjects multiple opportunities to observe and reproduce each motion sequence, while we looked for changes in ERP amplitude gradient that might accompany learning of the sequence.

Our goal was to determine how the amplitude gradient within a sequence might change with learning, as the sequence is viewed and encoded multiple times. We previously proposed (Agam and Sekuler 2007) two possible explanations for the serial-order dependence of the ERP amplitude. One hypothesis asserts that the amplitude gradient is a result of an uneven distribution of cognitive resources. Under this hypothesis, early segments benefit from elevated attention relative to later segments because encoding of later segments must be performed in parallel with rehearsal of the already-seen segments (Sederberg et al. 2006). Increased attention is known to have a positive influence on the amplitude of visual ERPs (Awh et al. 2000; Hillyard and Münte 1984; Hillyard et al. 1998; Luck et al. 2000). A decrement in attention, then, should manifest itself as reduced ERPs to segments in later serial positions. If this account were true, then repetition of the entire sequence should decrease, or “flatten” the amplitude gradient, as processing demands decrease with better familiarity with the already-seen sequence. An alternative hypothesis asserts that rather than reflecting diminishing resources, the decreasing amplitude represents an explicit encoding mechanism. We specifically suggested that the amplitude gradient indicates the construction of what has been termed a “primacy gradient” (Page and Norris 1998)—i.e., a magnitude-based code for serial order, in which the recall and production of items in a sequence are ordered according to the strength of activation in items' respective neural representations (Botvinick and Plaut 2006; Bullock and Rhodes 2003; Farrell and Lewandowsky 2002; Grossberg 1978; Page and Norris 1998). In contrast to an explanation that focuses on “limited resources,” this second account implies that the amplitude gradient should intensify, or at least persist, across repeated presentations of the sequence.

Our primary interest, then, was in changes in the amplitude of ERPs to segments at different positions within the sequence. The picture would not be complete, however, without also considering the baseline ERP amplitude and its possible change with learning. More specifically, if sequence encoding depends on amplitude changes, the code is only as reliable as the signal-to-noise ratio in the underlying signal; the relative change in amplitude is critical. A common and consistent observation in electrophysiology and neuroimaging is that when a stimulus is repeated, it elicits weaker neural responses compared with its first appearance, a phenomenon known as “repetition suppression” (RS; for review, see Grill-Spector et al. 2006). One should expect, then, that with repetition of the entire sequence, ERPs to individual segments will be attenuated relative to ERPs to segments in a novel sequence. Therefore we examined ERP amplitude changes on two different scales: one that considered changes in ERP amplitude within a single sequence and another that considered changes across sequences.

METHODS

Subjects and procedure

Fourteen human subjects (10 female, age range: 18–30 yr, all right-handed) participated after providing written informed consent to a protocol approved by the institutional review board of Brandeis University. The experimental paradigm was identical to that of our previous learning experiment (Agam et al. 2007), only here each stimulus sequence was presented three consecutive times on each trial. Each quasi-random motion stimulus was generated by the steady movement of a white disc (1° visual angle in diameter) against a black background on a computer screen, which subjects viewed from a distance of 57 cm. The disc moved along a series of five connected, straight segments, each 1.5° visual angle long. Each segment took 525 ms to complete, followed by a 225-ms pause between segments during which the disc was visible and stationary. The directions of the motion segments were randomized under two constraints. First, to minimize verbal encoding that could be used if the disc's path corresponded to the outline of some familiar object (Sekuler et al. 2003), segments could not intersect one another or even approach one another closer than 0.75° visual angle, half the length of a segment; second, the angle between consecutive segments was a random value drawn from a uniform distribution spanning 30 to 150°. After completing the series of motion segments, the white disc disappeared and a 3.75-s retention interval ensued. At the end of the retention interval, a colored disc appeared on the screen, prompting subjects to begin their imitation by drawing with a stylus on a graphic tablet. The location of the colored disc on the screen was yoked with a 1:1 ratio to the position of the stylus on the tablet, moving along with the stylus as the subjects were performing the imitation. The path traveled by the stylus was saved after each imitation for off-line analysis. Subjects were instructed to reproduce the motion trajectory as accurately as possible, but were not informed about the metrics that would be used for analysis. They were asked to reproduce the appropriate number of segments (five) and, to facilitate off-line automatic segmentation, to produce segments that were as straight as possible. More details of stimulus generation and the imitation task are available elsewhere (Agam and Sekuler 2007, 2008; Agam et al. 2005, 2007).

Figure 1 is a schematic diagram of the learning task (see also Supplemental Video Clip S1, showing the events comprising a typical trial).¹ Because we were interested in ERPs that accompany learning over repeated presentations, each sequence was presented three times. After every presentation, subjects tried to reproduce the motion sequence. The color of the disc representing the stylus changed systematically over successive presentations of the same sequence: red for the initial presentation, then green and blue for the second and third presentations, respectively. After completing the final imitation, subjects were shown color-coded line drawings of their three attempts of imitation, superimposed on a white line drawing of the entire path that had been traversed by the stimulus disc. Note that this was the first time subjects actually saw the entire path all at once; prior to that, the path was available only as the subjects' memory of individual motion segment directions. Each subject performed 192 trials (each comprising three presentations) over the course of four experimental sessions.

Fig. 1.

Schematic diagram of the imitation task (see also Supplemental Video Clip S1). Note that the actual display background was black and the color of the disc during stimulus presentation was white. The disc never left a trace, so subjects saw only its instantaneous position; the dashed lines are for illustration purposes. The feedback image is enlarged for clarity.

Behavioral data analysis

To score the accuracy of each imitation, a segmentation algorithm used temporal and spatial criteria to automatically break down each imitation into its component segments. In brief, the algorithm searched for points where the stylus stopped moving or where it changed its direction of motion. The extracted segments were then extrapolated by connecting the chosen breakpoints using straight lines (for more details of the algorithm and for comparison to other potential measures of imitation fidelity, see Agam et al. 2005). For inclusion in the analysis, we required that the number of segments in the reproduction match the number of segments in the stimulus in at least two of the imitations in a given trial. Otherwise, the entire trial (all three presentations) was excluded from analysis. For each reproduced segment identified by the algorithm, the error in reproduction was defined by the absolute difference in orientation between that segment and the corresponding segment in the stimulus. The smaller the orientation difference, the more accurately a segment had been reproduced from memory.

Electrophysiological recording and analysis

We recorded electroencephalographic signals using an Electrical Geodesics (Eugene, OR) system with 129 electrode sites, including two vertical, bipolar channels above and below each eye and one horizontal, bipolar eye channel. Data were recorded at 250 Hz. Data were cleaned of bad channels and re-referenced to the grand average using BESA (MEGIS Software, Munich, Germany) and were further analyzed using Matlab (The MathWorks, Natick, MA). For all analyses, data were notch-filtered at 60 Hz. For analysis of ERPs to motion segments, data were high-pass filtered at 1 Hz and low-pass filtered at 30 Hz. Blink artifacts were eliminated by rejecting epochs in which the difference between the maximum and minimum potential at any channel during an entire trial exceeded 150 μV. When such an artifact was found, the entire trial (including all segments) was excluded from analysis.

ERPs to the visual motion stimulus were measured by averaging signals from six posterior electrodes at the following 10/20 locations: P3, Pz, P4, O1, Oz, O2 (for a similar approach, see Vogel and Machizawa 2004; Vogel et al. 2005; Zhang and Luck 2009). We chose these electrodes because our previous study (Agam and Sekuler 2007) found the biggest amplitude gradient in posterior electrodes and because ERPs time-locked to visual stimuli are likely to be most pronounced at occipital and parietal sites. Since our visual stimuli were not lateralized, we did not examine left and right hemispheric ERPs separately. Data were averaged across trials for each subject and each subject's mean ERP was used for statistical analysis. Data were baseline-corrected by subtracting the mean potential during the first 100 ms of each segment's waveform. We did not use the common practice of using the period prior to the stimulus as baseline because that period, which corresponds to the end of the previous segment, would be inappropriate as a baseline when contrasting segments against each other. Note that all the statistical analyses were performed on the ERP amplitude (difference between minimum and maximum points), which is independent of the baseline.

ERP amplitude was quantified as the difference between the maximum and minimum potentials throughout the entire duration of a segment (750 ms). Because such an amplitude measure is inherently nonnormal, we used nonparametric statistical tests to assess the reliability of the amplitude differences across segment serial position and across repetitions of the entire sequence (for other examples for use of nonparametric statistics with ERP data, see Price 2000; Stroganova et al. 2007). We carried out three independent bootstrap procedures: one to evaluate the within-sequence amplitude gradient, one to evaluate changes in overall amplitude with repetition, and one to evaluate how within-sequence amplitude changes were affected by repetition. To examine the gradient within a sequence, we created 10,000 random data sets reflecting the null hypothesis that the slope of the amplitude regressed against serial position would be zero. For each random data set, serial position was randomly assigned (with replacement) to each segment's ERP on each trial. In each data set, the amplitudes of subjects' average, randomized ERPs were regressed against serial position, yielding a distribution of 10,000 regression slope values. The position, within the bootstrap distribution, of the slope derived from regression of the real data was used to determine the probability that such a slope would occur by chance. The effect of repetition on the mean ERP amplitude was tested using a similar procedure, but with random assignment of presentation order rather than serial position. To test the effect of sequence repetition on the amplitude gradient, we calculated the serial position-based slope within each repetition for each subject. These slopes were then randomly assigned to repetitions (but not shuffled between subjects), reflecting the null hypothesis that the slope of the amplitude gradient remains constant with repetition. This process was repeated 10,000 times to produce a distribution of regressions of amplitude slopes against presentation order. As in the other two bootstrap procedures, the actual result's position within the bootstrap distribution defined the probability that the result (e.g., increase in amplitude gradient slope with repetition) may have occurred by chance. All P values reported here for nonparametric tests reflect the number of simulations out of 10,000 that exceeded the slope of the real data (divided by two to adjust for a two-tailed test).

RESULTS

Behavioral results

Behavioral performance was defined by the accuracy with which the orientation of each segment in the original trajectory was reproduced in the imitation. Figure 2A shows the results of that analysis. We replicated previously reported serial position effects (Agam and Sekuler 2007; Agam et al. 2005) within a presentation [F(4,13) > 15.5, P < 10⁻⁷ for all presentations, repeated-measures ANOVA with factor serial position]. In addition, learning over sequence repetitions was clearly evident in the improvement in overall imitation accuracy (mean error across all five segments; Agam et al. 2007) following each presentation [F(2,13) = 69.5, P < 10⁻⁸, repeated-measures ANOVA with factor presentation]. Improvement was significant both between presentations 1 and 2 [t(13) = 9.56, P < 10⁻⁶] and presentations 2 and 3 [t(13) = 5.09, P < 0.001].

Fig. 2.

Behavioral results. A: accuracy of imitation. Segment orientation error is plotted against segment serial position for each one of the stimulus presentations. Error bars are within-subject SE (Cousineau 2005; Loftus and Masson 1994). B: practice and sequence-specific effects. Each group of bars represents data from one experimental session (numbered chronologically). Ordinate values are mean error across all segments in the stimulus sequences. Red, green, and blue bars denote the first, second, and third sequence presentations, respectively. Error bars are within-subject SE for each session.

It is important to distinguish between sequence-specific, repetition-based learning and general improvement in task performance with practice. This can be done by looking at how performance varies between different stages in the experiment. Using a similar learning paradigm and dividing trials according to chronological order, we have previously shown (Agam et al. 2007) that learning is trial-specific, i.e., orientation error resets to its initial value with each new sequence. We repeated this analysis with the current data set. Figure 2B shows mean orientation error across all segments for each presentation during each of the four experimental sessions. Although a repeated-measures ANOVA across sessions revealed a significant difference in error between sessions [F(3,13) = 14.2, P < 10⁻⁵], the only significant pairwise difference was between the first and second sessions [t(13) = 3.55, P = 0.004]. Improvement with sequence repetition, on the other hand, was significant during all sessions [F(2,13) = 43.9, 51.34, 36.0, 30.5, respectively; P < 10⁻⁶ for all sessions, ANOVA]. So, although the overall difference between performance in session one and performance in the other three sessions does show a general practice effect, the effect of repeating a particular sequence is strong and can be seen within each and every session.

Subjects' imitation slowed down with repetition: the time taken to complete the entire imitation was 3.34, 3.54, and 3.62 s for presentations 1 to 3, respectively [F(2,13) = 14.5, P < 10⁻⁴, ANOVA]. There was no difference between presentations in reaction time [time the stylus first touched the tablet; F(2,13) = 0.64, P = 0.54, ANOVA] or in the mean length of reproduced segments [F(2,13) = 2.4, P = 0.11, ANOVA].

ERP analysis

The number of different sequences included in the analysis was on average 181 per subject. There was no difference in number of trials between the three presentations of each sequence [F(2,13) = 0.02, P = 0.98, ANOVA]. ERPs were measured from a set of recording sites covering early visual areas (see methods) while subjects observed the motion sequences. Figure 3 shows ERPs for the entire sequence in each of the three presentations, time-locked to the onset of the first segment, using different filtering parameters. A negative slow wave accompanied the first presentation and decreased with subsequent repetition of the sequence (Fig. 3, A and B). This is consistent with previous reports of low-frequency negativity that is positively correlated with memory load (Mecklinger and Pfeifer 1996; Vogel and Machizawa 2004; Vogel et al. 2005). To study ERPs specifically evoked by individual motion segments, we used a 1-Hz high-pass filter that eliminated the slow-wave activity (Fig. 3C). As can be appreciated from Fig. 3A, the ERP associated with the first segment coincided with a large positive deflection. We interpret this deflection as a response to stimulus onset and/or changes in arousal (Agam and Sekuler 2007). Unlike ERPs to subsequent segments, the ERP to the initial segment could not be reliably measured (Fig. 3C) and its inclusion could have produced spurious results in comparisons of segment amplitudes. Therefore all further analyses included only segments from the second to fifth positions. All subsequent figures are based on data filtered between 1 and 30 Hz (Fig. 3C).

Fig. 3.

Slow-wave and evoked responses to motion sequences, time-locked to the onset of the first motion segments. A: 0–30 Hz. B: 0–0.5 Hz. C: 1–30 Hz. The red, green, and blue traces correspond to the first, second, and third presentations of each sequence, respectively. Vertical dashed lines indicate the onset of each motion segment.

Average ERP amplitude across segments

Our first analysis of the ERPs focused on how repetition of the stimulus sequence affects the average ERP amplitude. To determine the average ERP for each presentation, we collapsed the ERPs to motion segments 2 to 5 in every sequence into a single waveform and compared the corresponding waveforms from each of the three presentations. Figure 4A shows the aggregate waveform for each presentation and Fig. 4B shows the mean ERP amplitude (difference between most positive and most negative points) as a function of stimulus presentation order. Amplitude fell sharply following repetition of the stimulus sequence. There was a significant difference in ERP amplitude between presentations (P < 0.0001, bootstrap). Paired comparisons of successive presentations revealed a significant difference in amplitude between presentations 1 and 2 (P < 0.0001) but not between presentations 2 and 3 (P = 0.21).

Fig. 4.

Mean amplitude changes with sequence repetition. The left panel shows average event-related potentials (ERPs), filtered between 1 and 30 Hz, across 4 segments (at positions 2 to 5) of each sequence. The red, green, and blue traces correspond to the first, second, and third presentations of each sequence, respectively. The right panel shows the mean amplitude (difference between positive and negative peaks) of the ERPs for each repetition. Error bars represent within-subject SE.

Amplitude changes within the sequence

To examine ERP amplitude changes within each presentation of a sequence, we superimposed ERPs to individual segments at different serial positions and compared their respective amplitudes. Figure 5A shows the ERP waveforms for the four stimulus segments that occupied serial positions 2 to 5. Because each waveform was derived from a smaller amount of data compared with the aggregate waveforms in Fig. 4 (one quarter), these waveforms exhibited more variability. Figure 5B shows ERP amplitudes for each of the successive segments during each of the three presentations (see also Supplemental Fig. S1 for results of the same analysis at the five midline recording sites). Within-sequence decreases in amplitude were not diminished by stimulus repetition, but became steadily more pronounced. Whereas in the first presentation the amplitude gradient just approached significance (P = 0.06, bootstrap), the gradient was highly significant in the second and third presentations (P < 0.0001 for both). Most critically, we detected a significant interaction between presentation order and segment serial position (P = 0.04, bootstrap; see methods), confirming the relationship between sequence repetition and the amplitude gradient. Figure 5C shows changes in amplitude in each of three presentations relative to the second segment. In this case the gradient grew sharper during repeated presentations and the interaction between presentation and serial position was highly significant (P = 0.002, bootstrap).

Fig. 5.

Amplitude changes within repeated sequences. A: ERP waveforms for motion segments at serial positions 2 to 5, during the first (left), second (center), and third (right) sequence presentations. Data were filtered between 1 and 30 Hz. Each color corresponds to a segment at a different position. B: mean ERP amplitude (difference between positive and negative peaks) for each segment and presentation. C: ERP amplitude for each segment and repetition normalized to the amplitude of the ERP to the second segment in each presentation. Red, green, and blue lines denote first, second, and third presentations, respectively. Error bars represent within-subject SE for each repetition in B and the SE in C.

Relation of the amplitude gradient to behavioral performance

To assess how the amplitude gradient is related to imitation performance, we examined how individual subjects' success in reproducing the sequence was related to the strength of the gradient. We divided the subjects (median split) into better and worse performers based on mean segment orientation error at the end of the learning process, i.e., following the third imitation attempt (Fig. 6A). Figure 6B shows ERPs to segments at different positions for each group, also based on the third presentation. The amplitude gradient (Fig. 6C) was significant for the better-performing group (P < 0.0001), but not for the worse performers (P = 0.15, bootstrap). The group × serial position interaction in the ERP amplitude approached significance (P = 0.07, bootstrap), in spite of the small size of the groups, suggesting a link to behavioral performance: the sharper the gradient, the more accurate the reproduction of the sequence.

Fig. 6.

Relationship between amplitude gradient and behavioral performance. A: accuracy of the third and final imitations. Subjects were divided into 2 groups of equal size (7 subjects) based on mean orientation error: low error (green) and high error (red). B: ERPs to segments 2 to 5 in the third presentation for the better performers (left) and the worse performers (right). C: ERP amplitude (difference between positive and negative peaks) for each segment in the third presentation for each group. Error bars represent within-subject SE for each group.

Stimulus-specific learning was accompanied by a modest degree of general improvement with practice (Fig. 2). To ensure that the sharpening of the amplitude gradient was due to stimulus-specific learning and not to general practice effects, we obtained average ERPs for each session separately and examined changes in the ERP amplitude gradient across experimental sessions. In this case, we averaged across the three presentations of each sequence. A significant amplitude gradient was observed in sessions 1, 2, and 4 (P values for sessions 1–4: 0.0001, 0.0002, 0.14, 0.04; bootstrap). The slope of the amplitude gradient did not change significantly across sessions (P = 0.71, bootstrap), suggesting that general improvement with practice was not a substantial cause for changes of the amplitude gradient. We carried out a second control analysis to assess the importance of the speed–accuracy trade-off with respect to the ERP amplitude gradient. As mentioned earlier, the time subjects took to complete the reproduction of the presented sequence increased with repetition. In a manner analogous to the analysis in Fig. 6, we divided subjects into two groups according to the speed with which they completed the third imitation and compared the amplitude gradients between the two groups. Although both groups showed a significant decline in ERP amplitude with segment serial position (P = 0.0004 for the faster group, P = 0.05 for the slower group), the slopes of the gradients did not differ (P = 0.56).

DISCUSSION

Learning with repetition

Consistent with prior reports (e.g., Agam et al. 2007), subjects showed robust improvement in imitation accuracy with each repetition of the stimulus sequence. We also replicated the finding that subjects slow down their imitation with each repetition. We previously interpreted this as an indication of the presence of a temporal component in subjects' representation of a sequence, rather than simply a static shape (see Agam et al. 2007).

Repetition suppression

The decline in average ERP amplitude with sequence repetition (Fig. 4) is consistent with previous demonstrations of RS (e.g., Grill-Spector et al. 2006; Henson 2003; Henson and Rugg 2003; Henson et al. 2000, 2004; Maccotta and Buckner 2004; Schacter and Buckner 1998), but extends the range of stimuli known to elicit RS; the decreased response to an entire, temporally extended sequence of visual stimuli has not been demonstrated previously. Our results also imply that stimuli need not be familiar, i.e., to have preexisting neural representations, to produce RS, as has been suggested (Henson et al. 2000), but can comprise a preexperimentally unfamiliar set of motions.

Sharpening of the amplitude gradient

The amplitude gradient strengthens with repetition and does so in parallel with improved imitation accuracy. This relationship supports the claim that changes in ERP amplitude arise from a process that plays a functional role in sequence encoding. This strengthening of the amplitude gradient occurred despite the overall attenuation of ERPs during repeated presentation compared with the first observation of each sequence.

This result seems at odds with the most straightforward prediction of the first hypothesis introduced earlier—that the amplitude gradient reflects position-dependent changes in the availability of attentional resources. Such a situation would have arguably produced a weaker, not a stronger, gradient with repeated observations because familiarity with the sequence reduces the amount of cognitive resources needed to rehearse the sequence. If the amplitude gradient indeed reflects reduced attention, its sharpening would suggest that subjects emphasize early serial positions more with repeated presentation than with the novel sequence. This seems unlikely because the improvement in accuracy (Fig. 2A) is quite uniform for all serial positions. One caveat, however, in rejecting this hypothesis is that the baseline level is different between the first and the other presentations. Attentional enhancement may have hit some limit in the early segments of the first repetition, thus reducing the slope of the amplitude gradient.

If the second hypothesis were true and the amplitude gradient serves an explicit role in sequence encoding, what could that role be? One intriguing possibility is that it reflects the construction of a “primacy gradient” in working memory, i.e., a graded pattern of activation strengths, ordered by the serial position of each activated item (Botvinick and Plaut 2006; Bullock and Rhodes 2003; Farrell and Lewandowsky 2002; Grossberg 1978; Page and Norris 1998; Rhodes et al. 2004). Although it has been repeatedly validated by computer simulations, the magnitude code's physiological origin remains unknown. Specifically, it is not clear when in the learning process it can be constructed. It may be that differences in activation between sequence components occur as early as when the sequence is presented. Consider the following: the subject knows that he or she has to remember the seen motion segments in their correct order. Therefore the steady decrease in the amplitude of evoked visual responses could reflect an adjustment of neural activity in service of a magnitude code, so that during the retention interval activation strengths for successive segments mirror the serial order of the segments.

One clear difference between our behavioral and ERP results is that although subjects showed a slight improvement in imitation accuracy at the fifth segment (Fig. 2A), ERP amplitude continued to decline (Fig. 5). It is important to note that theoretical models of magnitude coding attribute the behavioral recency effect to reduced interference from neural representations of neighboring items in the sequence: item representations are activated at different strengths mirroring serial order, with the last item's activation being the weakest. Because of a noisy magnitude coding mechanism, the activation of item n + 1 might exceed that of item n, a situation that would lead to a transposition error. Whereas middle items are prone to interference from two adjacent items, one before and one after, the first and the final items have only one adjacent, interfering neighbor. This leads to a smaller probability of an item exchange error due to noise at the edges of the sequence and thus to a one-item recency effect in the error curve (Agam et al. 2005; Page and Norris 1998). Our current results are not inconsistent with such an account. Another way in which our ERP results do not mirror behavioral learning effects is that they seem to plateau after the second presentation; both RS and amplitude gradient changes were minor between the second and third presentations, but behavioral performance continued to improve following the third presentation. We think our ERP measures may not have been sensitive enough to capture changes in brain activity associated with improvement on the third presentation, which was small relative to the initial improvement, between the first and second presentations.

Because of the lack of previous reports in the literature, we view the magnitude code interpretation as preliminary. First of all, predictions of the theoretical model with respect to evoked visual responses are unclear. Furthermore, some caveats about our data must be taken into account: EEG signals reflect electrical activity pooled across a large number of neurons, so one cannot make inferences about spiking rates or the spatial spread of spiking activity. The degree of noise in our data precluded an examination of imitation accuracy and order errors on a trial-by-trial basis, which could have strengthened the link to behavior. Finally, our design did not allow an analysis of the amplitude of the response to the first segment, which is obviously critical in any magnitude coding scheme. Our findings, then, could be used as the foundation for further study using other, complementary methods such as monkey electrophysiology to examine the neural basis of sequence encoding.

Summary

With repeated viewing of a stimulus motion sequence, the corresponding ERP amplitudes undergo two distinct, but concurrent changes: the primacy gradient is strengthened at the same time that the overall, mean signal level declines. How might these two modes of amplitude change, across sequence presentations and across segments within a sequence, be related? RS is known to correlate with improved behavioral performance (Grill-Spector et al. 2006; Maccotta and Buckner 2004) and has been linked to heightened predictability of a stimulus (Summerfield et al. 2008). Given the nature of our paradigm, it is entirely plausible that the decline in amplitude with sequence repetition reflects improved ability to predict the future path of the disc once its path had already been seen. Such predictability would free cognitive resources that, during the initial presentation, were needed for initial processing of the basic physical properties of the sequence, but could now be devoted to ordering successive segments more effectively. In other words, we suggest that the decline in mean signal strength (RS) reflects reduced processing demands, which lead to improved representation of the sequence in memory, as manifested by changes in amplitude with segment serial position and with more accurate reproduction of the sequence.

GRANTS

This work was supported in part by National Science Foundation–sponsored Science of Learning Center/Center of Excellence for Learning in Education, Science, and Technology Grant SBE-0354378 and National Institute of Mental Health Grant MH-068404.