Dynamic cortical involvement in implicit anticipation during statistical learning

Abstract

The prediction of future events is fundamental in a large number of critical neurobehavioral contexts including implicit motor learning. This learning process relies on the probabilities with which events occur, and is a dynamic phenomenon. The aim of present study was to investigate the development of anticipatory processes during implicit learning. A decision making task was employed in which the frequency of trial types was manipulated such that one trial type was disproportionately prevalent as compared to the remaining three trial types. A 275 channel whole-head magnetoencephalography (MEG) system was used to investigate the spatiotemporal distribution of event-related desynchronization (ERD) and synchronization (ERS). The results revealed that oscillations within the alpha (10–12 Hz) and beta (14–30 Hz) frequencies were associated with anticipatory processes in distinct networks in the course of learning. During early phases of learning the contralateral motor cortex, the anterior cingulate, the caudate and the inferior frontal gyrus showed ERDs within beta and alpha frequencies, putatively reflecting preparation of next motor response. As the task progressed, alpha ERSs in occipitotemporal regions and putamen likely reflect perceptual anticipation of the forthcoming stimuli.

1. Introduction

The ability to extract regularities embedded in a random sequence of events plays a central role in human cognition. Behavioral evidence suggest that the process through which people become sensitive to regularities in the environment is automatic and occurs implicitly in the absence of awareness that one is learning [5]. In everyday situations many events follow one another on a probabilistic way rather than according to a fully predictable pattern. Studies of statistical learning processes have employed probabilistic sequential paradigms in which certain sequences of trials occur more frequently than others but do not follow a fixed sequence of events. In those studies the improvement of performance for stimuli that are highly frequent as compared to less frequent indicated that people implicitly learn statistical regularities [15,20]. The mechanisms underlying implicit learning are not fully understood, but most evidence suggests that an important process is the development of an automatic prediction of upcoming events based on the extracted regularities [10,12,17]. This has been especially true during implicit statistical learning [9,21]. However, though the ability to predict future events during implicit learning is well established, little is known about the brain circuits underling the development of such predictive processes. It has been suggested that implicit learning involves anticipation and preparation of the next motor response [6,10]. A recent neuroimaging study demonstrated that the medial temporal lobe and perceptual anticipation in visual areas play a critical role during implicit statistical learning [21]. However, the relatively poor temporal resolution of neuroimaging methods makes it difficult to track the subsecond time course of activation in different brain areas as people anticipate an upcoming stimulus. It has been suggested that implicit learning relies on the dynamic involvement of different brain areas over time suggesting that with experience people are increasingly able to make predictions about future events during the course of learning [3,13,19]. Therefore, it is conceivable that different sets of cortical regions are dynamically involved in implicit anticipation triggered by statistical learning. In the current study, we employed a neuroimaging technique with superior spatiotemporal resolution – magnetoencephalography (MEG). Event-related oscillatory responses can be quantified using the event-related desynchronization/synchronization method in which a relative decrease in the power in specific frequency bands during information processing is called event related desynchronization (ERD) and a relative increase in the power is called event-related synchronization (ERS). We adapted a Stroop-like task that we have previously employed [7]. Unbeknownst to the participants the frequency with which various trial types occurred was manipulated, such that one trial type was disproportionately prevalent in comparison to the other trial types. The aim of the current study was to evaluate the spatio-temporal proprieties of anticipatory oscillatory activity with regard to frequency of stimulus exposure, the progressive modulation of anticipatory oscillatory activity over time for the different exposure conditions, and to study whether possible anticipatory activity in specific brain regions was related to a change in behaviors. Based on previous results we expected that participants would exploit past experience with the statistical structure of stimulus sequences for successful anticipation of upcoming events. We predicted that motor as well visual regions might contribute to the formation of anticipatory behavior and the adjustment of participants’ predictions and anticipation over the course of the experiment. Previous studies have repeatedly demonstrated that activity changes within the alpha band (8–12 Hz) can be used as an indicator of localized brain activation in motor and visual regions in a variety of cognitive tasks including implicit learning [2,24]. We predicted that power changes in alpha may reflect anticipation of forthcoming stimuli to the extent to which participants had learned the predictability of the highly frequent stimuli. To test the specificity of the effects of anticipatory behavior on alpha band oscillations, additional frequency ranges were also included in our analysis.

Participants:

Eight healthy volunteers (4 women, 4 men; mean age 29.76; SD = 5.82) participated in this study. All participants were right handed native English speakers. None of the participants had a history of neurological or psychiatric disease. Informed written consent was obtained from all participants. The study was in accordance with the Helsinki Declaration and it was approved by the National Institute of Mental Health Institutional Review Board.

Behavioral procedures:

Participants were presented with a series of trials, each of which consisted of images of two animals of different sizes that were presented side by side on a computer monitor. In all trials, one image appeared larger than the other. The size of the animal images was either congruent or incongruent with the relative size in the real world. For example, a congruent stimulus was an image of a large elephant next to an image of a small ant, while an incongruent stimulus was an image of a large ant next to an image of a small elephant. Participants were instructed to press one of two buttons with their right hand to indicate which animal was larger in the real world, either the one on the left or the one on the right. The animal pairs were presented for 1 s, followed by an inter-trial interval in which a crosshair was presented for 1 s. Therefore, each trial lasted for 2 s. There were 300 trials in total, thus the entire experiment lasted for 10 min. We presented a disproportionately large number of trials (60%) with the larger, congruently sized animal appearing on the left side of space (congruent left). The other three trial types (congruent right, incongruent left, incongruent right) equally comprised the remaining 40% of the trial types. The trials were pseudo-randomized so that the probability rules (60% of the congruent left trial type and 40% of the other trial types) held in each successive quarter set of 75 trials each. No more than three of the similar type were consecutively presented. Participants were given no information about the rules by which the series of trials was constructed. They saw the series of trials presented one after the other without interruptions. To test the implicitness of learning participants were debriefed after the experiment.

Image acquisition procedures:

MEG signals were recorded in a magnetically shielded room using a helmet-shaped CTF 275-channel whole head system (CTF Systems Inc.) at a sampling rate of 600 Hz. Stimuli were presented to subjects via Presentation^® software (www.neurobs.com). The computer image was projected onto a screen 60 cm in front of the subject’s eyes. For each person, a series of volumetric MRI scans was co-registered with their MEG head coordinate system. Anatomical MRI scans were obtained using a 1.5 Tesla GE scanner.

Data analysis:

The raw data were filtered in 3rd gradient mode for noise reduction along with DC offset removal and high-pass filtering at 0.61 Hz. The presence of an adequate MEG signal was determined by visual inspection of the raw signal and trials that contain artifacts were rejected. The data were divided into four frequency bands: theta (5–8 Hz), lower alpha (8–10 Hz), upper alpha (8–12 Hz), beta (14–30 Hz), and gamma (30–50 Hz). As the principal interest in the present study was the prediction that the time course of the activation in the brain regions would differ at the beginning of the experiment as compared to the end, the most frequent trial type of congruent left were split into quarters of 75 trials each so as to index change across time. To achieve equality of frequencies of occurrence of the most frequent trials in each of successive quarter and the less frequent trials we compared each quarter of the most frequent trial type (congruent left) to all the trials of the less frequent trial type (congruent right). We chose the congruent right stimulus type because congruency was held constant; the only difference being the prevalence with which it was presented and which button was the correct response.

Synthetic aperture magnetometry (SAM):

SAM was used to reconstruct topographical maps of brain source power from band-limited MEG data [23]. A calculation of source power was performed for 7.5 mm³ voxels throughout the brain volume. For each participant, the SAM image was normalized to Z-scores by the pooled variance across the volume. For analysis of brain activity, only trials on which responses were correct were included. To define the spatiotemporal sequence of those cerebral regions active during different phases of the experiment SAM analysis were performed on the active epoch from −500 ms before the onset of the imperative stimulus to +1000 ms after stimulus onset. Trials were segmented using a sliding 300 ms time window moved with 100 ms increments. The reference interval of 300 ms was epoched from the inter-trial interval starting −800 to −500 ms before stimulus onset. The amplitude was obtained by computing a pseudo-F ratio between the power in the active and the reference interval. Band power changes were expressed as the percentage of a decrease (ERD) or increase (ERS) in band power. The averaged ERD/ERS were calculated for frequent and infrequent stimuli at each time window over the course of experiment, in each active epoch. Statistical analyses (t-tests) determined which voxels were significantly active at each time window with a permutation test used to take care of the problem that the voxels are not computed independently. The locations of peaks in the mean image were determined from AFNI’s Talairach atlas. Statistical analyses (ANOVA) determined which voxels were significantly active during the anticipation and the presentation of the most frequent trials relative to the infrequent trials. The random permutation analyses were performed to correct for multiple comparisons. The voxels with a p value < 0.05 were considered significant. CTF converted images were aligned into Talairach view using AFNI [8].

Correlations between brain activity and behavioral performance:

To examine the relationship between RTs and the anticipatory activation changes that were occurring during the course of sequence learning we used a region of interest (ROI) approach [1,21]. First, separate region of interest (ROIs) were created for each participant. The MRI scans were used to draw individualized ROI templates corresponding to a standardized coordinate frame from Talairach atlas. The regions of interest were set to include the whole brain, bilaterally, with 7.5 voxel mm resolution. These individualized ROI templates were used to interrogate the mean images series (corrected for multiple comparisons). The MEG signals for each time window preceding onset of the most frequent stimuli were extracted and then averaged for each participant and each ROI. We examined the relationship between these anticipatory activities (from −500 ms to 0) from each ROI and participants’ RTs difference between the most frequent and the infrequent trials for each frequency band. These brain behavior correlations helped to assess which anticipatory signals were related to facilitated response times on the most frequent trials over the course of experiment. These correlations were evaluated for significance using a threshold of p < 0.05

2. Results

Behavioral data:

The mean percent correct were 97.1% (SD = 1.4) and did not differ across the quarters (F(3,21) = 1.52, p = 0.2) nor between the most frequent and the infrequent trials (p = 0.1). Reaction time estimates were based only on correct trial response. A decrease in response times across the four quarters was observed for both the most frequent trials (F(3,21) = 4.36, p = 0.01) and infrequent trial types (F(3,21) = 3.73, p = 0.02. There was a significant difference in RT between trials that occurs more frequently and the less frequent trials (F(1,7) = 46.2, p = 0.0002). We confirmed this by using separate comparisons between the most frequent trials and each of the three types of the less frequent trials. There was a significant difference in RT between the most frequent trials and the congruent right trials (p = 0.05) as there were RT differences between the most frequent trials and the incongruent left (p = 0.005) and the incongruent right trials (p = 0.001), respectively. While the RT for the frequent trials was greater than that for the infrequent trials, this difference was more apparent during the initial learning (first quarter of the trials sequence) than during the late learning phase. As an exploratory analysis we examined the improvement in RT within the first quarter of the trials sequence. There was a significant difference (in all but three participants) between the first and second half of the first quarter for the frequent trials (p < 0.05). Conversely, none of the participants showed significant difference between these two blocks for the infrequent trials (p > 0.05). There was no significant interaction between frequency of stimuli presentation and the quarters (F(3,21) = 1.18, p = 0.3). There were considerable differences between the most frequent trial type and the less frequent trial type within the first (p = 0.07) and second quarter (p = 0.01) and fourth quarter (p = 0.04). There was no significant change in the mean RT between the most and less frequent trials in the third quarter (p = 0.1).

MEG data:

The main results of the experiment are presented in Fig. 1. Preceding the presentation of the most frequent visual stimuli we found two distinct ERD/ERS responses. First, in the first quarter of the trials sequence ERDs in the alpha 10–12 Hz frequency were observed at −500 ms before stimulus onset mainly in the right caudate and anterior cingulate. Additionally, ERDs in the alpha 8–10 Hz frequency were observed in the left inferior frontal gyrus and the left anterior cingulate. Beta rhythm ERDs were found (from −200 ms to +100 ms) in the left motor area and right premotor regions. Second, in the third and fourth quarters ERSs response were found in the alpha range 10–12 Hz (from −500 ms to 0) mainly in temporalparietaloccipital areas including the left middle temporal areas, the right middle occipital gyrus and the right inferior parietal areas. After the stimulus onset the presentation of the most frequent stimuli elicited alpha ERDs (10–12 Hz) response in the first quarter of trial sequence, in the left inferior frontal gyrus and caudate. In addition alpha ERSs response (10–12 Hz) were observed in the right and left sensorymotor areas and temporoparietaloccipital regions bilaterally. In the third and fourth quarters alpha ERDs were observed in the putamen, right and left inferior frontal gyrus and left superior temporal gyrus. The presentation of the infrequent stimuli elicited alpha ERD/ERS responses in a similar spatiotemporal profile. The overall ANOVA on the ERD/ERS data of the most frequent trials versus the infrequent trials revealed difference in the first quarter of trials sequence with a significant higher alpha ERDs preceding the presentation of the most frequent trials (from −500 ms to 0) in the right caudate and right cingulate in the alpha frequency 10–12 Hz and in the left inferior frontal gyrus in the alpha frequency 8–10 Hz. In the third and fourth quarters alpha ERSs responses (10–12 Hz) (from −400 ms to 0) were more pronounced in right parietaloccipital regions, left middle and temporal regions preceding the most frequent trials (p < 0.01).

Fig. 1.

(a) Group map corresponding to 300 ms interval (from −200 ms to +100 ms) preceding the most frequent trials during the initial phase of learning at beta frequency (Hz 14–30) (p < 0.01). PCG: precentral gyrus; PM: premotor area. b) Group map corresponding to 500 ms interval (from −500 ms to 0) preceding the most frequent trials during later phases of learning at alpha frequency (Hz 10–12) (p < 0.01). MTG: middle temporal gyrus; PL: parietal Lobe; MOG: middle occipital gyrus. The figures show data projected onto a Talairach space template.

Correlations:

The right middle occipital area exhibited activation (ERD/ERS) in the alpha frequency (10–12 Hz) that correlated (negative correlation) with RTs (RT differences between the most frequent trials and the less frequent trials) (p < 0.05).

3. Discussion

We sought to examine the development of neural anticipatory processes in the context of an implicit learning paradigm. For the most frequent trials, responses became quicker (compared to with infrequent trials) as the task progressed, indicating a rapid increase in task performance learning and efficiency. The main finding of this study is that anticipatory related changes in oscillatory activity occurred over time in anatomically separate brain areas. During the initial learning phase (first quarter of the trials sequence) a prominent activity within beta and alpha frequencies was found in a network of temporal and motor related areas. This activation decayed across the subsequent three quarters of the trial sequence. Subsequently, during the late learning phase (third and fourth quarters) alpha band responses were more pronounced in temporal and occipital regions. Participants were not instructed to consider stimulus sequence, nor did they report awareness of stimulus sequences after the imaging session. These activations must therefore reflect the automatic neural anticipation of events. Moreover, it should be noted that the cortical activations associated with the most frequent trials should be distinct from those associated with general expectancy aspects of the task, which have been subtracted by the sequence of infrequent trials. These findings correspond with a recent neuroimaging study of implicit learning that reported activation changes in cortical motor areas during early encoding whereas during late learning encoding activation changes were localized in occipitotemporal regions suggesting that two distinct networks, with different time courses, contribute to implicit learning processes, even in the absence of significant performance changes [19]. It should be noted that our results are consistent with those of other studies and suggest a dissociation between electrophysiological and behavioral indicators of implicit learning. Participants who reported that were unaware of the statistical structure of series and who showed no RT benefits for the most frequent trials as compared with the less frequent trials in the first and third quarters nevertheless showed considerable differences of brain activity. It is conceivable that behavioral and electrophysiological measures differ in their sensitivity. In our study, during the initial learning phase preceding the presentation of the most frequent trials, relative to the infrequent trials, higher alpha ERD values in the anterior cingulate, caudate, middle temporal gyrus (10–12 Hz) and left inferior frontal gyrus (8–10 Hz) were observed. These findings are compatible with the results of earlier studies that reported that alpha desynchronization responses were associated with anticipation of motor events [2] and implicit learning [24]. Additionally, beta ERD, which is usually considered a correlate of motor activation during motor preparation [2], was found in the contralateral motor cortex and the right premotor cortex. ERD/ERS in the alpha and beta frequencies can both represent a state of anticipatory processing capacity. Recently, Reber [18], drawing on numerous neuroimaging and neuropsychological studies, described implicit learning as a form of general plasticity within processing networks that adaptively improve function via experience. We expanded these findings suggesting that with experience processing networks develop automatic predictions about stimuli. As far as localization was concerned, these findings are in agreement with previous studies of implicit learning which reported that primary motor cortices play a significant role in the preparation of movements during procedural learning [10]. Several studies reported activation changes in the caudate and cingulate associated with the ability to extract predictability in a series of stimuli during implicit learning of spatial sequences [3,19]. The caudate is also engaged when participants are exposed to stimuli that violate a previously repeating pattern suggesting a role of this structure in the formation of implicit predictive models [15]. The present results suggest that the motor cortices and the motor related regions are engaged even prior to the presentation of the next stimulus and are in agreement with the hypothesis that learning might be based on the preparation of the next motor response [6]. The evidence that neural anticipation in cortical and sub-cortical motor areas emerged early during the initial learning phase suggest that motor anticipation can operate very quickly with little exposure to statistical regularities. Finally, the demonstration of changes in the left inferior frontal gyrus suggest that in the present experimental context this area, may not be specific to language aspects per se, but is also responsive to probabilistic features in time and engaged in predictive behavior. During the third and fourth task’s quarters we found a prominent anticipatory alpha ERS in the occipitaltemporal regions and in the putamen. Pre-stimulus alpha ERS in occipitotemporal regions have been reported preceding target detection probably reflecting expectancy processes [11]. These findings are in agreement with the results of previous electrophysiological [4] and neuroimaging studies [14,19,21] that reported that the striatum and the occipitotemporal regions are engaged during perceptual anticipation triggered by implicit statistical learning. It is attractive to speculate that activity in associative visual regions may reflect priming mechanisms, after repeated processing of the most frequent stimuli, which may be involved in implicit perceptual anticipation [4]. Indeed, the notable feature of the associative visual areas activation was the correlation between anticipatory activation and subsequent behavioral measures of learning. Although the present results provide convincing evidence for anticipatory processes, they do not allow us to decide whether these anticipations are based on the learning of stimulus sequence including stimulus–stimulus associations, memories of higher order chunks or by-products of the stimulus response relations [16,22]. It should be noted that in the present experiment participants would need to learn the probability of occurrence of one of four possible stimuli presented in a pseudo-random order and in a continuous stream. This random order makes it difficult to form predictions. Moreover, participants reported that they had not used any particular strategies to predict the next stimulus. We cannot rule out the possibility that they responded on each trial based on some sub-sequences of previous events occurring in the series including the previous predictions and the correctness of these predictions and/or to whole series of trials. We are aware of several limitations of the current study. First, our study is limited by the small sample, therefore the development of implicit anticipatory behavior remains to be rigorously tested with larger sample sizes. Second, ERD/ERS responses are calculated in reference to a prestimulus baseline, thus they might be influenced by the absolute frequency power in the prestimulus interval. Third, MEG is generally less sensitive to deeper sources. Therefore, although results yielded a confined topography of spectral amplitude enhancements in the basal ganglia, which corresponds well with the topography reported in various neuroimaging studies of implicit learning, conclusions about activity increases in those subcortical structures have to be drawn with caution. Fourth, we did not use off-line measures of statistical learning such as the familiarity test. However, anticipation may be a qualitatively distinct learning effect and the off-line familiarity test may be not well-suited for studying implicit learning [21]. In contrast, electrophysiological measures may serve as an on-line measure of the development of implicit learning that can be obtained continuously while participants perform implicit learning tasks.

4. Conclusions

We conclude that, based on the set of areas activated, motor and perceptual anticipation are both important during implicit statistical learning. The results of this study have potentially important implications for the role of implicit predictive processes in implicit motor learning. The data show that anticipatory behavior is a dynamic process implemented in the brain by a distributed network that involves anatomically dissociable components at different time points.

HIGHLIGHTS.

• Development of the prediction of future events during implicit statistical learning.

• Study is conducted using a neuroimaging technique with high temporal resolution: MEG.

• The spatiotemporal distribution of ERD/ERS is identified.

• Anticipatory behavior involves motor and perceptual anticipation at different time points.