Prediction error and repetition suppression have distinct effects on neural representations of visual information

Matthew F Tang; Cooper A Smout; Ehsan Arabzadeh; Jason B Mattingley

doi:10.7554/eLife.33123

. 2018 Dec 14;7:e33123. doi: 10.7554/eLife.33123

Prediction error and repetition suppression have distinct effects on neural representations of visual information

Matthew F Tang ^1,^2,^✉, Cooper A Smout ^1,², Ehsan Arabzadeh ^2,³, Jason B Mattingley ^1,^2,⁴

Editors: Christopher Summerfield⁵, Timothy E Behrens⁶

PMCID: PMC6312401 PMID: 30547881

Abstract

Predictive coding theories argue that recent experience establishes expectations in the brain that generate prediction errors when violated. Prediction errors provide a possible explanation for repetition suppression, where evoked neural activity is attenuated across repeated presentations of the same stimulus. The predictive coding account argues repetition suppression arises because repeated stimuli are expected, whereas non-repeated stimuli are unexpected and thus elicit larger neural responses. Here, we employed electroencephalography in humans to test the predictive coding account of repetition suppression by presenting sequences of visual gratings with orientations that were expected either to repeat or change in separate blocks of trials. We applied multivariate forward modelling to determine how orientation selectivity was affected by repetition and prediction. Unexpected stimuli were associated with significantly enhanced orientation selectivity, whereas selectivity was unaffected for repeated stimuli. Our results suggest that repetition suppression and expectation have separable effects on neural representations of visual feature information.

Research organism: Human

Introduction

At any moment in time, the brain receives more sensory information than can be responded to, creating the need for selection and efficient processing of incoming signals. One mechanism by which the brain might reduce its information processing load is to encode successive presentations of the same stimulus in a more efficient form, a process known as neural adaptation (Fairhall et al., 2001; Kvale and Schreiner, 2004; Smirnakis et al., 1997). Such adaptation has been observed across different sensory modalities and species, and has been suggested as a potential mechanism for enhancing the coding efficiency of individual neurons and neuronal populations (Adibi et al., 2013; Benucci et al., 2013; Maravall et al., 2007). A particular form of neuronal adaptation, known as repetition suppression, is characterised by attenuated neural responses to repeated presentations of the same stimulus (Diederen et al., 2016; Gross et al., 1967; Keller et al., 2017; Movshon and Lennie, 1979; Rasmussen et al., 2017). Here, we asked whether predictive coding theory, which assumes that sensory processing is influenced by prior exposure, can account for changes in neural representations observed with repetition suppression.

The phenomenon of repetition suppression has been widely exploited to investigate neural representations of sensory information. Repeated exposures allow for more efficient representation of subsequent stimuli, as manifested in improved behavioural performance despite a significant reduction in neural activity (Henson and Rugg, 2003; Schacter and Buckner, 1998). Repetition suppression paradigms have been used extensively in human neuroimaging because they are commonly considered to be analogous to the single-cell adaptation effects observed in animal models (see Barron et al., 2016 for review). The exact relationship between the effects seen in human neuroimaging studies and animal neurophysiology has, however, yet to be fully established.

The view that repetition suppression observed in human neuroimaging studies reflects neuronal adaptation has recently been challenged by hierarchical predictive coding theories (Auksztulewicz and Friston, 2016; Summerfield et al., 2008). These theories argue that the brain interprets incoming sensory events based on what would be expected from the recent history of exposure to such stimuli (Friston, 2005; Rao and Ballard, 1999). According to these theories, predictions are generated within each cortical area and are bi-directionally propagated between higher and lower areas, including to primary sensory regions, allowing for more efficient representation of expected stimuli. When there is a precise expectation, incoming information can be efficiently represented by recruiting a small pool of relevant neurons (Friston, 2005). When there is a mismatch between an expectation and the stimulus presented, that is, when there is a prediction error, the stimulus is less efficiently represented and thus elicits a larger neural response.

The majority of evidence for predictive coding comes from human neuroimaging experiments in which the presentation of an unexpected stimulus generates a larger response than the presentation of an expected stimulus. In studies employing electroencephalography (EEG) and magnetoencephalography (MEG), this effect is known as the mismatch negativity (Garrido et al., 2009; Näätänen et al., 2007; Wacongne et al., 2011), where an unexpected stimulus evokes significantly greater negativity than an expected stimulus. To date, however, no study has tested a key premise of predictive coding, namely, that expected stimuli are more efficiently encoded in the brain relative to unexpected stimuli, in terms of their elementary feature representations. Nor has any previous investigation examined whether the mismatch negativity response is associated with a change in neural tuning to stimulus features such as orientation.

To test the hypothesis that prediction error can account for repetition suppression effects, Summerfield et al. (2008) introduced an experimental paradigm in which the identity of a face stimulus was either repeated in 80% of trials (making the repetition expected) or was changed in 80% of trials (making the repetition unexpected). There was greater attenuation of the BOLD response in the fusiform face area when a face repetition was expected, relative to when it was unexpected, suggesting that repetition suppression is reduced by unexpected stimuli. This attenuation of repetition suppression by failures of expectation has also been replicated using fMRI (Larsson and Smith, 2012) and M/EEG, using high-level stimuli such as faces (Summerfield et al., 2011), and simple stimuli such as auditory tones (Todorovic and de Lange, 2012; Todorovic et al., 2011).

A potential reconciliation of the relationship between expectation and repetition suppression comes from work showing that while expectations decrease the overall amount of neural activity, they can also yield sharper representations of sensory stimuli (Kok et al., 2012). This work goes beyond conventional neuroimaging approaches, which typically only measure overall levels of neural activity (Buckner et al., 1998; Kourtzi and Kanwisher, 2001; Tootell et al., 1995). Such amplitude changes could in principle be produced by one or more different types of change in the underlying neural representations. For instance, both sharpening, where the response to only unpredicted features is suppressed, and gain reduction, where a multiplicative suppression occurs for all features, could be associated with decreased population activity, even though the amount of information carried by the representations will be markedly different. Recently introduced multivariate pattern analytic approaches to human neuroimaging – specifically forward encoding modelling – allow for the quantification of stimulus-selective information contained within patterns of neural activity in human observers (Brouwer and Heeger, 2009; Garcia et al., 2013; King et al., 2016; Kok et al., 2017; Myers et al., 2015; Salti et al., 2015; Wolff et al., 2017). This approach goes beyond typical multivariate pattern analyses (which normally produce only accuracy scores) by quantifying neural representations evoked by sensory stimuli to reveal both the accuracy and the tuning fidelity for the specific feature-dimension of interest.

Here, we used multivariate forward encoding methods to test whether repetition suppression and expectation have different effects on the way the brain represents visual information, in this case the orientation of grating stimuli. To anticipate the results, we found that soon after stimulus onset, repetition suppression had no effect on visual orientation selectivity, but violated expectations were associated with a significantly increased orientation-selective response through gain modulation, with no corresponding change in response fidelity. This representation was transiently re-activated at around 200 ms post-stimulus onset, suggesting that feedback influences initial sensory encoding of an unexpected stimulus, which in turn allows for updating of the sensory prior.

Results

We used a modified version of the paradigm introduced by Summerfield et al. (2008), replacing the face stimuli used in that study with oriented Gabors. These low-level stimuli allowed us to quantify the degree of orientation selectivity in EEG activity to determine how the representation of orientation is affected by prediction error and repetition suppression. Each of 15 observers participated in two EEG sessions. On each trial, two Gabors were presented sequentially (100 ms presentation, 600 ms stimulus onset asynchrony), and these stimulus pairs either repeated or alternated in their orientation (Figure 1A, Video 1). The predictability of the repeated and alternating pairs was varied in a block-wise manner to manipulate expectation. In a repeating block, the orientations of the two Gabors in a pair repeated in 80% of trials, and alternated in the remaining 20%. These contingencies were reversed in the alternating block (Figure 1B). The orientations of successive stimuli across a block were randomised to limit any accumulated effects of adaptation and prediction. As repetition suppression and expectation form orthogonal dimensions of the task, the design allowed us to isolate their respective contributions to neural responses. Participants completed an unrelated task of discriminating (red vs green) rare coloured Gabors (which occurred on 10% of trials).

Figure 1. — (A) Schematic of the stimuli and timing used in the experiment. Participants viewed a rapid stream of pairs of Gabors and monitored for an infrequent coloured target (10% of trials). The stimulus orientations were pseudorandomly varied across trials between 0° and 160° (in 20° steps), allowing estimation of orientation-selective information contained within patterns of EEG activity. (B) The orientation of the pairs of Gabors could either repeat or alternate. In one type of block, 80% of trials were orientation repeats and the remaining 20% alternated (Repeating blocks); in the other type of block, these contingencies were reversed (Alternating blocks).

Video 1. Example of a stimulus sequence of Gabors in a typical alternating block.

Download video file^{(202.5KB, mp4)}

DOI: 10.7554/eLife.33123.003

Open in a new tab

Repetition suppression and prediction error affect the overall level of neural activity

The Gabors elicited a large response over occipital-parietal areas (Figure 2A). Consistent with previous work (Cui et al., 2016; Keller et al., 2017; Rentzeperis et al., 2012; Summerfield et al., 2011; Todorovic et al., 2011; Todorovic and de Lange, 2012; Tootell et al., 1998), there was a significant repetition suppression effect (Repeat < Alternating), such that the response to repeated stimuli was significantly reduced compared with the response to alternating stimuli (Figure 2A). The repetition suppression effect was evident over a large cluster of occipital-parietal electrodes at two time intervals: an early effect from 79 to 230 ms, and a later effect from 250 to 540 ms after the onset of the second stimulus (cluster p < 0.025; Figure 2B and caption). A large cluster of frontal electrodes mirrored the repetition suppression effect with a similar time course: the ERP over these frontal sites had the same pattern, but was reversed in sign, suggesting it originated from the same dipole as the occipital response.

Also consistent with previous results (Garrido et al., 2009; Summerfield et al., 2011; Todorovic et al., 2011; Todorovic and de Lange, 2012), there was a significant expectation effect (Expected < Unexpected). Specifically, there was a significantly greater negativity for unexpected versus expected stimuli, and this effect was most prominent over a cluster of occipital-parietal electrodes around 75–150 ms after stimulus presentation (Figure 2C). As with the repetition suppression result described above, there was an expectation effect of opposite polarity over occipital-parietal electrodes. This effect was significant at an early time point post-stimulus (79–130 ms), but not at later time points (320–390 ms; Figure 2D). Finally, there was no interaction between repetition suppression and expectation (i.e., no significant positive or negative clusters, all p > 0.05). Taken together, these results reveal both repetition suppression and expectation effects in the neural data, which were indexed separately as shown in Figure 2.

We conducted a further traditional peak analysis, to aid comparison with previously published studies on the mismatch negativity (Garrido et al., 2013; Näätänen et al., 2007; Saarinen et al., 1992). We bandpass filtered the ERPs (2–40 Hz) to recover the stereotypic waveform (Figure 2C) and examined two classic early components – the N1 and P1 – averaged across a broad grouping of occipital-parietal electrodes (O1, O2, Oz, POz, PO7, PO3, PO8, PO4, P3, Pz, P2). As in previous studies (Dehaene et al., 2001; Caharel et al., 2009), we defined the P1 as the largest positivity between 80 and 110 ms after stimulus presentation, and the N1 as the largest negativity between 90 and 130 ms after stimulus presentation. A relatively wide temporal window was used to capture inter-individual response variation. As expected, for the P1 component, the repeated stimulus evoked a significantly smaller positivity (t(14) = 3.03, p = 0.009) than the alternating stimulus (Figure 2D), reflecting a repetition suppression effect. There was no such effect of expectation on the P1 (t(14) = 0.26, p = 0.80). By contrast, as predicted from previous work (Garrido et al., 2013; Näätänen et al., 2007; Saarinen et al., 1992), analysis of the N1 component showed that the unexpected stimulus evoked a significantly greater negativity than the expected stimulus, (t(14) = 5.75, p < 0.0001). The repetition suppression effect was also present in the N1 (t(14) = 2.39, p = 0.03), but critically in the opposite direction to that of the expectation effect.

Expectations increase orientation-selective information contained within patterns of EEG activity

We next examined the key question of whether repetition suppression and expectation differentially affect neural representations of orientation information. To do this, we used a forward encoding approach to reconstruct orientation-selective information contained within the multivariate pattern of EEG activity distributed across the scalp (Figure 3; see Materials and methods for details). Briefly, this technique transforms EEG sensor-level responses into tuned ‘feature’ channels (Brouwer and Heeger, 2009; Garcia et al., 2013; Kay et al., 2008; Myers et al., 2015), in this case, orientation-selective features. For each trial, the presented orientation was convolved with a canonical, orientation-selective tuning function and regressed against the pattern of EEG activity across all sensors at each time point. This created a spatial filter of the multivariate EEG activity that differentiated orientations (Figure 3D). These weights were then inverted to reconstruct the model and multiplied against an independent set of test trials to produce responses in the modelled orientation channels. These sets of responses were then used to evaluate the degree of orientation selectivity in those trials. The procedure was repeated for all time points in the trial, and a cross-validated approach was used until all trials had been used for both training and testing.

As shown in Figure 3, the forward encoding revealed a strong, orientation-selective response derived from the multivariate pattern of EEG activity. This orientation-tuned response was evident from ~50 to ~470 ms after stimulus onset, and peaked between ~120 and 250 ms (Figure 3C). Examination of the regression weights revealed that this response was largely driven by activity centred over occipital-parietal areas (Figure 3D).

To examine our central question of whether repetition suppression and expectation have differential effects on neural representations of orientation, we split and averaged the results of the forward encoding by trial type, and fitted these with Gaussians (see Materials and methods) to quantify orientation selectivity (Figure 4). Repetition suppression did not affect the amount of orientation selectivity contained within the EEG data, with similar selectivity for repeated and alternating trials. This was the case even though the repeated trials had a markedly smaller EEG response over occipital and parietal electrodes (see Figure 2A), where the forward encoding model was maximally sensitive. This result is consistent with the ‘efficient representation’ hypothesis of repetition suppression (Gotts et al., 2012), which argues that the overall neural response is smaller with repetition suppression due to more efficient coding of stimulus information.

Figure 4—figure supplement 1. — (A) Amount of orientation-selective information (given by the amplitude of the fitted Gaussian) from the EEG signal in response to the second Gabor in a pair, shown separately for repetition suppression (upper panel) and expectation (lower panel). The thick black line indicates significant differences between the conditions (two-tailed cluster-permutation, alpha p < 0.05, cluster alpha p < 0.05, N permutations = 20,000). (B) Population tuning curves averaged over the significant time period (79–185 ms) depicted in panel A. The curves, shown as fitted Gaussians, illustrate how overall stimulus representations are affected by repetition and expectation. While there was no difference in orientation tuning for repeated versus alternate stimuli (upper panel), the amplitude of the orientation response increased significantly, and the baseline decreased, for unexpected relative to expected stimuli. Error bars indicate ±1 standard error.

Examining the effect of expectation revealed a markedly different pattern of results. As shown in Figure 4A, at 79–185 ms after the onset of the second stimulus in the pair, orientation-selectivity increased significantly (p < 0.0001) when the stimulus was unexpected relative to when it was expected, and this effect arose at the earliest stages of the brain’s response to that stimulus. Moreover, the expectation signal contained enhanced information about the specific features of the stimulus that violated the expectation, in this case the orientation of the second grating. We conducted the same statistical tests on the three other parameters defining the Gaussian function (namely, the width, centre orientation and baseline) to determine how repetition suppression and expectation might affect other properties of the neural representation. There was no reliable influence of repetition suppression on any of these Gaussian parameters (all ps > 0.32). For expectation, there was a significant decrease in baseline activity over the same time period as observed for the increase in amplitude (79–185 ms, p = 0.001), but there were no significant effects for the other parameters (all ps > 0.30).

We followed up this initial analysis to ensure we did not miss any small effects of repetition suppression or expectation on any aspects of stimulus representation. We increased the signal-to-noise by averaging the stimulus reconstruction over this early time period (79–185 ms after stimulus presentation), and fitted Gaussians to each participant’s data individually (Figure 4B). This again showed that the amplitude of the response was significantly (t(14) = 3.34, p = 0.005) higher for unexpected (M = 0.67, SE = 0.06) than for expected (M = 0.41, SE = 0.03) stimuli. By contrast, the width of the representations was similar for unexpected (M = 29.62°, SE = 4.72°) and expected (M = 26.72°, SE = 2.74°) stimuli, (t(14) = 0.78, p = 0.45). There was also a small but non-significant (t(14) = 1.94, p = 0.06) trend for a smaller baseline response (i.e., non-orientation tuned activity) in the unexpected (M = −0.01, SE = 0.07) than in the expected (M = 0.13, SE = 0.02) condition. For comparison, we also averaged the same time period for the repetition suppression conditions, and found similar curves for the repeated and alternating trials (all ps > 0.18). This analysis confirms the previous result, which employed more conservative nonparametric cluster-based testing.

It might be argued that the particular baseline period we chose for the encoding analyses - namely from −100 to 0 ms before the onset of the second Gabor in each pair – biased the results by incorporating a purely top-down expectation template triggered by the orientation of the first Gabor (Kok et al., 2017). To rule out this possibility, we performed a further forward encoding analysis where we baselined the raw EEG data to the mean activity from −100 to 0 ms before the first Gabor in each pair. Critically, this control analysis involved a baseline period over which it was not possible to form a top-down expectation of the orientation of the second Gabor based on the orientation of the first. This analysis yielded the same pattern of results as the original analysis (Figure 4—figure supplement 1), such that the unexpected stimulus evoked significantly greater orientation selectivity than the expected stimulus (p = 0.02). Also in line with the original analyses, the width of the representation was not affected by expectation (p = 0.44), and there was no effect of repetition suppression on orientation selectivity (p = 0.64). We can thus be confident that the effect of expectation on orientation selectivity that we report here, based on our forward encoding analyses, is not an artefact of the baselining procedure.

We also used a number of approaches to determine whether repetition suppression and expectation interacted to affect orientation selectivity. First, we took the difference scores between the combination of factors (e.g., expected repetition minus unexpected repetition, and expected alternation minus unexpected alternation) and compared these using the same cluster-based permutation testing outlined above. This analysis revealed no significant interactions between the factors for any parameter (all ps > 0.10). Second, we found the largest orientation-selectivity, defined by the amplitude of the fitted Gaussian, across the 600 ms following stimulus presentation. For each participant, this resulted in a single value for the four conditions. Each of these values was subjected to a two-way repeated-measures ANOVA, which again revealed no significant interaction between the factors (all ps > 0.30)

To further examine whether orientation-selectivity contained within the overall pattern of EEG activity differed for unexpected and expected stimuli, we used multivariate discriminant analysis to perform traditional backward decoding (Grootswagers et al., 2017; Kamitani and Tong, 2005; King and Dehaene, 2014; Marti et al., 2015). This approach (Figure 5) yielded the same pattern of results as that revealed by the forward encoding approach described above. The same cross-validation procedure was used as in the forward encoding analysis, but accuracy was now defined as the proportion of trials labelled with the correct orientation. To facilitate comparison with the results of Kok et al. (2012), we took the peak classification accuracy within a 600 ms window after presentation of the second grating within each pair. This analysis confirmed the results of the forward encoding: orientations shown in unexpected trials were classified better than orientations shown in expected trials (F(1,14) 76.42, p < 0.00001). Again, there was no effect of repetition on classification accuracy (F(1,14) = 0.027, p = 0.87); nor was there a significant interaction (F(1,14) = 2.52, p = 0.13). This suggests the finding is not specific to the analysis method but rather reflects how expectation affects the representation of sensory information in general.

Figure 5. — The dotted line indicates chance performance (1/9 orientations). The error bars indicate ±1 standard error of the mean.

Expectation affects the temporal stability of stimulus representations

Next, we examined whether repetition suppression and expectation affected dynamic, ongoing stimulus representations by using cross-temporal generalisation (King and Dehaene, 2014; King et al., 2014; Myers et al., 2015; Spaak et al., 2017; Stokes et al., 2013). To do this, we used the same forward encoding approach as in the previous analysis, but now the weights were derived from one time point on one set of trials, and then applied at every time point in the test trials. Again, a cross-validation approach was used, with all trials serving as both training and test. This analysis examined whether the same spatial pattern of EEG activity that allowed for orientation selectivity generalised to other time points, thus revealing whether the pattern of orientation-selective activity was stable or changed over time.

As shown in Figure 6, optimal orientation selectivity was on-axis (training time equals test time) between 100 ms and 300 ms after stimulus presentation, suggesting that the stimulus representation changed dynamically over time (King and Dehaene, 2014). There was also significant off-axis orientation-selectivity from 100 to 500 ms after stimulus presentation, suggesting that some aspects of the neural representation of orientation were stable over time.

There was no effect of repetition suppression on temporal generalisation of orientation information (upper panels of Figure 6), suggesting that repetition suppression did not affect the temporal stability of neural representations of orientation. Examining the effect of expectation on cross-temporal generalisation confirmed that there was significantly more on-axis orientation selectivity when the stimulus was unexpected than when it was expected (cluster p = 0.02). This increased on-axis orientation selectivity generalised off-axis at around 300–400 ms after stimulus onset (cluster p = 0.01), suggesting that the same representation that is activated to process the expectation is reactivated later as the stimulus continues to be processed. Such a signal could constitute the prior of the prediction, as this should be updated on the basis of incoming sensory evidence, which in turn would likely require reactivation of the unexpected stimulus.

Discussion

Our findings demonstrate that repetition suppression and expectation have distinct effects on neural representations of simple visual stimuli. Repetition suppression had no effect on orientation selectivity, even though the neural response to repeated stimuli was significantly reduced over occipito-parietal areas. Unexpected stimuli, on the other hand, showed significantly increased orientation-selectivity relative to expected stimuli. This same early representation of the unexpected stimulus appeared to be reactivated at 200–300 ms after the initial neural response, supporting the idea that sensory expectations may be updated through comparison with incoming sensory evidence. These results suggest that repetition suppression and expectation are separable and independent neural computations.

Our work provides a significant advance in understanding how predictions allow the brain to process incoming sensory information by comparing what is expected with what actually occurs. How expectations affect neural responses has been extensively investigated using mismatch negativity paradigms in which an unexpected stimulus causes a larger neural response than an expected stimulus (Bekinschtein et al., 2009; Garrido et al., 2009; Näätänen et al., 2007). Such mismatch responses to an unexpected stimulus have often been attributed to the generation of a prediction error that updates expectation based on a conflict between sensory evidence and the prior (Garrido et al., 2009). To date, however, most studies have focused exclusively on the overall magnitude of neural responses to unexpected events, rather than assessing the quality of stimulus-specific information potentially contained within such responses. As noted above, enhanced neural activity to unexpected visual events could reflect a differential response to one of a number of possible stimulus features, or simply an increase in baseline activity associated with a non-selective response. By examining how expectation affects the representation of an elementary feature dimension – in this case, orientation – our results imply the operation of at least two distinct neural processes at separate times following stimulus onset. Incoming sensory information is first evaluated against the prior (which occurs early after stimulus presentation). When an unexpected stimulus is detected and generates a prediction error, the representation is amplified through gain enhancement. Later, around 300 ms after stimulus presentation, this same representation is reactivated to update the expectation against the initially predicted representation.

According to predictive coding theory, expected stimuli should be more efficiently represented than unexpected stimuli largely because the reduced neural response still encodes stimuli with the same fidelity (Friston, 2005). A more efficient response could be due to sharpening of neuronal tuning to stimulus features, or to a reduction in the gain of evoked neural responses. Our results support the latter interpretation. Specifically, there was no evidence that a fulfilled expectation leads to a sharper representation of orientation information. Our findings might imply that the brain needs to have more information about an unexpected stimulus so that a correct response can be made. Our findings thus provide a novel insight into how predictive coding might change neural representations of sensory information.

The lack of evidence for sharpening of neural tuning in the current results is in contrast to the findings of a previous study (Kok et al., 2012), in which a high-level prediction error led to ‘sharper’ multivariate decoding for expected versus unexpected visual stimuli. In their study, Kok et al. (2012) used an auditory tone to cue the orientation of a subsequent visual stimulus, and found significantly reduced off-label classification accuracy for predicted than for unpredicted stimuli. They concluded that predictions cause sharpening of stimulus representations. More recently, using the same task combined with a forward encoding approach, Kok et al. (2017) showed that response gain is increased for a predicted stimulus.

It is natural to ask why the results of the current study differ from those of Kok and colleagues outlined above. One possible explanation lies in the different approaches used to generate expectations across the studies. Specifically, whereas Kok et al. manipulated expectations by pairing an auditory cue with a visual stimulus, we exploited the properties of the visual stimuli themselves (i.e. their orientation) to generate expectations within blocks of trials. An intriguing possibility is that predictions requiring integration of sensory events from two or more modalities lead to increased sharpening, whereas predictions made within a single sensory modality lead to decreased gain. This might in turn relate to the noted differences between simple ‘local’ and higher-order ‘global’ type predictions (Bekinschtein et al., 2009; King et al., 2014), which lead to distinct patterns of stimulus-selective decoding. A similar discrepancy relating to the effects of attention on sensory representations has been widely discussed, with some studies finding sharpening of stimulus representations with attention, and others showing gain enhancement (Liu et al., 2007; Maunsell, 2015; Maunsell and Treue, 2006; Treue and Martínez Trujillo, 1999). The differences between these results may potentially have arisen because the tasks relied upon different types of attention (e.g., spatial versus feature-based). Future studies could determine whether this same divergence occurs for prediction effects.

The current work applied multivariate model-based approaches to EEG data to determine how prediction and repetition suppression affect neural representations of perceptual information. We chose to use EEG so we could recover the temporal dynamics of these effects – something that would not be possible with the BOLD signal used in fMRI – and because EEG is the most widely used tool for measuring expectation effects in human participants (see Garrido et al., 2009 and Paavilainen, 2013 for review), thus facilitating comparison of our findings with those of other studies. We estimated orientation selectivity using all EEG electrodes distributed across the scalp for two principal reasons. First, we wanted to limit experimenter degrees of freedom (Simmons et al., 2011) potentially introduced through the post-hoc selection of subsets of electrodes. Second, given the broad spatial resolution of EEG, we reasoned that activity recorded from electrodes at any given scalp location could potentially carry important feature-selective information from a number of neural sources. The results revealed that orientation-selective information appears largely driven by electrodes over occipital-parietal regions (Figure 3D), consistent with a number of previous studies that employed visual decoding of M/EEG data (Cichy et al., 2014; Cichy et al., 2015; Stokes et al., 2015). As noted above, however, it is entirely possible that the effects we observed here arose from sources well beyond the occipital and parietal regions, or even potentially outside the visual cortical hierarchy. Limitations in the temporal and spatial resolution of current human imaging methods make it impossible to pinpoint the timing and location of interactions between visual areas that might reflect the cascade of predictions and prediction errors involved in sensory encoding. By combining the current paradigm and multivariate modelling with invasive recordings in animal models – for example, using calcium imaging or extracellular electrode recordings – it should be possible to test some of the key claims of predictive coding theory that we have examined here, but at the level of individual neurons.

Surprisingly, few studies have used invasive recording methods to test how predictive coding affects stimulus representations at the neuronal level. One study in macaques (Kaliukhovich and Vogels, 2011) used a design similar to that of Summerfield and colleagues, but with high-level objects (fractals and real-world objects) as stimuli. That study found that expectation did not attenuate repetition suppression in either spiking activity or local field potentials within the inferior temporal cortex. A later fMRI study in humans (Kovács et al., 2013) used a similar stimulus set, and also found no attenuation of repetition suppression by expectation in the same cortical region. A follow-up study provided a potential explanation for these findings by showing that the attenuation of neural responses associated with repetition suppression is found with familiar stimuli, but not with unfamiliar stimuli (Grotheer and Kovács, 2014). Viewed in this light, the stimulus sets used by (Kaliukhovich and Vogels, 2011) might not have been sufficiently familiar to yield effects of expectation in their non-human primate observers.

Other work has shown that context plays an important role in determining the magnitude of neuronal responses to sensory events. Thus, for example Ulanovsky et al. (2003) found that rare auditory stimuli generate significantly larger responses in primary auditory cortical neurons than more commonly occurring stimuli. This result has been interpreted as a single-neuron analogue of the mismatch negativity, but the design used in the study did not control for adaptation effects, thus making it difficult to draw an unambiguous comparison with the current work. In the visual domain, oddball stimuli have also been found to modulate neuronal activity in rats, characterised by an enhancement of responses in the higher-order latero-intermediate area (Vinken et al., 2017). Moreover, Fiser et al. (2016) found that neurons in mouse primary visual cortex show a greater response when task-irrelevant visual stimuli that had been presented during training were omitted, suggesting that an established expectation had been violated. This result is consistent with the literature on the mismatch negativity, in which the omission of an expected stimulus results in a large prediction error (Garrido et al., 2009; Wacongne et al., 2011). In non-human primates, neurons in the inferior temporal cortex show an enhanced response to unexpected relative to expected stimuli (Kaposvari et al., 2018), and population decoding accuracy is higher for unexpected compared with expected stimuli (Kumar et al., 2017). Critically, however, no study has simultaneously recorded neuronal activity in multiple cortical regions to determine whether predictions generated in one area refine responses in a second area, as postulated by predictive coding theory (Friston, 2005; Rao and Ballard, 1999). Such a direct demonstration is necessary to provide a strong test of the central notion that cortical areas pass signals between themselves in order to generate expectations.

Unlike the effects of expectation, there is a large body of electrophysiological work showing that sensory adaptation influences neuronal activity (Adibi et al., 2013; Adibi et al., 2013; Felsen et al., 2002; Kohn and Movshon, 2004; Patterson et al., 2013). For instance, there is a sharpening of stimulus selectivity in MT neurons following 40 s of adaptation to a drifting grating (Kohn and Movshon, 2004). As we have highlighted, however, prolonged adaptation is likely also associated with a significant prediction that the next stimulus will be the same as the previous one. Perhaps more relevant to the current results, Patterson et al. (2013) found that the width of orientation tuning in V1 is only marginally sharpened following brief (400 ms) periods of adaptation. Again, however, their study did not control for expectation, so it is impossible to determine the role of predictive coding in their observations. Our finding that repetition suppression did not affect the bandwidth of orientation selectivity measured using EEG is also consistent with models of orientation adaptation based on human psychophysical data, which suggest that adaptation does not affect the tuning width of the adapted neural populations (Clifford, 2002; Clifford, 2014; Dickinson et al., 2010; Dickinson et al., 2017; Tang et al., 2015).

In summary, we have shown that repetition suppression and expectation differentially affect the neural representation of simple, but fundamental, sensory features. Our results further highlight how the context in which a stimulus occurs, not just its features, affect the way it is represented by the brain. Our findings suggest encoding priority through increased gain might be given to unexpected events, which in turn could potentially speed behavioural responses. This prioritised representation is then re-activated at a later time, supporting the idea that feedback from higher cortical areas reactivates an initial sensory representation in early cortical areas.

Materials and methods

Participants

A group of 15 healthy adult volunteers (nine females, median age = 20.5 years, range = 18 to 37 years) participated in exchange for partial course credit or financial reimbursement (AUD$20/hr). We based our sample size on work that investigated the interaction between repetition suppression and prediction error (Summerfield et al., 2008), and that used forward encoding modelling to investigate orientation selectivity using MEG with a comparable number of trials as the current study (Myers et al., 2015). Each person provided written informed consent prior to participation, and had normal or corrected-to-normal vision. The study was approved by The University of Queensland Human Research Ethics Committee and was in accordance with the Declaration of Helsinki.

Experimental setup

The experiment was conducted inside a dimly-illuminated room with the participants seated in a comfortable chair. The stimuli were displayed on a 22-inch LED monitor (resolution 1920 × 1080 pixels, refresh rate 120 Hz) using the PsychToolbox presentation software (Brainard, 1997; Pelli, 1997) for MATLAB (v7.3). Viewing distance was maintained at 45 cm using a chinrest, meaning the screen subtended 61.18° x 36.87° (each pixel 2.4’ x 2.4’).

Visual task

The stimuli were Gabors (diameter: 5°, spatial frequency: 2 c/°, 100% contrast) presented centrally in pairs for 100 ms, separated by 500 ms (600 ms stimulus onset asynchrony) with a variable (650 to 750 ms) inter-stimulus interval between trials. Across the trials, the orientations of the Gabors were evenly spaced between 0° and 160° (in 20° steps) so we could reconstruct orientation selectivity contained within the EEG response using forward encoding modelling. The relationship of the orientations of the pairs Gabors was also used to construct the different repetition suppression and prediction conditions. The orientation presented in the second Gabor in the pair could either repeat or alternate with respect to the orientation of the first Gabor. In the alternation trials, the orientation of the first Gabor was drawn randomly, without replacement, from an even distribution of orientations that was different to the orientation of the second Gabor. To vary the degree of prediction, in half of the blocks 80% of the trials had repeated orientations and 20% of the trials had alternating orientations, whereas in the other half of the blocks these contingencies were reversed. This design allowed us to separately examine the effects of repetition suppression and prediction because of the orthogonal nature of the blocked design. The blocks of 135 trials (~3 mins) switched between the expectation of a repeating or alternating pattern, with the starting condition counterbalanced across participants.

The participants’ task was to monitor the visual streams for rare, faintly coloured (red or green) Gabors and to discriminate the colour as quickly and accurately as possible. Any trial with a coloured target was excluded from analysis. The orientation match between the pairs was made to be consistent with the dominant contingency (i.e. repeated or alternating) within that block. Pilot testing was used prior to the main experiment to set the task at approximately threshold, to ensure that participants focused exclusively on the colour-discrimination task rather than the orientation contingencies associated with prediction and repetition. Only one participant reported being aware of the changing stimulus contingencies across the blocks when asked at the end of the experiment, and excluding this participant’s data had no effect on the key results reported here. Self-paced breaks were provided between each of the 20 blocks within a session, at which time feedback was provided on performance in the preceding block. Each participant completed two sessions of 2700 trials each (5400 trials in total), with each session lasting around 70 min of experimental time and 45 min of EEG setup.

EEG acquisition and pre-processing

Continuous EEG data were recorded using a BioSemi Active Two system (BioSemi, Amsterdam, Netherlands). The signal was digitised at 1024 Hz sampling rate with a 24-bit A/D conversion. The 64 active scalp Ag/AgCl electrodes were arranged according to the international standard 10–20 system for electrode placement (Oostenveld and Praamstra, 2001) using a nylon head cap. As per BioSemi system design, the common mode sense and driven right leg electrodes served as the ground, and all scalp electrodes were referenced to the common mode sense during recording.

Offline EEG pre-processing was performed using EEGLAB in accordance with best practice procedures (Bigdely-Shamlo et al., 2015; Keil et al., 2014). The data were initially down-sampled to 256 Hz and subjected to a 0.5 Hz high-pass filter to remove slow baseline drifts. Electrical line noise was removed using clean_line.m, and clean_rawdata.m in EEGLAB (Delorme and Makeig, 2004) was used to remove bad channels (identified using Artifact Subspace Reconstruction), which were then interpolated from the neighbouring electrodes. Data were then re-referenced to the common average before being epoched into segments around each stimulus pair (−0.5 s to 1.25 s from the first stimulus in the pair). Systematic artefacts from eye blinks, movements and muscle activity were identified using semi-automated procedures in the SASICA toolbox (Chaumon et al., 2015) and regressed out of the signal. After this stage, any trial with a peak voltage exceeding ±100 uV was excluded from the analysis. The data were then baseline corrected to the mean EEG activity from −100 to 0 ms before the presentation of the second Gabor in the pair. Critically, the orientations of the first and second gratings were precisely balanced across the conditions to avoid any systematic bias in orientation information being carried forward by the first grating within each pair. Specifically, for every unexpected stimulus presented in the second grating there was an equal number of every other orientation that was expected to be presented. As the analysis we employed used a regression-based approach, any carry over of orientation-selective information from the first to the second grating therefore could not systematically bias the results.

Experimental design

We used a modified version of a factorial design that has previously been used to separately examine the effects of repetition suppression and prediction error (Kaliukhovich and Vogels, 2011; Kovács et al., 2013; Summerfield et al., 2008; Summerfield et al., 2011; Todorovic et al., 2011; Todorovic and de Lange, 2012). By comparing the two repeat conditions with the two alternating conditions, we could examine repetition suppression while controlling for different levels of expectation. Conversely, by comparing across the expected and unexpected trials, we could examine prediction error while controlling for repetition suppressi.

The relationship between the pairs of orientations for the different expectation conditions was based on the original study (Summerfield et al., 2008), and on other studies (Kaliukhovich and Vogels, 2011; Kovács et al., 2013) that examined the interaction between repetition suppression and expectation. In the repeating condition, the orientation of the second Gabor is expected to be the same as the orientation of the first, whereas in the alternating condition the orientation of the second Gabor is expected to be different from that of the first. This relationship between the expected orientations of the stimuli in the alternating condition is slightly different to another modification of the paradigm which employed a more limited range of stimuli (Todorovic et al., 2011; Todorovic and de Lange, 2012). Specifically, the paradigm introduced by Todorovic and colleagues used two or three auditory tones of different frequencies. In their alternating condition, the expectation was that one tone would follow another (i.e. 1000 Hz and then 1032 Hz); this was then violated when a 1000 Hz tone was repeated. In this paradigm, an exact frequency was expected in the alternating condition, a design feature that differs from the paradigm used in the current work where there was no specific expectation of the orientation of the second Gabor based on the orientation of the first in the alternating condition. Instead the expectation in the alternating condition was that the orientation would change, and this could be violated by repeating the orientation. In this sense, there was no specific expectation about the second orientation in the alternating condition. Instead, the rule concerened the alternation or repetition of the first orientation. We did not implement the Todorovic et al. paradigm because of the combinatorial explosion of stimulus conditions it would require to measure orientation selectivity (such that every orientation is predicted by another orientation). Future work could investigate how this subtle change in paradigm design affects the encoding of stimulus information.

Forward encoding modelling

We used a forward encoding approach to estimate the amount of orientation-selective information contained in the EEG data at each time point of the trial. This approach differs from standard decoding approaches by modelling each presented orientation as a continuous variable of a set of tuned orientation-selective channels. The forward-encoding technique has been successfully used to reconstruct colour (Brouwer and Heeger, 2009), spatial (Sprague and Serences, 2013) and orientation (Ester et al., 2016) selectivity in fMRI data. More recently, the same approach has been applied to EEG and MEG data, which have inherently better temporal resolution than fMRI (Garcia et al., 2013; Kok et al., 2017; Myers et al., 2015; Wolff et al., 2017).

We applied forward encoding modelling to determine how repetition suppression and prediction error affected orientation selectivity. To do this, the second orientation (Figure 7A) in the Gabor pair in each trial was used to construct a regression matrix, with separate regressors for the nine orientations used across the experiment. This regression matrix was convolved with a set of basis functions (half cosines raised to the 8^th power (Figure 7C), which allowed complete and unbiased coverage of orientation space) to allow us to pool similar information patterns across nearby orientations (Brouwer and Heeger, 2009). We used this tuned regression matrix to estimate time-resolved orientation selectivity contained within the EEG activity in a 16 ms sliding window, in 4 ms steps (Figure 7B; Myers et al., 2015). To avoid overfitting, we used a leave-one-out cross-validation procedure where the regression weights were estimated for a training set and applied to an independent test set (Figure 7D). All trial types (including target trials) were used in training and test sets. This was done by solving the linear equation:

Figure 7. — (A) Participants viewed individual gratings at fixation, each with a specific orientation. (B) Neural activity evoked by each grating was measured over the entire scalp. (C) Evoked neural responses were convolved with canonical orientation-selective functions (grey lines in C) to determine coefficients for the different orientations (coloured dots and lines, which match the colours of the outlined gratings in A). These coefficients were then used to generate a regression matrix. (D) General linear modelling was used on a subset of training trials to generate weights for each channel. These weights were inverted and simultaneously applied to an independent test set of data to recover orientation selectivity in the EEG activity. As EEG activity has high temporal resolution, we can apply the procedure to many epochs following stimulus presentation to determine the temporal dynamics of orientation processing (see Figure 3).

B_{1} = W C_{1}

(1)

where B₁ (64 sensors x N training trials) is the electrode data for the training set, C₁ (nine channels x N training trials) is the tuned channel response across the training trials, and W is the weight matrix for the sensors we want to estimate (64 sensors x nine channels). W can be estimated using least square regression to solve equation (2):

W = (C_{1} C_{1}^{T})^{- 1} C_{1}^{T} B_{1}

(2)

The channel response in the test set C₂ (nine channels x N test trials) was estimated using the weights in (2) and applied to activity in B₂ (64 sensors x N test trials).

C^{2} = (W W^{T}) W^{T} B^{2}

(3)

We repeated this process by holding one trial out as test, and training on the remaining trials until all trials had been used in test and training. The procedure was repeated for each trial within the trial epoch. We then shifted all trials to a common orientation, meaning that 0° corresponded to the orientation presented on each trial.

The reconstructed channel activations were separated into the four conditions, and averaged over trials. These responses were then smoothed with a Gaussian kernel with a 16 ms window, and fitted with a Gaussian function (4) using non-linear least square regression to quantify the amount of orientation selective activity.

G (x) = A e x p (- \frac{{(x - ϕ)}^{2}}{2 σ^{2}}) + C

(4)

Where A is the amplitude representing the amount of orientation selective activity, $ϕ$ is the orientation the function is centred on (in degrees), $σ$ is the width (degrees) and C is a constant used to account for non-orientation selective baseline shifts.

Multivariate pattern analysis

We conducted a multivariate pattern analysis to build upon the initial forward encoding results which showed that unexpected stimuli elicit greater orientation selectivity than expected stimuli. This analysis used the same data as the forward encoding analysis. We used the classify function from Matlab 2017a with the ‘diaglinear’ option to implement a Naive Bayes classifier. For each time point, we used the same cross-validation procedure as the forward encoding modelling with the same averaging procedure to select train and test sets of data. The classifier was given the orientations of the training data and predicted the orientation of the test data. A trial was labelled correct if the presented orientation was produced. To facilitate comparison of the results with those of Kok et al. (2012), we found the peak classification accuracy for each participant in the 600 ms following stimulus presentation. The same wide time window was used across conditions to accommodate large inter-individual differences in peak classification without biasing the results toward one particular condition.

Statistical testing

A non-parametric sign permutation test was used to determine the null distribution for testing (Wolff et al., 2017). This method makes no assumptions about the underlying shape of the null distribution. This was done by randomly flipping the sign of the data for the participants with equal probability. Fifty thousand (50,000) permutations were used for the time-series data, whereas only 5000 were used for the temporal generalisation plots because of the significantly greater computational demands.

Cluster-based non-parametric correction (50,000 permutations for timeseries and 5000 for temporal generalisation) was used to account for multiple comparisons, and determined whether there were statistical differences between the contrasting conditions. Paired-samples t-tests were used to follow up the analysis in Figure 4 within a specified time window, and no correction was applied. A two-way repeated measures ANOVA (implemented using GraphPad Prism 7.0 c, La Jolla, CA) was used to analyse the multivariate pattern analysis results shown in Figure 5.

Acknowledgements

This work was supported by the Australian Research Council (ARC) Centre of Excellence for Integrative Brain Function (ARC Centre Grant CE140100007) to JBM and EA, and by an ARC Discovery Project (DP170100908) to EA. JBM was supported by ARC Australian Laureate Fellowship (FL110100103).

Funding Statement

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Contributor Information

Matthew F Tang, Email: m.tang1@uq.edu.au.

Christopher Summerfield, University of Oxford, United Kingdom.

Timothy E Behrens, University of Oxford, United Kingdom.

Funding Information

This paper was supported by the following grants:

Australian Research Council CE140100007 to Ehsan Arabzadeh, Jason B Mattingley.
Australian Research Council DP170100908 to Ehsan Arabzadeh.
Australian Research Council FL110100103 to Jason B Mattingley.

Additional information

Competing interests

No competing interests declared.

Author contributions

Conceptualization, Formal analysis, Writing—original draft, Writing—review and editing.

Conceptualization, Writing—review and editing.

Ethics

Human subjects: Each participant provided written informed consent prior to participation. The study was approved by The University of Queensland Human Research Ethics Committee (approval number 2012000392) and was in accordance with the Declaration of Helsinki.

Additional files

Transparent reporting form

elife-33123-transrepform.pdf^{(273.2KB, pdf)}

DOI: 10.7554/eLife.33123.011

Data availability

The EEG data have been deposited on Dryad 10.5061/dryad.3d7kq

The following dataset was generated:

Tang M, Smout C, Arabzadeh E, Mattingley J. 2018. Data from: Prediction Error and Repetition Suppression Have Distinct Effects on Neural Representations of Visual Information. Dryad Digital Repository.

Reference

Adibi M, McDonald JS, Clifford CW, Arabzadeh E. Adaptation improves neural coding efficiency despite increasing correlations in variability. Journal of Neuroscience. 2013;33:2108–2120. doi: 10.1523/JNEUROSCI.3449-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
Auksztulewicz R, Friston K. Repetition suppression and its contextual determinants in predictive coding. Cortex. 2016;80:125–140. doi: 10.1016/j.cortex.2015.11.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
Barron HC, Garvert MM, Behrens TEJ. Repetition suppression: a means to index neural representations using BOLD? Philosophical Transactions of the Royal Society B: Biological Sciences. 2016;371:e20150355. doi: 10.1098/rstb.2015.0355. [DOI] [PMC free article] [PubMed] [Google Scholar]
Bekinschtein TA, Dehaene S, Rohaut B, Tadel F, Cohen L, Naccache L. Neural signature of the conscious processing of auditory regularities. PNAS. 2009;106:1672–1677. doi: 10.1073/pnas.0809667106. [DOI] [PMC free article] [PubMed] [Google Scholar]
Benucci A, Saleem AB, Carandini M. Adaptation maintains population homeostasis in primary visual cortex. Nature Neuroscience. 2013;16:724–729. doi: 10.1038/nn.3382. [DOI] [PMC free article] [PubMed] [Google Scholar]
Bigdely-Shamlo N, Mullen T, Kothe C, Su K-M, Robbins KA. The PREP pipeline: standardized preprocessing for large-scale EEG analysis. Frontiers in Neuroinformatics. 2015;9:B153. doi: 10.3389/fninf.2015.00016. [DOI] [PMC free article] [PubMed] [Google Scholar]
Brainard DH. The Psychophysics Toolbox. Spatial Vision. 1997;10:433–436. doi: 10.1163/156856897X00357. [DOI] [PubMed] [Google Scholar]
Brouwer GJ, Heeger DJ. Decoding and reconstructing color from responses in human visual cortex. Journal of Neuroscience. 2009;29:13992–14003. doi: 10.1523/JNEUROSCI.3577-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
Buckner RL, Goodman J, Burock M, Rotte M, Koutstaal W, Schacter D, Rosen B, Dale AM. Functional-anatomic correlates of object priming in humans revealed by rapid presentation event-related fMRI. Neuron. 1998;20:285–296. doi: 10.1016/S0896-6273(00)80456-0. [DOI] [PubMed] [Google Scholar]
Caharel S, d'Arripe O, Ramon M, Jacques C, Rossion B. Early adaptation to repeated unfamiliar faces across viewpoint changes in the right hemisphere: evidence from the N170 ERP component. Neuropsychologia. 2009;47:639–643. doi: 10.1016/j.neuropsychologia.2008.11.016. [DOI] [PubMed] [Google Scholar]
Chaumon M, Bishop DV, Busch NA. A practical guide to the selection of independent components of the electroencephalogram for artifact correction. Journal of Neuroscience Methods. 2015;250:47–63. doi: 10.1016/j.jneumeth.2015.02.025. [DOI] [PubMed] [Google Scholar]
Cichy RM, Pantazis D, Oliva A. Resolving human object recognition in space and time. Nature Neuroscience. 2014;17:455–462. doi: 10.1038/nn.3635. [DOI] [PMC free article] [PubMed] [Google Scholar]
Cichy RM, Ramirez FM, Pantazis D. Can visual information encoded in cortical columns be decoded from magnetoencephalography data in humans? NeuroImage. 2015;121:193–204. doi: 10.1016/j.neuroimage.2015.07.011. [DOI] [PubMed] [Google Scholar]
Clifford CW. Perceptual adaptation: motion parallels orientation. Trends in Cognitive Sciences. 2002;6:136–143. doi: 10.1016/S1364-6613(00)01856-8. [DOI] [PubMed] [Google Scholar]
Clifford CW. The tilt illusion: phenomenology and functional implications. Vision Research. 2014;104:3–11. doi: 10.1016/j.visres.2014.06.009. [DOI] [PubMed] [Google Scholar]
Cui Y, Wang YV, Park SJ, Demb JB, Butts DA. Divisive suppression explains high-precision firing and contrast adaptation in retinal ganglion cells. eLife. 2016;5:e19460. doi: 10.7554/eLife.19460. [DOI] [PMC free article] [PubMed] [Google Scholar]
Dehaene S, Naccache L, Cohen L, Bihan DL, Mangin JF, Poline JB, Rivière D. Cerebral mechanisms of word masking and unconscious repetition priming. Nature Neuroscience. 2001;4:752–758. doi: 10.1038/89551. [DOI] [PubMed] [Google Scholar]
Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods. 2004;134:9–21. doi: 10.1016/j.jneumeth.2003.10.009. [DOI] [PubMed] [Google Scholar]
Dickinson JE, Almeida RA, Bell J, Badcock DR. Global shape aftereffects have a local substrate: A tilt aftereffect field. Journal of Vision. 2010;10:e5. doi: 10.1167/10.13.5. [DOI] [PubMed] [Google Scholar]
Dickinson JE, Morgan SK, Tang MF, Badcock DR. Separate banks of information channels encode size and aspect ratio. Journal of Vision. 2017;17:27. doi: 10.1167/17.3.27. [DOI] [PubMed] [Google Scholar]
Diederen KM, Spencer T, Vestergaard MD, Fletcher PC, Schultz W. Adaptive prediction error coding in the human midbrain and striatum facilitates behavioral adaptation and learning efficiency. Neuron. 2016;90:1127–1138. doi: 10.1016/j.neuron.2016.04.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ester EF, Sutterer DW, Serences JT, Awh E. Feature-Selective Attentional Modulations in Human Frontoparietal Cortex. Journal of Neuroscience. 2016;36:8188–8199. doi: 10.1523/JNEUROSCI.3935-15.2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
Fairhall AL, Lewen GD, Bialek W, de Ruyter Van Steveninck RR. Efficiency and ambiguity in an adaptive neural code. Nature. 2001;412:787–792. doi: 10.1038/35090500. [DOI] [PubMed] [Google Scholar]
Felsen G, Shen YS, Yao H, Spor G, Li C, Dan Y. Dynamic modification of cortical orientation tuning mediated by recurrent connections. Neuron. 2002;36:945–954. doi: 10.1016/S0896-6273(02)01011-5. [DOI] [PubMed] [Google Scholar]
Fiser A, Mahringer D, Oyibo HK, Petersen AV, Leinweber M, Keller GB. Experience-dependent spatial expectations in mouse visual cortex. Nature Neuroscience. 2016;19:1658–1664. doi: 10.1038/nn.4385. [DOI] [PubMed] [Google Scholar]
Friston K. A theory of cortical responses. Philosophical Transactions of the Royal Society of London B Biological Sciences. 2005;360:815–836. doi: 10.1098/rstb.2005.1622. [DOI] [PMC free article] [PubMed] [Google Scholar]
Garcia JO, Srinivasan R, Serences JT. Near-real-time feature-selective modulations in human cortex. Current Biology. 2013;23:515–522. doi: 10.1016/j.cub.2013.02.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
Garrido MI, Kilner JM, Stephan KE, Friston KJ. The mismatch negativity: a review of underlying mechanisms. Clinical Neurophysiology. 2009;120:453–463. doi: 10.1016/j.clinph.2008.11.029. [DOI] [PMC free article] [PubMed] [Google Scholar]
Garrido MI, Sahani M, Dolan RJ. Outlier responses reflect sensitivity to statistical structure in the human brain. PLoS Computational Biology. 2013;9:e1002999. doi: 10.1371/journal.pcbi.1002999. [DOI] [PMC free article] [PubMed] [Google Scholar]
Gotts SJ, Chow CC, Martin A. Repetition priming and repetition suppression: a case for enhanced efficiency through neural synchronization. Cognitive Neuroscience. 2012;3:227–237. doi: 10.1080/17588928.2012.670617. [DOI] [PMC free article] [PubMed] [Google Scholar]
Grootswagers T, Wardle SG, Carlson TA. Decoding dynamic brain patterns from evoked responses: a tutorial on multivariate pattern analysis applied to time series neuroimaging data. Journal of Cognitive Neuroscience. 2017;29:677–697. doi: 10.1162/jocn_a_01068. [DOI] [PubMed] [Google Scholar]
Gross CG, Schiller PH, Wells C, Gerstein GL. Single-unit activity in temporal association cortex of the monkey. Journal of Neurophysiology. 1967;30:833–843. doi: 10.1152/jn.1967.30.4.833. [DOI] [PubMed] [Google Scholar]
Grotheer M, Kovács G. Repetition probability effects depend on prior experiences. Journal of Neuroscience. 2014;34:6640–6646. doi: 10.1523/JNEUROSCI.5326-13.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
Henson RN, Rugg MD. Neural response suppression, haemodynamic repetition effects, and behavioural priming. Neuropsychologia. 2003;41:263–270. doi: 10.1016/S0028-3932(02)00159-8. [DOI] [PubMed] [Google Scholar]
Kaliukhovich DA, Vogels R. Stimulus repetition probability does not affect repetition suppression in macaque inferior temporal cortex. Cerebral Cortex. 2011;21:1547–1558. doi: 10.1093/cercor/bhq207. [DOI] [PubMed] [Google Scholar]
Kamitani Y, Tong F. Decoding the visual and subjective contents of the human brain. Nature Neuroscience. 2005;8:679–685. doi: 10.1038/nn1444. [DOI] [PMC free article] [PubMed] [Google Scholar]
Kaposvari P, Kumar S, Vogels R. Statistical learning signals in macaque inferior temporal cortex. Cerebral Cortex. 2018;28:250–266. doi: 10.1093/cercor/bhw374. [DOI] [PubMed] [Google Scholar]
Kay KN, Naselaris T, Prenger RJ, Gallant JL. Identifying natural images from human brain activity. Nature. 2008;452:352–355. doi: 10.1038/nature06713. [DOI] [PMC free article] [PubMed] [Google Scholar]
Keil A, Debener S, Gratton G, Junghöfer M, Kappenman ES, Luck SJ, Luu P, Miller GA, Yee CM. Committee report: publication guidelines and recommendations for studies using electroencephalography and magnetoencephalography. Psychophysiology. 2014;51:1–21. doi: 10.1111/psyp.12147. [DOI] [PubMed] [Google Scholar]
Keller AJ, Houlton R, Kampa BM, Lesica NA, Mrsic-Flogel TD, Keller GB, Helmchen F. Stimulus relevance modulates contrast adaptation in visual cortex. eLife. 2017;6:e21589. doi: 10.7554/eLife.21589. [DOI] [PMC free article] [PubMed] [Google Scholar]
King JR, Dehaene S. Characterizing the dynamics of mental representations: the temporal generalization method. Trends in Cognitive Sciences. 2014;18:203–210. doi: 10.1016/j.tics.2014.01.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
King JR, Gramfort A, Schurger A, Naccache L, Dehaene S. Two distinct dynamic modes subtend the detection of unexpected sounds. PLoS ONE. 2014;9:e85791. doi: 10.1371/journal.pone.0085791. [DOI] [PMC free article] [PubMed] [Google Scholar]
King JR, Pescetelli N, Dehaene S. Brain mechanisms underlying the brief maintenance of seen and unseen sensory information. Neuron. 2016;92:1122–1134. doi: 10.1016/j.neuron.2016.10.051. [DOI] [PubMed] [Google Scholar]
Kohn A, Movshon JA. Adaptation changes the direction tuning of macaque MT neurons. Nature Neuroscience. 2004;7:764–772. doi: 10.1038/nn1267. [DOI] [PubMed] [Google Scholar]
Kok P, Jehee JF, de Lange FP. Less is more: expectation sharpens representations in the primary visual cortex. Neuron. 2012;75:265–270. doi: 10.1016/j.neuron.2012.04.034. [DOI] [PubMed] [Google Scholar]
Kok P, Mostert P, de Lange FP. Prior expectations induce prestimulus sensory templates. PNAS. 2017;114:10473–10478. doi: 10.1073/pnas.1705652114. [DOI] [PMC free article] [PubMed] [Google Scholar]
Kourtzi Z, Kanwisher N. Representation of perceived object shape by the human lateral occipital complex. Science. 2001;293:1506–1509. doi: 10.1126/science.1061133. [DOI] [PubMed] [Google Scholar]
Kovács G, Kaiser D, Kaliukhovich DA, Vidnyánszky Z, Vogels R. Repetition probability does not affect fMRI repetition suppression for objects. Journal of Neuroscience. 2013;33:9805–9812. doi: 10.1523/JNEUROSCI.3423-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
Kumar S, Kaposvari P, Vogels R. Encoding of predictable and unpredictable stimuli by inferior temporal cortical neurons. Journal of Cognitive Neuroscience. 2017;29:1445–1454. doi: 10.1162/jocn_a_01135. [DOI] [PubMed] [Google Scholar]
Kvale MN, Schreiner CE. Short-term adaptation of auditory receptive fields to dynamic stimuli. Journal of Neurophysiology. 2004;91:604–612. doi: 10.1152/jn.00484.2003. [DOI] [PubMed] [Google Scholar]
Larsson J, Smith AT. fMRI repetition suppression: neuronal adaptation or stimulus expectation? Cerebral Cortex. 2012;22:567–576. doi: 10.1093/cercor/bhr119. [DOI] [PMC free article] [PubMed] [Google Scholar]
Liu T, Larsson J, Carrasco M. Feature-based attention modulates orientation-selective responses in human visual cortex. Neuron. 2007;55:313–323. doi: 10.1016/j.neuron.2007.06.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
Maravall M, Petersen RS, Fairhall AL, Arabzadeh E, Diamond ME. Shifts in coding properties and maintenance of information transmission during adaptation in barrel cortex. PLoS Biology. 2007;5:e19. doi: 10.1371/journal.pbio.0050019. [DOI] [PMC free article] [PubMed] [Google Scholar]
Marti S, King JR, Dehaene S. Time-resolved decoding of two processing chains during dual-task interference. Neuron. 2015;88:1297–1307. doi: 10.1016/j.neuron.2015.10.040. [DOI] [PMC free article] [PubMed] [Google Scholar]
Maunsell JH, Treue S. Feature-based attention in visual cortex. Trends in Neurosciences. 2006;29:317–322. doi: 10.1016/j.tins.2006.04.001. [DOI] [PubMed] [Google Scholar]
Maunsell JHR. Neuronal mechanisms of visual attention. Annual Review of Vision Science. 2015;1:373–391. doi: 10.1146/annurev-vision-082114-035431. [DOI] [PMC free article] [PubMed] [Google Scholar]
Movshon JA, Lennie P. Pattern-selective adaptation in visual cortical neurones. Nature. 1979;278:850–852. doi: 10.1038/278850a0. [DOI] [PubMed] [Google Scholar]
Myers NE, Rohenkohl G, Wyart V, Woolrich MW, Nobre AC, Stokes MG. Testing sensory evidence against mnemonic templates. eLife. 2015;4:e09000. doi: 10.7554/eLife.09000. [DOI] [PMC free article] [PubMed] [Google Scholar]
Näätänen R, Paavilainen P, Rinne T, Alho K. The mismatch negativity (MMN) in basic research of central auditory processing: a review. Clinical Neurophysiology. 2007;118:2544–2590. doi: 10.1016/j.clinph.2007.04.026. [DOI] [PubMed] [Google Scholar]
Oostenveld R, Praamstra P. The five percent electrode system for high-resolution EEG and ERP measurements. Clinical Neurophysiology. 2001;112:713–719. doi: 10.1016/S1388-2457(00)00527-7. [DOI] [PubMed] [Google Scholar]
Paavilainen P. The mismatch-negativity (MMN) component of the auditory event-related potential to violations of abstract regularities: a review. International Journal of Psychophysiology. 2013;88:109–123. doi: 10.1016/j.ijpsycho.2013.03.015. [DOI] [PubMed] [Google Scholar]
Patterson CA, Wissig SC, Kohn A. Distinct effects of brief and prolonged adaptation on orientation tuning in primary visual cortex. Journal of Neuroscience. 2013;33:532–543. doi: 10.1523/JNEUROSCI.3345-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
Pelli DG. The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision. 1997;10:437–442. doi: 10.1163/156856897X00366. [DOI] [PubMed] [Google Scholar]
Rao RP, Ballard DH. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience. 1999;2:79–87. doi: 10.1038/4580. [DOI] [PubMed] [Google Scholar]
Rasmussen RG, Schwartz A, Chase SM. Dynamic range adaptation in primary motor cortical populations. eLife. 2017;6:e21409. doi: 10.7554/eLife.21409. [DOI] [PMC free article] [PubMed] [Google Scholar]
Rentzeperis I, Nikolaev AR, Kiper DC, van Leeuwen C. Relationship between neural response and adaptation selectivity to form and color: an ERP study. Frontiers in Human Neuroscience. 2012;6 doi: 10.3389/fnhum.2012.00089. [DOI] [PMC free article] [PubMed] [Google Scholar]
Saarinen J, Paavilainen P, Schöger E, Tervaniemi M, Näätänen R. Representation of abstract attributes of auditory stimuli in the human brain. NeuroReport. 1992;3:1149–1151. doi: 10.1097/00001756-199212000-00030. [DOI] [PubMed] [Google Scholar]
Salti M, Monto S, Charles L, King J-R, Parkkonen L, Dehaene S. Distinct cortical codes and temporal dynamics for conscious and unconscious percepts. eLife. 2015;4:e05652. doi: 10.7554/eLife.05652. [DOI] [PMC free article] [PubMed] [Google Scholar]
Schacter DL, Buckner RL. Priming and the brain. Neuron. 1998;20:185–195. doi: 10.1016/S0896-6273(00)80448-1. [DOI] [PubMed] [Google Scholar]
Simmons JP, Nelson LD, Simonsohn U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science. 2011;22:1359–1366. doi: 10.1177/0956797611417632. [DOI] [PubMed] [Google Scholar]
Smirnakis SM, Berry MJ, Warland DK, Bialek W, Meister M. Adaptation of retinal processing to image contrast and spatial scale. Nature. 1997;386:69–73. doi: 10.1038/386069a0. [DOI] [PubMed] [Google Scholar]
Spaak E, Watanabe K, Funahashi S, Stokes MG. Stable and dynamic coding for working memory in primate prefrontal cortex. The Journal of Neuroscience. 2017;37:6503–6516. doi: 10.1523/JNEUROSCI.3364-16.2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
Sprague TC, Serences JT. Attention modulates spatial priority maps in the human occipital, parietal and frontal cortices. Nature Neuroscience. 2013;16:1879–1887. doi: 10.1038/nn.3574. [DOI] [PMC free article] [PubMed] [Google Scholar]
Stokes MG, Kusunoki M, Sigala N, Nili H, Gaffan D, Duncan J. Dynamic coding for cognitive control in prefrontal cortex. Neuron. 2013;78:364–375. doi: 10.1016/j.neuron.2013.01.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
Stokes MG, Wolff MJ, Spaak E. Decoding rich spatial information with high temporal resolution. Trends in Cognitive Sciences. 2015;19:636–638. doi: 10.1016/j.tics.2015.08.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
Summerfield C, Trittschuh EH, Monti JM, Mesulam MM, Egner T. Neural repetition suppression reflects fulfilled perceptual expectations. Nature Neuroscience. 2008;11:1004–1006. doi: 10.1038/nn.2163. [DOI] [PMC free article] [PubMed] [Google Scholar]
Summerfield C, Wyart V, Johnen VM, de Gardelle V. Human scalp electroencephalography reveals that repetition suppression varies with expectation. Frontiers in Human Neuroscience. 2011;5:e67. doi: 10.3389/fnhum.2011.00067. [DOI] [PMC free article] [PubMed] [Google Scholar]
Tang MF, Dickinson JE, Visser TA, Badcock DR. The broad orientation dependence of the motion streak aftereffect reveals interactions between form and motion neurons. Journal of Vision. 2015;15:4. doi: 10.1167/15.13.4. [DOI] [PubMed] [Google Scholar]
Todorovic A, van Ede F, Maris E, de Lange FP. Prior expectation mediates neural adaptation to repeated sounds in the auditory cortex: an MEG study. Journal of Neuroscience. 2011;31:9118–9123. doi: 10.1523/JNEUROSCI.1425-11.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
Todorovic A, de Lange FP. Repetition suppression and expectation suppression are dissociable in time in early auditory evoked fields. Journal of Neuroscience. 2012;32:13389–13395. doi: 10.1523/JNEUROSCI.2227-12.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
Tootell RB, Reppas JB, Dale AM, Look RB, Sereno MI, Malach R, Brady TJ, Rosen BR. Visual motion aftereffect in human cortical area MT revealed by functional magnetic resonance imaging. Nature. 1995;375:139–141. doi: 10.1038/375139a0. [DOI] [PubMed] [Google Scholar]
Tootell RB, Hadjikhani NK, Vanduffel W, Liu AK, Mendola JD, Sereno MI, Dale AM. Functional analysis of primary visual cortex (V1) in humans. PNAS. 1998;95:811–817. doi: 10.1073/pnas.95.3.811. [DOI] [PMC free article] [PubMed] [Google Scholar]
Treue S, Martínez Trujillo JC. Feature-based attention influences motion processing gain in macaque visual cortex. Nature. 1999;399:575–579. doi: 10.1038/21176. [DOI] [PubMed] [Google Scholar]
Ulanovsky N, Las L, Nelken I. Processing of low-probability sounds by cortical neurons. Nature Neuroscience. 2003;6:391–398. doi: 10.1038/nn1032. [DOI] [PubMed] [Google Scholar]
Vinken K, Vogels R, Op de Beeck H. Recent visual experience shapes visual processing in rats through stimulus-specific adaptation and response enhancement. Current Biology. 2017;27:914–919. doi: 10.1016/j.cub.2017.02.024. [DOI] [PubMed] [Google Scholar]
Wacongne C, Labyt E, van Wassenhove V, Bekinschtein T, Naccache L, Dehaene S. Evidence for a hierarchy of predictions and prediction errors in human cortex. PNAS. 2011;108:20754–20759. doi: 10.1073/pnas.1117807108. [DOI] [PMC free article] [PubMed] [Google Scholar]
Wolff MJ, Jochim J, Akyürek EG, Stokes MG. Dynamic hidden states underlying working-memory-guided behavior. Nature Neuroscience. 2017;20:864–871. doi: 10.1038/nn.4546. [DOI] [PMC free article] [PubMed] [Google Scholar]

eLife. doi: 10.7554/eLife.33123.016

Decision letter

Editor: Christopher Summerfield¹

Reviewed by: Peter Kok², Hans Op de Beeck³

In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.

Thank you for submitting your article "Prediction Error and Repetition Suppression Have Distinct Effects on Neural Representations of Visual Information" for consideration by eLife. Your article has been reviewed by Timothy Behrens as the Senior Editor, a Reviewing Editor, and three reviewers. The following individuals involved in review of your submission have agreed to reveal their identity: Peter Kok (Reviewer #1); Hamed Nili (Reviewer #3).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

All the reviewers expressed enthusiasm for your work and recognised the quality of the study and the potential import of the findings. Their concerns focused on a number of issues that would need to be addressed for a revised manuscript to be acceptable to eLife.

1) The details of several of the analyses were unclear, as noted by all three reviewers. In particular, exactly what was used for training and what for test was unclear for some of the later figures. There is also a seeming discrepancy between Figure 2C and the text. Like reviewer 1, I was surprised by the form of the ERPs in Figure 2; please clarify how these were obtained, if not by standard methods. Reviewer 2 also questions the interpretation of the results presented in Figure 7; it would be important to address that at revision.

2) The reviewers noted that the Introduction and Discussion section were at times misaligned with the results themselves. In particular, the allusion to predictive coding seemed to be quite loose in places; please make clear which results support predictive coding, and which do not. It would be important for this work to be discussed in the context of previous studies using ERPs/MEG in conjunction with this paradigm, such as Kok et al., 2017. The results described here seem very different from those in that paper (and also discrepant with de Gardelle et al., 2011); the reviewers ask that you at minimum acknowledge and discuss these inconsistencies.

3) Reviewer 2 raises a key issue about the provenance of the neural signals. Are you able to offer any reassurance that these signals are really "perceptual" in origin?

4) Reviewer 3 asks how the decoding varies with the interaction of the two factors. This seems like a potentially important point that is unaddressed in the current version.

5) Please also take care to address the other minor points raised by the reviewers.

Reviewer #1:

This study by Tang et al., investigates the effects of expectation and repetition on the neural processing of visual stimuli. The authors use both conventional EEG amplitude measurements, as well as forward model decoding to study the neural representations of the stimuli. They find that unexpected stimuli are represented substantially better in the neural signal than expected ones – an intriguing result. This is an interesting topic, investigated using excellent methods that seem executed well. I do have some questions, many of which are for clarification.

1) The authors discuss a prediction error (Expected < Unexpected) effect, but from Figure 2C it seems that expected stimuli (blue) evoke more activity than unexpected ones (red). Please explain.

2) I am not an EEG expert, so please excuse me for my naivety, but I would have expected to see ERPs to consist of early components of alternating polarity, i.e. P1-N1-P2-N2-P3. However, here the amplitude of the EEG signal is only ever positive post-stimulus – is this the result of a specific aspect of the paradigm, or the preprocessing of the data?

3) When studying the effect of repetition on orientation-selective signals, can the authors distinguish whether any modulations would be the result of actual adaptation/repetition suppression, versus lingering activity evoked by the first grating affecting the decoding? Of course, neural adaptation in early sensory regions is the thing the authors are interested in – but imagine that the first grating is still being processed in, for instance, parietal and frontal areas at the time the second grating is presented. Then the decoder trying to decode the second grating would pick up, and be influenced by, these parietal and frontal signals evoked by the first grating, even though they do not reflect 'true' repetition suppression or adaptation. Is this a concern?

4) In Figure 6, there is a significant off-axis difference in decoding between expected and unexpected trials, which is very interesting. However, I'm having trouble interpreting the fact that this cluster appears above (rather than below, or both below and above) the diagonal. Which axis reflects 'training time' and which 'testing time'?

5) Figure 7 requires some clarification. In the text, the authors say "Here, we trained the forward encoding model on the orientation of the Gabor that was actually presented (stimulus driven), or on the orientation that was expected based on the first Gabor in the pair (non-stimulus driven)." Which one was true for Figure 7; trained on the presented or expected orientation? Or did this different between quadrants of the matrices? From the text, it seems to me that the decoder is always trained on the T1 orientations – however, if this is true, I don't understand why there is such strong decoder for both unexpected and expected gratings in the off-diagonal quadrants, given that we're only looking at the alternation trials here. That is, given that the T2 orientation different from the T1 orientation, how come it's possible to train on T1 (i.e. negative training times) and decode T2 (positive testing times)? Of course, an argument could be made for the unexpected alternation condition, given that there T1 is expected to repeat, but this is not true for the expected alternation condition.

6) "More interestingly, in the unexpected alternation condition there was significantly better on-axis orientation selectivity for the T1 orientation between 150 and 300 ms after the onset of T2, relative to the expected alternation condition (upper right quadrant of right panel in Figure 7)." So, if I understand correctly, in the post-T2 time window, there is both increased evidence for the T1 orientation (Figure 7), as well as for the T2 orientation (Figure 6), for unexpected compared to expected gratings? This is quite interesting, and surprising. Can the authors confirm that this interpretation of their effects is correct?

7) Was the decoder always trained on an equal mixture of expected and unexpected and repeating and alternating trials, to make sure the decoder was not biased towards one of these categories? For instance, given that expected gratings are much more numerous than unexpected ones, the decoder may be better at decoding expected trials if the trial counts are not balanced during training.

8) "It is noteworthy, however, that in their study Kok et al., (2012) employed a 'backward' decoding analysis to quantify sharpness, rather than forward encoding as here, which might account for the discrepant findings." I'm not sure I follow this reasoning, given that the authors replicate their own findings using a 'backward' decoding analysis as well. Also note that an MEG study using the same paradigm (Kok et al., 2017) did use forward modelling, and also found improved decoding for expected vs. unexpected gratings.

9) "An intriguing possibility is that combining predictions generated across distinct cortical areas (e.g., visual and auditory) leads to sharpening of tuning, whereas predictions generated within a single cortical area lead to gain modulation." This in indeed an intriguing possibility. I would like to note, however, that it seems unlikely that the predictions in the current study are generated in a single cortical region (e.g. V1), given their complexity. After all, it is not simply grating 1 that predicts the orientation of grating 2, but the orientation of grating 1 in conjunction with the current context ("am I in a repetition or alternation block?"). This type of context seems unlikely to be encoded in early visual cortex but may perhaps be signalled by higher level regions in frontal cortex (or elsewhere) instead.

10) "The data were then baseline corrected to the average EEG activity from -100 to 0 ms before the presentation of the second Gabor pair." The phrasing is unclear. Does this mean that activity was baseline before presentation of the first or the second member of each pair?

Reviewer #2:

Tang et al., present an EEG study in which they investigate how prediction error and repetition suppression affect neural representations of visual information. They manipulated two factors: (i) whether two stimuli repeated in a trial or not; (ii) whether a repeat was very likely to occur or not. The authors performed a forward encoding analysis to characterize orientation selectivity. They show that repetition suppression decreases overall responsiveness but does not affect orientation selectivity. In contrast, unexpected stimuli, when compared to expected stimuli, only showed a minor difference in overall responsiveness, but instead seemed to be associated with a larger gain in orientation tuning curves without a change in tuning width.

This study addresses important questions and provides relevant findings on these issues. The experimental design and implementation is done sufficiently well, the analyses are well thought off, and the manuscript is overall clearly written.

Nevertheless, several major revisions will be necessary before the actual implications of the findings are clear. I will first go into a few important concerns.

First, it is unclear to me which neural signal underlies the orientation selectivity that is measured. Even if we would accept that it is a signal from visual regions, it must be very far removed from the responses of single neurons or orientation columns. What are the different possibilities, and what are the implications of this uncertainty for our understanding of the neural correlates of prediction error processing? However, before such questions are addressed, it first has to be clarified to what extent the findings relate to visual processing. If I understand correctly, the forward encoding modelling to estimate orientation selectivity considers all electrodes across the entire brain. Therefore, it is impossible to determine which electrodes contribute to the orientation selectivity and its modulation by expectations. Is it possible to perform further analyses to clarify this? Imagine that the expectation effect would be largely driven by frontal electrodes, then the results would not necessarily tell us much about perceptual processing.

Second, and partially related to the first, subsection “Repetition suppression and prediction error affect the overall level of neural activity” and Figure 2 show that repetition and expectation effects are found both in occipital-parietal and in frontal electrodes. The effect for repetitions in occipital-parietal electrodes (repeat < alternation) is in the expected direction. However, the expectation effect seems to be in the wrong direction in occipital-parietal electrodes (expected > unexpected). From a predictive coding perspective, unexpected repetitions/alternations should result in a higher prediction error and thus higher response. Just like alternation trials result in a higher response. How can we interpret the further multivariate results if this main effect of expectation is hard to reconcile with what we see for repetition suppression and with what we would expect based on e.g. the fMRI experiment of Summerfield et al.,?

Third, and again somewhat related to the previous concern, at many points the relationship between prediction errors, repetition suppression, and neuronal adaptation is not characterized well. A few examples:

In the Abstract prediction errors are claimed to provide a possible explanation of repetition suppression. However, in previous work (as well as in this study), expectation effects are dissociated from repetition suppression. The latter can be explained perfectly well by stimulus driven adaptation effects. These expectation effects are smaller and less robust than stimulus driven adaptation, suggesting a minor modulatory role of perceptual expectation.

Introduction: "Here we asked whether a predictive coding theory can account for the changes in neural representations observed with repetition suppression." How is this question answered?

Fourth, the authors should adjust their claims and interpretation to the strength of the evidence provided, from abstract to Discussion section. First, in order to do this, the statistical reporting should be more specific. The authors should report actual p-values (of course rounded to a reasonable precision), instead of arbitrary thresholds, such as p<.025 or p<.05. Also, for the statistics shown mostly in figures, such as Figure 6 and Figure 7, it would be important to mention the highest p-value obtained, and not just say p<0.05. Most effects seem to be close to 0.05, which should be reflected upon. One p-value signals a very low probability (p =.0049; subsection “Prediction errors increase the amount of orientation-selective information contained within patterns of EEG activity”), however, it is the result of a circular analysis. The time window on which this analyses was done was pre-selected based on a statistical test on the same data. This should be indicated. Overall, the amount of evidence presented in this paper seems sufficient for a publication but not strong enough to just pass without further discussion and adjusting the interpretations to the strength of the evidence.

Fifth, the orientation selectivity is determined and tested using the second gabor in the trials. If you find a difference between two conditions A and B, with A showing better tuning than B, how can you know whether it is A that has an increased tuning or B a decreased tuning? To me it seems that the orientation selectivity on the first gabor in the trials would be a good baseline/reference. So why did you not include the first gabor in the trials in your orientation selectivity analyses? You conclude "prediction error was associated with a significantly increased orientation-selective response through a gain modulation soon after the stimulus was presented", but how can you differentiate this from "The absence of prediction error was associated with a significantly decreased orientation-selective response through a gain modulation soon after the stimulus was presented". When split in subsets of trials according to the relationship with the second stimulus, these analyses on the first gabor would also serve as a useful statistical control for the analyses presented in Figure 2. If the statistical thresholding is stringent enough, then doing the same analysis on the first stimulus should not give false positives.

Reviewer #3:

I found the paper very interesting.

While they show that the univariate effects of repetition and expectation are comparable (same areas, different sign), they show that while repetition doesn't affect the selectivity of representations, expectation leads to an increase in selectivity by gain modulation.

Given the importance of the multivariate analysis in directly leading to the main claim of the paper, I have a few comments on it:

1) The authors use an encoding approach to test for information processing. That is very clearly explained in the Materials and methods section. At the last step, they fit a Gaussian (Equation 4), which has 4 parameters. So, for example Figure 3C quantifies the orientation selectivity by parameter A. Now whilst that is an interesting fact, it might be the case that there are other differences in the two (e.g. that σ is smaller in the repeated than the non-repeated).

2) The multivariate analysis that compares orientation selectivity in repeat/ alternate and expected/unexpected is key to the claims of the paper. However, the authors could have looked at all 4 possible combinations (similar to what they do for the LDA visualized in Figure 5). For example, it would be interesting to test if orientation selectivity in the expected is different when the stimulus is repeated or alternated. Figure 4 would be a good summary if there was no interaction between the two at any time point.

3) Figure 5 is an interesting control analysis. However, there is no information on how they performed cross validation, what time points are averaged to give the result and how stable the results are!

4) Although I find the cross-temporal analysis on the weights from the forward encoding model interesting, I think that some speculations could be directly tested with pattern similarity analysis. For example, for the unexpected stimulus, they could directly test whether the representation of the unexpected is reactivated at 300-400 ms. They could do non-parametric statistics for the inference. I also make the same suggestion for analysis of Figure 7.

[Editors’ note: this article was subsequently rejected after discussions between the reviewers, but the authors were invited to resubmit after an appeal against the decision.]

Thank you for submitting your revised submission entitled "Prediction Error and Repetition Suppression Have Distinct Effects on Neural Representations of Visual Information" for consideration by eLife. Your revised article has been reviewed by three peer reviewers, and the evaluation has been overseen by a Reviewing Editor and a Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Peter Kok (Reviewer #1); Hamed Nili (Reviewer #3).

Our decision has been reached after extensive consultation between the reviewers. Based on these discussions and the individual reviews below, we regret to inform you that we have decided to reject your submission in its current form.

As you can see from the comments below, all of the reviewers – and in particular #1 and #2 – felt that their comments had been insufficiently addressed at revision. The reviewing editor has read your revised manuscript and rebuttal with care and agrees. He has flagged the following major issues as follows:

(i) One major premise of the manuscript is that you claim to see a univariate effect of unexpected > expected trials. However, the plot you present in Figure 2 shows precisely the converse. The reviewers found your argument that this effect reverses over more anterior sites (not shown) unconvincing, particularly given that the major focus of the paper is on predictive coding in hierarchical sensory systems. This, coupled with the unconventional presentation of the ERPs (which do not resemble potentials that the reader would typically expect to see over visual regions), led the reviewers to doubt the claims made with regard to this signal.

(ii) Reviewers #1 and #2 raised substantive issues about the train/test approach that you took for the multivariate analyses. The paradigm you have employed allows the viewer to form expectations about the second grating on the basis of the first. This means that the neural signal elicited by the first grating may partly encode an expectation of the second grating, and the neural signal elicited by the second grating might partly include carryover from the first grating. Uncertainty over whether the effects you described have a straightforward interpretation was exacerbated by the asymmetry in the cross-validation plots. The reviewers remain unconvinced by your rather cursory replies to their questions regarding this point.

(iii) The reviewers still feel that the link to predictive coding is very loose. The manuscript continues to be motivated by the predictive coding framework, even though several of your results seem to directly contradict what might be expected under this theory. The reviewers found the exegesis to be rather contradictory, and the theoretical claims confusing in the light of the results.

(iv) There are continued concerns from reviewers #2 and #3 that your analyses are at least partly circular, and that there may be oddities introduced by your baselining techniques (reviewer #1).

Together, this prompted the decision to reject the manuscript as it stands. If you would like to appeal this decision, you are welcome to do so, but I would suggest that it is unlikely that the reviewers will be sympathetic to an appeal without a serious revision to the paper that addresses their points in full. For my part, I think that this manuscript has promise, but I feel that there are simply too many inconsistencies in the current version to warrant publication in eLife.

Reviewer #1:

Unfortunately, I feel that not all concerns have been satisfactorily addressed. In its current form, I am not fully confident that the results described reflect effects of expectation on perceptual representations. The authors' responses to my concerns expressed in the previous round have not fully taken away these concerns, as I detail below.

Original point 1. I'm still not quite sure how to interpret the amplitude results displayed in Figure 2C-D. It seems to me, from Figure 2C, that expected stimuli evoke a larger response than unexpected ones, yet the authors interpret this as a prediction error effect, with a larger neural response to unexpected stimuli (over frontal electrodes). In their response, they state that there was "more negativity associated with the unexpected stimulus", but I'm not sure how this follows from Figure 2C-D.

Original point 2. According to the authors, the fact that the ERP waveforms is (almost) only positive, rather than a sequence of positive and negative components, is the result of the minimal preprocessing applied (only a 0.5 Hz high-pass filter). However, from my limited experience with EEG, I would expect positive and negative components even with minimal preprocessing. I'm still somewhat confused by this.

Original point 5. I still don't understand why there is such strong off-diagonal decoding in Figure 7, given that the orientation of the second grating was randomized, as the authors state in the legend. In the orientation of the second grating was randomized, how is it possible to decode it when training on the first grating?

In response to my original point #10, the authors explain that the EEG data were baseline corrected to the interval -100 to 0 ms before presentation of the second Gabor in the pair. Isn't this problematic, in a paradigm in which grating 1 can lead to a prediction about grating 2? If there is an expectation signal present just before the appearance of grating 2 (as is quite likely), this would 'contaminate' the baseline, and effects found elsewhere (before -100 ms or after 0 ms) might actually reflect a negative image of expectation effects present in the pre-stimulus 2 interval.

Reviewer #2:

I thank the authors for their revision, which addresses some of the earlier concerns.

There are a few remaining issues, addressing them would improve the manuscript further.

About the localization of the signal:

Central to predictive coding theories is the idea that sensory predictions which are generated higher up in the hierarchy modulate responses in lower levels of sensory cortices. This statement is much more specific than the more general idea that: recent experience establishes expectations in the brain, which manifest somewhere and somehow in the encoding of sensory input. In the current state of the manuscript, this important nuance seems to be missing. Given that predictive coding theories have very specific hierarchical predictions, the argument that the authors wanted to "limit experimenter degrees of freedom potentially introduced through the post-hoc selection of subsets of electrodes" is simply not valid here. This is not a trivial issue, because as long as we do not constrain a theory, it cannot be falsified.

I understand that the authors might not want to or might not be able to localize their effects of expectation, but that limits the extent to which these findings are instructive about predictive coding theory and more generally about how the brain processes unexpected sensory events. Even if the general orientation selectivity is mainly determined by the signal at occipital and parietal electrodes, there is no guarantee that any interaction between expectation and orientation selectivity originates from the same signal in these electrodes. For example, I wonder whether a potential explanation for the discrepancy with Kok et al.'s work could be that their effects were localized to visual cortex? Perhaps in this study their expectation effects do not originate from in visual cortex, but for example only in frontal areas?

Without such information, the findings of this study reduce to: (a) that repetition suppression and expectation effects are dissociable (which already has been demonstrated by for example Todorovic and de Lange, 2012) and (b) expectation increases selectivity for (Gabor) stimuli somewhere in the brain for this particular task. The fact that this would be hard to reconcile with other findings in early visual cortex is then explained away by the argument that "as this is a relatively complex phenomenon it is likely to yield different outcomes when different levels of prediction are manipulated". This reasoning does not echo the idea of predictive coding as universal principle of cortical responses. A good theory should be able to make general predictions that are valid in a (relatively) broad range of situations, without requiring ad hoc explanations for different experiments.

The authors should be more explicit about these problems.

Specific points:

"One mechanism by which the brain reduces its information processing load is to encode successive presentations of the same stimulus in a more efficient form, a process known as neural adaptation": The authors have added references for this sentence, but I think they still can't justify the statement. Right now, the sentence reads like the function of adaptation is a settled case. Given our current lack of understanding of the mechanisms underlying adaptation, any suggestions about its function remain highly speculative. I suggest you change this sentence to "Neural adaptation is one mechanism by which the brain might reduce its information…".

In their rebuttal the authors confirm that they find Expected > Unexpected in occipital-parietal electrodes (while Expected < Unexpected in central frontal electrodes). This is a finding that contradicts the hallmark of predictive coding theory, namely that neural response strength signals prediction error. The authors should address this discrepancy. On the other hand, the authors still use the term "prediction errors" in reference to their results. For example, in the Discussion section they say: "We found that prediction errors were associated with increased gain of stimulus representations…". Yet, as they acknowledge, they can't know where the change in stimulus presentations took place, while they only find prediction errors in a frontal cluster.

Comment 5:

I understand that care should be taken for these comparisons, but still it is very valid to compare with the first stimulus. Take the field of single-unit animal studies of repetition suppression, would they not always show data/analyses for the first stimulus also? I think it is an important statistical control for various effects (as I mentioned in my original email).

Reviewer #3:

I think the revised document is much more clear and transparent.

After reading the responses and details of Figure 5, I have a slight concern that it might suffer from some circularity.

The figure is very clear and the results are consistent. However, the time period for which these results are displayed can affect the trend.

Ideally one would want to see the classification accuracy for different "conditions" as a function of time and see how and when the expected/unexpected x repeat/alternate differ in terms of orientation information. However, I think a figure like Figure 5 is also fine provided that all the information is given in the Materials and methods section and the figure caption.

In the Materials and methods section you can also state if you are averaging in time and then computing the classification accuracy or vice versa.

I think the details of which Matlab function they used is unnecessary, but this is key.

[Editors’ note: what now follows is the decision letter after the authors submitted for further consideration.]

Thank you for choosing to send your work entitled "Prediction Error and Repetition Suppression Have Distinct Effects on Neural Representations of Visual Information" for consideration at eLife.

Your letter of appeal has been considered by a Senior Editor and a Reviewing editor, and we are prepared to consider a revised submission. However, I should point out that the reviewers have requested a new analysis and the reviewers and editors are all in agreement that our continued consideration of this MS would be contingent on the outcome of this analysis.

I paste below the comments from one of our reviewers (reviewer 1; reviewer 2 was broadly happy with the MS you resubmitted). There are 3 points. We agree that points 1 and 3 can be dealt with by acknowledging limitations to the data/paradigm in the discussion or elsewhere in the main text. Point 2 is more thorny and the reviewer requests a further analysis, because the paper seems to hinge principally on this finding. You have argued that baselining in this period is unbiased because of balancing of trial types in your design. The reviewer is worried (and we agree) that this is not an adequate response, because on expected trials, the baselining will remove trial-specific information that relates to the expectation of the stimulus and potentially affect the levels of encoding of expected vs. unexpected stimuli. To be crystal clear, I precis from our discussion:

"The interval the authors use as the baseline for all their analyses (as far as i can tell), is the interval between gratings 1 and 2, i.e. the time at which they can form an expectation about the upcoming stimulus. So, if there is a neural expectation signal in this interval, using this as baseline affects all (decoding) analyses. For instance, in their Figure 4B, decoding of expected gratings is worse than unexpected – could this be because for the expected trials they've subtracted a pre-stimulus expectation signal that was present in the baseline?"

The reviewer requests that you replicate this analysis baselining from a different period (e.g. before stimulus onset when there is no risk of contamination with expectation signals). Contingent on this analysis working (which, in the event that your explanations were correct, it certainly should), and on satisfactory addressing of points 1 and 3 with discussion of limitations, I think we would be happy to go forward with the paper. I think if an analysis of the sort suggested does not work, the reviewers might be skeptical about whether the claims in the paper stand up.

Reviewer 1:

I find the univariate EEG much easier to interpret now, and I applaud the authors for making the effort to improve this. However, I still have concerns about the decoding analyses.

1) With regard to the strong off-diagonal decoding, it seems the authors themselves do not fully understand this either. They suggest it may be to do with the ongoing processing of grating 1 during the processing of grating 2, but would such ongoing late processing really be expected to lead to almost as strong off-diagonal as diagonal decoding, as Figure 7 suggests? I also find the very strong decoding signals in the pre-grating1 testing time disturbing; why would there be decoding prior to presentation of the first grating? (I come back to this below as well.)

2) In response to my concern about the baseline procedure, i.e. baselining on the interval between the two gratings, the authors state that this is not a concern, since the trials were balanced "so there was an equal number of different orientations of the initial grating leading to each subsequent (unexpected) grating within a pair." I don't understand how this addresses my concerns. They say, "the difference between the conditions only arises after presentation of the second stimulus in the pair". While that may be true when comparing an expected repetition to an unexpected alternation, and an expected alternation to an unexpected repetition (i.e., in those comparisons, participants had the same expectation), it is *not* true when comparing e.g. an expected repetition to an unexpected repetition. To be clear, I am not raising a concern about the signal from grating 1 itself carrying over into the grating 2 period, I am concerned about an expectation signal, induced by grating 1 and the context (repetition vs. alternation block), being present in the interval between gratings 1 and 2, i.e. the baseline period.

To be specific, what I am concerned about is that there may be a neural signal in the interval between the two gratings that reflects the expected orientation. This does not need to be carry-over from the bottom-up signal caused by the first grating, it may be a genuine top-down expectation induced template of the expected orientation. If such a pre-second-stimulus expectation template exists, then in the 'Expected repetition' trials the current baselining procedure would lead to confounds, since this template is in the baseline period, and therefore a negative reflection of this template would be introduced throughout the rest of the trial timeline, i.e. during grating 1 and grating 2. This process would affect the different conditions differently, since in 'Unexpected repetition' and 'Expected alternation' trials participants cannot form an expectation of a specific trials, hence no expectation template would be expected to be present, and in the 'Unexpected alternation' trials a template might be present, but it doesn't match the presented orientation (of the second grating). Regardless of this specific confound, in more general terms baselining on a period during which participants form expectations in a subset of the trials seems problematic. The only way I would see of addressing this issue, is to baseline the trials on the EEG data prior to the first grating, when no expectation can be formed yet. (Though from their response to some of the other points, I take the authors as saying that the relationship between the orientation of the second grating on trial n and the first grating on trial n+1 is not perfectly controlled, in which case this approach would not be valid. And indeed, in Figure 7, there seems to be strong decoding signal prior to the presentation of grating 1, which would either be caused by some relationship between grating 2 on the previous trial and grating 1 on the current trial, or by the baselining concern I raise above.)

3) This is a more general point about the experiment design, that I didn't raise before as I only realised it fully in this round. Expected alternation trials were as follows: a certain orientation was presented as the first grating, and then participants could form the expectation that it would alternate, meaning that it was equally likely to be any other orientation. So that's a very unspecific expectation. By contrast, in a previous study orthogonalising expectation and repetition (Todorovic and De Lange), 'expected alternations' meant that participants had a 75% valid expectation of exactly which stimulus would appear. That is, in that study, participants had a 75% chance of predicting the exact stimulus coming up in both expected repetition and expected alternation trials, making those two conditions nicely symmetrical. In the current study however, expected alternation trials ('any orientation can come up, just probably not the one I've just seen') are not at all similar to expected repetition trials, and it seems a bit of a stretch to average these two conditions together and call them 'expected' orientations. In other words, I'm not sure how successful the current design is in orthogonalising expectation and repetition.

eLife. 2018 Dec 14;7:e33123. doi: 10.7554/eLife.33123.017

Author response

All the reviewers expressed enthusiasm for your work and recognised the quality of the study and the potential import of the findings. Their concerns focused on a number of issues that would need to be addressed for a revised manuscript to be acceptable to eLife.

1) The details of several of the analyses were unclear, as noted by all three reviewers. In particular, exactly what was used for training and what for test was unclear for some of the later figures. There is also a seeming discrepancy between Figure 2C and the text. Like reviewer 1, I was surprised by the form of the ERPs in Figure 2; please clarify how these were obtained, if not by standard methods. Reviewer 2 also questions the interpretation of the results presented in Figure 7; it would be important to address that at revision.

We have extensively clarified, throughout the revision, the exact nature of test and training datasets and the backwards decoding analysis. We also provide full details on the statistical tests used throughout the paper, and have clarified the form of the ERPs displayed (see our responses to reviewer 1, comment 2; and reviewer 3, comment 3).

2) The reviewers noted that the Introduction and Discussion section were at times misaligned with the results themselves. In particular, the allusion to predictive coding seemed to be quite loose in places; please make clear which results support predictive coding, and which do not. It would be important for this work to be discussed in the context of previous studies using ERPs/MEG in conjunction with this paradigm, such as Kok et al., 2017. The results described here seem very different from those in that paper (and also discrepant with de Gardelle et al., 2011); the reviewers ask that you at minimum acknowledge and discuss these inconsistencies.

We have revised the terminology used throughout the manuscript and have focused in particular on our use of the term “predictive coding”. We have now better aligned our experimental manipulations and results with the content of the Introduction and Discussion section. In the revision, we have drawn attention to the differences between the findings of Kok et al., (2017) and the results of the current study (see responses to reviewer 1, comment 9).

3) Reviewer 2 raises a key issue about the provenance of the neural signals. Are you able to offer any reassurance that these signals are really "perceptual" in origin?

Forward encoding modelling uses linear regression to find patterns of neural activity selective for stimulus features, in this case orientations. The analysis, therefore, focuses on how repetition and expectation affect stimulus features, which are perceptual features. There is some suggestion that this signal is most apparent in electrodes over the occipital-parietal cortex, but the low spatial resolution of EEG makes us reluctant to draw finer grained anatomical distinctions about the exact cortical loci of the effects we observed; indeed, this was not the central aim of our study. We have extensively revised the Discussion section to clarify the analyses and also to highlight any limitations in the conclusions we can draw (see responses to reviewer 2, comments 1-3).

4) Reviewer 3 asks how the decoding varies with the interaction of the two factors. This seems like a potentially important point that is unaddressed in the current version.

We have included a number of additional analyses in the revised manuscript that test for interactions between repetition suppression and expectation. We used a number of different approaches to examine these effects, but consistently found no significant interaction between the two factors. These analyses support the major conclusion of the current work, namely, that repetition suppression and expectation have distinct effects on sensory representations (see our response to reviewer 3, comment 2).

Reviewer #1:

This study by Tang et al., investigates the effects of expectation and repetition on the neural processing of visual stimuli. The authors use both conventional EEG amplitude measurements, as well as forward model decoding to study the neural representations of the stimuli. They find that unexpected stimuli are represented substantially better in the neural signal than expected ones – an intriguing result. This is an interesting topic, investigated using excellent methods that seem executed well. I do have some questions, many of which are for clarification.

1) The authors discuss a prediction error (Expected < Unexpected) effect, but from Figure 2C it seems that expected stimuli (blue) evoke more activity than unexpected ones (red). Please explain.

The prediction effect in which the unexpected stimulus evoked a more positive response than an expected stimulus was found over a cluster of central-frontal electrodes. Over the occipital channels, there was more negativity associated with the unexpected stimulus. For Figure 2C, we plotted the effect from a collection of occipital-parietal sensors (O1, O2, Oz, POz, PO7, PO3, PO8, PO4) to facilitate comparison with the repetition suppression effect. If we had used a bandpass filter there would have been a larger negativity for unexpected stimuli (relative to expected stimuli) over these electrodes.

2) I am not an EEG expert, so please excuse me for my naivety, but I would have expected to see ERPs to consist of early components of alternating polarity, i.e. P1-N1-P2-N2-P3. However, here the amplitude of the EEG signal is only ever positive post-stimulus – is this the result of a specific aspect of the paradigm, or the preprocessing of the data?

This relates to the previous question. The shapes of the ERPs in the present study are driven by two main factors: (1) the brief (100 ms) presentation time of the grating stimuli, and (2) the nature of the pre-processing steps. The traditional shape of ERPs comes largely from bandpass filtering the waveform, generally between 2-5 Hz and 20-40 Hz. This filtering yields the familiar positive and negative deflections in the waveform. In the current study, only a 0.5 Hz high-pass filter was applied at the beginning of pre- pass filter because we did not wish to quantify ERP components. Instead we used a contemporary cluster-based permutation approach across electrodes and time (Oostenveld et al., (2011), to determine whether there were differences between conditions for the ERPs. We wanted to keep the data as close to their ‘raw’ form as possible for use in the forward encoding modelling, which was the main focus of the study, so we could be as near to the actual brain activity as possible and avoid potential confounds that filtering can introduce.

3) When studying the effect of repetition on orientation-selective signals, can the authors distinguish whether any modulations would be the result of actual adaptation/repetition suppression, versus lingering activity evoked by the first grating affecting the decoding? Of course, neural adaptation in early sensory regions is the thing the authors are interested in – but imagine that the first grating is still being processed in, for instance, parietal and frontal areas at the time the second grating is presented. Then the decoder trying to decode the second grating would pick up, and be influenced by, these parietal and frontal signals evoked by the first grating, even though they do not reflect 'true' repetition suppression or adaptation. Is this a concern?

This is an excellent question. It is unlikely that any lingering effects of the first stimulus would have contaminated the neural response because all trial types, both repeating and alternating, were used to train the encoding model. The difference between the orientations of the gratings in the alternating pairs were evenly balanced. This should have penalised the model from finding spatial representations for orientations carried by the preceding stimulus, because in the alternating trials these were uncorrelated with the inputs to the model. This issue might have posed a potential problem had we separately trained the model on each condition.

4) In Figure 6, there is a significant off-axis difference in decoding between expected and unexpected trials, which is very interesting. However, I'm having trouble interpreting the fact that this cluster appears above (rather than below, or both below and above) the diagonal. Which axis reflects 'training time' and which 'testing time'?

We have updated the axis labels in Figure 6 to show training and testing times. We found significant off-axis orientation selectivity when we generalised an early training time to a later test time, but not vice versa (training at a later time and testing at an earlier time). We believe that we found this because the EEG response to the grating was stronger and more consistent shortly after the presentation of the grating, leading to more stable orientation encoding, whereas these responses were less correlated at later time points. The stronger, more correlated signal leads to better encoding performance which helps with generalization. This can lead to asymmetric temporal generalization.

5) Figure 7 requires some clarification. In the text, the authors say "Here, we trained the forward encoding model on the orientation of the Gabor that was actually presented (stimulus driven), or on the orientation that was expected based on the first Gabor in the pair (non-stimulus driven)." Which one was true for Figure 7; trained on the presented or expected orientation? Or did this different between quadrants of the matrices? From the text, it seems to me that the decoder is always trained on the T1 orientations – however, if this is true, I don't understand why there is such strong decoder for both unexpected and expected gratings in the off-diagonal quadrants, given that we're only looking at the alternation trials here. That is, given that the T2 orientation different from the T1 orientation, how come it's possible to train on T1 (i.e. negative training times) and decode T2 (positive testing times)? Of course, an argument could be made for the unexpected alternation condition, given that there T1 is expected to repeat, but this is not true for the expected alternation condition.

We apologise for this ambiguity. For this particular analysis we used the orientation of the first Gabor in the pair and compared times when this was expected to repeat (unexpected alternation) and expected to change (expected alternation).

We have rewritten this section (copied below) in the revision to clarify these issues.

“Here, we trained the forward encoding model on the orientation of the first Gabor in the pair to determine how the interaction between the expectation and the incoming sensory information was compared. We anticipated a lower expectation that the orientation would repeat for an expected alternating trial, relative to trials in the unexpected alternation condition, where the repeat should have been expected.”

6) "More interestingly, in the unexpected alternation condition there was significantly better on-axis orientation selectivity for the T1 orientation between 150 and 300 ms after the onset of T2, relative to the expected alternation condition (upper right quadrant of right panel in Figure 7)." So, if I understand correctly, in the post-T2 time window, there is both increased evidence for the T1 orientation (Figure 7), as well as for the T2 orientation (Figure 6), for unexpected compared to expected gratings? This is quite interesting, and surprising. Can the authors confirm that this interpretation of their effects is correct?

Yes, this is our interpretation of this result. We interpret the increased information about both the first and second Gabors as reflecting what the larger MMN response represents. The second Gabor is registered as an error and the prior is updated in light of this error and the previous expectation.

7) Was the decoder always trained on an equal mixture of expected and unexpected and repeating and alternating trials, to make sure the decoder was not biased towards one of these categories? For instance, given that expected gratings are much more numerous than unexpected ones, the decoder may be better at decoding expected trials if the trial counts are not balanced during training.

We trained across all types of trials (expected/unexpected, repeated/alternating, and target trials) before splitting the results by trial type. We did this so we had equal power to detect effects across conditions, and to avoid biasing the results in favour of any particular outcome. Note, for example, that we actually observed better orientation selectivity for unexpected stimuli than for expected stimuli, which is the opposite of what one might predict if the results were being driven purely by trial numbers across conditions. We have clarified the nature of the training and test sets in the revised manuscript (subsection “Forward encoding modelling”).

8) "It is noteworthy, however, that in their study Kok et al., (2012) employed a 'backward' decoding analysis to quantify sharpness, rather than forward encoding as here, which might account for the discrepant findings." I'm not sure I follow this reasoning, given that the authors replicate their own findings using a 'backward' decoding analysis as well. Also note that an MEG study using the same paradigm (Kok et al., 2017) did use forward modelling, and also found improved decoding for expected vs. unexpected gratings.

We agree with this point. We have removed these sentences from the Discussion section of the revised manuscript, and added the reference to recently published Kok et al., (2017) paper.

9) "An intriguing possibility is that combining predictions generated across distinct cortical areas (e.g., visual and auditory) leads to sharpening of tuning, whereas predictions generated within a single cortical area lead to gain modulation." This in indeed an intriguing possibility. I would like to note, however, that it seems unlikely that the predictions in the current study are generated in a single cortical region (e.g. V1), given their complexity. After all, it is not simply grating 1 that predicts the orientation of grating 2, but the orientation of grating 1 in conjunction with the current context ("am I in a repetition or alternation block?"). This type of context seems unlikely to be encoded in early visual cortex but may perhaps be signalled by higher level regions in frontal cortex (or elsewhere) instead.

The reviewer is absolutely correct, and in fact this was the sentiment we wished to convey in the original manuscript. We apologise for the lack of clarity. We have now rewritten the relevant paragraph in the revised manuscript to better explain our interpretation (Discussion section).

10) "The data were then baseline corrected to the average EEG activity from -100 to 0 ms before the presentation of the second Gabor pair." The phrasing is unclear. Does this mean that activity was baseline before presentation of the first or the second member of each pair?

Thank you for drawing this ambiguity to our attention. We have revised the sentence to read: “The data were then baseline corrected to the mean EEG activity from -100 to 0 ms before the presentation of the second Gabor in the pair.”

Reviewer #2:

Tang et al., present an EEG study in which they investigate how prediction error and repetition suppression affect neural representations of visual information. They manipulated two factors: (i) whether two stimuli repeated in a trial or not; (ii) whether a repeat was very likely to occur or not. The authors performed a forward encoding analysis to characterize orientation selectivity. They show that repetition suppression decreases overall responsiveness but does not affect orientation selectivity. In contrast, unexpected stimuli, when compared to expected stimuli, only showed a minor difference in overall responsiveness, but instead seemed to be associated with a larger gain in orientation tuning curves without a change in tuning width.

This study addresses important questions and provides relevant findings on these issues. The experimental design and implementation is done sufficiently well, the analyses are well thought off, and the manuscript is overall clearly written.

Nevertheless, several major revisions will be necessary before the actual implications of the findings are clear. I will first go into a few important concerns.

First, it is unclear to me which neural signal underlies the orientation selectivity that is measured. Even if we would accept that it is a signal from visual regions, it must be very far removed from the responses of single neurons or orientation columns. What are the different possibilities, and what are the implications of this uncertainty for our understanding of the neural correlates of prediction error processing? However, before such questions are addressed, it first has to be clarified to what extent the findings relate to visual processing. If I understand correctly, the forward encoding modelling to estimate orientation selectivity considers all electrodes across the entire brain. Therefore, it is impossible to determine which electrodes contribute to the orientation selectivity and its modulation by expectations. Is it possible to perform further analyses to clarify this? Imagine that the expectation effect would be largely driven by frontal electrodes, then the results would not necessarily tell us much about perceptual processing.

As with all non-invasive imaging methods used in humans, including EEG, fMRI and MEG, the neural signals measured are a proxy for a variety of biological signals, including neuronal spiking, local field potentials, etc. Here we chose EEG for its fine temporal resolution as we wished to examine the time course rather than the discrete anatomical locus of repetition and expectation effects in response to simple visual stimuli. As noted by the reviewer, therefore, we need to be cautious in attributing any measured effects to distinct neuronal populations. Only invasive methods employed in animal models can achieve this kind of precision. On the other hand, it is reasonable for us to want to better understand the nature of prediction and repetition effects in people, and indeed there is a rich published history using these non-invasive approaches in humans, much of which we cite in our manuscript.

With regard to our study, we estimated orientation-selectivity using all EEG electrodes across the scalp, consistent with previous pioneering work that has employed this data-driven approach (Garcia et al., 2013; Myers et al., 2015). Figure 3D shows how the model allocates weights to each electrode across subjects at two different time points (50 – 100 ms, and 150 – 200 ms). As is clear from these topographies, the electrodes over occipital and parietal regions are more highly weighted than electrodes over other regions, including frontal areas, suggesting that it is these posterior electrodes that contribute most to the orientation-selective response we measured. This finding is consistent with previously-published work showing orientation selectivity over visual areas in EEG, MEG and fMRI recordings (Cichy et al., 2015; Cichy et al., 2014; Marti and Dehaene, 2017; Stokes et al., 2015).

Having said this, we remain agnostic as to whether more anterior brain areas might also contribute to the effects we found. As noted above, non-invasive imaging measures, including fMRI, provide only a proxy for activity at the level of single neurons. Indeed, the sluggish BOLD response measured with fMRI would not be capable of revealing the fine temporal structure of repetition and expectation effects we have reported here using EEG. It is for future investigations to determine whether individual neurons in visual and non-visual areas might show activity consistent with the whole-brain patterns we have uncovered. To this end, we have recently commenced experiments in which we are measuring neural activity using fMRI in humans and calcium imaging in mice, with a view to determining the effects of repetition and expectation in early visual areas. Preliminary results suggest that prediction effects do indeed arise in primary visual cortex.

We have added a paragraph in the revision discussion to clarify these issues (Discussion section).

Second, and partially related to the first, subsection “Repetition suppression and prediction error affect the overall level of neural activity” and Figure 2 show that repetition and expectation effects are found both in occipital-parietal and in frontal electrodes. The effect for repetitions in occipital-parietal electrodes (repeat < alternation) is in the expected direction. However, the expectation effect seems to be in the wrong direction in occipital-parietal electrodes (expected > unexpected). From a predictive coding perspective, unexpected repetitions/alternations should result in a higher prediction error and thus higher response. Just like alternation trials result in a higher response. How can we interpret the further multivariate results if this main effect of expectation is hard to reconcile with what we see for repetition suppression and with what we would expect based on e.g. the fMRI experiment of Summerfield et al.,?

Please see our answer to this question in response to reviewer 1, comment 1.

Third, and again somewhat related to the previous concern, at many points the relationship between prediction errors, repetition suppression, and neuronal adaptation is not characterized well. A few examples: In the Abstract prediction errors are claimed to provide a possible explanation of repetition suppression. However, in previous work (as well as in this study), expectation effects are dissociated from repetition suppression. The latter can be explained perfectly well by stimulus driven adaptation effects. These expectation effects are smaller and less robust than stimulus driven adaptation, suggesting a minor modulatory role of perceptual expectation.

The reviewer is correct that our study, as well as previous work, has suggested that expectation and repetition suppression are separable. However, to our knowledge no previous study has examined how these factors influence the representation of sensory information (as opposed to overall changes in response amplitude). It is important to test this idea, as a strong version of predictive coding theory argues that expectation can fully explain repetition suppression. From this standpoint, adaptation is explainable by expectation; the system expects to see a repeat, and when it occurs the neuronal response is reduced. We believe it is important to test whether, and how, expectations modulate adaptation. We have revised the Abstract, Introduction and Discussion section to clarify how the current study addresses this issue.

Introduction: "Here we asked whether a predictive coding theory can account for the changes in neural representations observed with repetition suppression." How is this question answered?

The main conclusion of our study is that prediction error, but not repetition suppression, affects the neural representation of basic perceptual information (in this case, the gain associated with orientation tuning). This suggests that they are indeed separable processes. We have clarified this issue in the revision (Discussion section).

Fourth, the authors should adjust their claims and interpretation to the strength of the evidence provided, from abstract to Discussion section. First, in order to do this, the statistical reporting should be more specific. The authors should report actual p-values (of course rounded to a reasonable precision), instead of arbitrary thresholds, such as p<.025 or p<.05. Also, for the statistics shown mostly in figures, such as Figure 6 and Figure 7, it would be important to mention the highest p-value obtained, and not just say p<0.05. Most effects seem to be close to 0.05, which should be reflected upon. One p-value signals a very low probability (p =.0049; subsection “Prediction errors increase the amount of orientation-selective information contained within patterns of EEG activity”), however, it is the result of a circular analysis. The time window on which this analyses was done was pre-selected based on a statistical test on the same data. This should be indicated. Overall, the amount of evidence presented in this paper seems sufficient for a publication but not strong enough to just pass without further discussion and adjusting the interpretations to the strength of the evidence.

We now have included exact p-values for all reported statistical tests.

In the revised manuscript, we have provided a more thorough description of the analysis reported in subsection “Prediction errors increase the amount of orientation-selective information contained within patterns of EEG activity””. Briefly, we initially tested all four parameters of the Gaussian fits to the data (amplitude/gain, centre orientation, width/tuning, and baseline) using non-permutation cluster-based testing (Oostenveld et al., 2011; Wolff et al., 2017), which is a conservative analytic approach. This yielded a difference between expected and unexpected stimuli at an early time point for the amplitude (gain) parameter, but not for the width parameter. We averaged the data across the significant time points to increase signal-to-noise, then applied a conventional t-test to each of the four parameters again to ensure we did not miss any small but significant effects. Given that we had already found a significant main effect of amplitude (gain) in the initial (conservative) cluster-based test, the conclusions we draw from the subsequent t tests do not alter the central conclusion of a change in gain with expectation.

Fifth, the orientation selectivity is determined and tested using the second gabor in the trials. If you find a difference between two conditions A and B, with A showing better tuning than B, how can you know whether it is A that has an increased tuning or B a decreased tuning? To me it seems that the orientation selectivity on the first gabor in the trials would be a good baseline/reference. So why did you not include the first gabor in the trials in your orientation selectivity analyses? You conclude "prediction error was associated with a significantly increased orientation-selective response through a gain modulation soon after the stimulus was presented", but how can you differentiate this from "The absence of prediction error was associated with a significantly decreased orientation-selective response through a gain modulation soon after the stimulus was presented". When split in subsets of trials according to the relationship with the second stimulus, these analyses on the first gabor would also serve as a useful statistical control for the analyses presented in Figure 2. If the statistical thresholding is stringent enough, then doing the same analysis on the first stimulus should not give false positives.

Using the first target as the comparison stimulus would be misleading in this context, for the following reason. We inserted a randomised inter-trial interval of 600 – 1250 ms (i.e., between the second stimulus in one pair and the first stimulus in the next pair), but there was a fixed interval (500 ms) between the first and second stimuli within each pair. This design feature means that there was less consistent ongoing activity for the first grating compared with the consistent activity before the second grating. This difference is likely to make the suggested comparison problematic. The issue is avoided by comparing how the representation of the second grating is affected by the different stimulus manipulations which have matched properties. We have now clarified throughout the revision that our finding of increased orientation selectivity is relative to the expected stimuli.

Reviewer #3:

I found the paper very interesting.

While they show that the univariate effects of repetition and expectation are comparable (same areas, different sign), they show that while repetition doesn't affect the selectivity of representations, expectation leads to an increase in selectivity by gain modulation.

Given the importance of the multivariate analysis in directly leading to the main claim of the paper, I have a few comments on it:

1) The authors use an encoding approach to test for information processing. That is very clearly explained in the Materials and methods section. At the last step, they fit a Gaussian (Equation 4), which has 4 parameters. So, for example Figure 3C quantifies the orientation selectivity by parameter A. Now whilst that is an interesting fact, it might be the case that there are other differences in the two (e.g. that σ is smaller in the repeated than the non-repeated).

This is an excellent point. We have now included this analysis in the revised manuscript (subsection “Prediction errors increase the amount of orientation-selective information contained within patterns of EEG activity”). Our original conclusions remain unchanged.

2) The multivariate analysis that compares orientation selectivity in repeat/ alternate and expected/unexpected is key to the claims of the paper. However, the authors could have looked at all 4 possible combinations (similar to what they do for the LDA visualized in Figure 5). For example, it would be interesting to test if orientation selectivity in the expected is different when the stimulus is repeated or alternated. Figure 4 would be a good summary if there was no interaction between the two at any time point.

Thank you for this suggestion. We had in fact already performed precisely your suggested analyses but chose not to include the details in the original manuscript. We have now included these further results (subsection “Prediction errors increase the amount of orientation-selective information contained within patterns of EEG activity”), as requested. To summarise, using a number of different approaches we found no evidence for a significant interaction between the factors at any time point.

3) Figure 5 is an interesting control analysis. However, there is no information on how they performed cross validation, what time points are averaged to give the result and how stable the results are!

We apologise for omitting this information. We have now included a detailed description of the methods in the revised manuscript (see subsection “Multivariate pattern analysis”).

4) Although I find the cross-temporal analysis on the weights from the forward encoding model interesting, I think that some speculations could be directly tested with pattern similarity analysis. For example, for the unexpected stimulus, they could directly test whether the representation of the unexpected is reactivated at 300-400 ms. They could do non-parametric statistics for the inference. I also make the same suggestion for analysis of Figure 7.

We thank the reviewer for making this suggestion, as we are not experts in RSA. We conducted the analysis in Figure 6 and Figure 7 to determine whether the same pattern of neural response was stable over time. Consistent with previous work, we found that selectivity was best on-axis, but that there was still significant generalization of these classifiers. We found that generalization is enhanced when there is an unexpected stimulus, and because this pattern of activity was orientation selective, we concluded that the same pattern of neural activity was re-activated. We are hesitant to introduce yet another analysis approach in the current manuscript, because it would require a different approach to training the model, and there are already a large number of distinct analyses in the current study.

[Editors’ note: the author responses to the second round of peer review follow.]

Our decision has been reached after extensive consultation between the reviewers. Based on these discussions and the individual reviews below, we regret to inform you that we have decided to reject your submission in its current form.

As you can see from the comments below, all of the reviewers – and in particular #1 and #2 – felt that their comments had been insufficiently addressed at revision. The reviewing editor has read your revised manuscript and rebuttal with care and agrees. He has flagged the following major issues major issues are as follows:

(i) One major premise of the manuscript is that you claim to see a univariate effect of unexpected > expected trials. However, the plot you present in Figure 2 shows precisely the converse. The reviewers found your argument that this effect reverses over more anterior sites (not shown) unconvincing, particularly given that the major focus of the paper is on predictive coding in hierarchical sensory systems. This, coupled with the unconventional presentation of the ERPs (which do not resemble potentials that the reader would typically expect to see over visual regions), led the reviewers to doubt the claims made with regard to this signal.

We originally presented event-related potentials (ERPs) that had not been bandpass filtered, in contrast to the convention for showing mismatch negativity waveforms. This caused the ERPs to have a positive polarity over occipital channels. We took this approach because bandpass filtering can introduce artefacts, and because such filtering is not necessary when peak analyses are not being undertaken. However, to address the concerns of reviewers 1 and 2, we now show the ERP results in the conventional manner. The ERPs were bandpass filtered so that they produced the stereotypic peaks and troughs, as shown in the revised Figure 2. As expected, a greater negativity was found for unexpected than for expected stimuli (Figure 2C) over occipital-parietal electrodes. The ERPs we show are thus entirely consistent with those reported previously for the visual MMN (Näätänen et al., 2007; Saarinen et al., 1992).

(ii) Reviewers #1 and #2 raised substantive issues about the train/test approach that you took for the multivariate analyses. The paradigm you have employed allows the viewer to form expectations about the second grating on the basis of the first. This means that the neural signal elicited by the first grating may partly encode an expectation of the second grating, and the neural signal elicited by the second grating might partly include carryover from the first grating. Uncertainty over whether the effects you described have a straightforward interpretation was exacerbated by the asymmetry in the cross-validation plots. The reviewers remain unconvinced by your rather cursory replies to their questions regarding this point.

As we highlight in detail in our response to reviewer 1 (Comment 4), we employed a balanced number of orientations for the initial grating within each pair. The orientations of these initial gratings were not only uncorrelated over trials, they were also balanced for every orientation of the (unexpected) second grating. Critically, we used a regression-based method to recover orientation selectivity, so that the effects of the first stimulus on the second would sum to zero across trials, and thus could not bias the results for encoding of the second stimulus within each pair.

Nevertheless, despite these points in favour of the analytic approach we used, we have now also conducted the exact analysis suggested by reviewer 2 (Comment 4). This revealed the pattern of results expected by the reviewer (as outlined in detail below) but suffers from a stimulus confound as we argued previously.

Having undertaken this analysis as requested, we again wish to point out that the paradigm was deliberately designed such that the orientation leading into the second grating was precisely matched across conditions. This allowed us to directly compare the conditions of interest without any stimulus confounds. By contrast, it is important to note that the first grating within each pair was subject to different conditions across the trials. For example, the interval between the offset of the second grating in the preceding trial and the onset of the first grating in the next trial was selected randomly between 650 and 750 ms. Thus, by definition, the appearance of the first grating within each pair was inherently unpredictable (because the timing of its onset was randomised across trials). Moreover, no effort was made to control the orientation of the stimulus preceding the first grating in each pair. We therefore stand by our contention that the results of our original analysis are unconfounded, whereas those arising from the suggested additional analysis are. If the reviewers and editors feel it is imperative we report this additional analysis in the revised manuscript we shall do so, but at this point we have not included it.

(iii) The reviewers still feel that the link to predictive coding is very loose. The manuscript continues to be motivated by the predictive coding framework, even though several of your results seem to directly contradict what might be expected under this theory. The reviewers found the exegesis to be rather contradictory, and the theoretical claims confusing in the light of the results.

As outlined in detail below, we have endeavoured to provide a more nuanced account of predictive coding theory and the hypotheses that arise from it in terms of feature encoding, as addressed in our manuscript. In particular, we highlight that predictive coding theory has been largely based on analyses of the overall magnitude of neural responses to unexpected stimuli in human neuroimaging investigations. Critically, previous studies have not determined how prediction and repetition affect the quality of neural representations of elementary stimulus features such as orientation, as we have done. Our data are consistent with some of the predictions arising from the theory, and are inconsistent with others. We have now highlighted these inconsistencies in the revised manuscript and have drawn attention to some of the constraints our data impose on the theory. Note, however, that our aim in framing our work in terms of predictive coding should not be construed as endorsement of the theory. We remain agnostic as to whether predictive coding theory provides an optimal conceptualisation of the effects of repetition and expectation on neural responses and believe the results we report add value to the literature irrespective of this influential framework.

(iv) There are continued concerns from reviewers #2 and #3 that your analyses are at least partly circular, and that there may be oddities introduced by your baselining techniques (reviewer #1).

We deliberately took a conservative approach in our statistical analyses, and were careful to avoid circularity. As outlined above, we have now conducted the analysis suggested by reviewer 2. As for reviewer 3, we have implemented further checks to avoid circularity. We selected the peak classification accuracy for each participant across a wide timing window, and this window was identical between conditions (Repeat/Alternate and Expected/Unexpected). We did not select time points based on differences between the conditions, but instead found the peak classification accuracy across the trial. We have included full details of our statistical choices in the Results section and Materials and methods sections. In response to reviewer 1, we have made explicit that an equal number of orientations went into every prediction error, and these were uncorrelated across trials for each observer. As we employed a regression-based forward encoding approach, any influence of the first and second gratings will cancel to zero and thus not systematically bias responses to the second grating.

Together, this prompted the decision to reject the manuscript as it stands. If you would like to appeal this decision, you are welcome to do so, but I would suggest that it is unlikely that the reviewers will be sympathetic to an appeal without a serious revision to the paper that addresses their points in full. For my part, I think that this manuscript has promise, but I feel that there are simply too many inconsistencies in the current version to warrant publication in eLife.

Reviewer #1:

Unfortunately, I feel that not all concerns have been satisfactorily addressed. In its current form, I am not fully confident that the results described reflect effects of expectation on perceptual representations. The authors' responses to my concerns expressed in the previous round have not fully taken away these concerns, as I detail below.

1) Original point 1. I'm still not quite sure how to interpret the amplitude results displayed in Figure 2C-D. It seems to me, from Figure 2C, that expected stimuli evoke a larger response than unexpected ones, yet the authors interpret this as a prediction error effect, with a larger neural response to unexpected stimuli (over frontal electrodes). In their response, they state that there was "more negativity associated with the unexpected stimulus", but I'm not sure how this follows from Figure 2C-D.

2) Original point 2. According to the authors, the fact that the ERP waveforms is (almost) only positive, rather than a sequence of positive and negative components, is the result of the minimal preprocessing applied (only a 0.5 Hz high-pass filter). However, from my limited experience with EEG, I would expect positive and negative components even with minimal preprocessing. I'm still somewhat confused by this.

Points 1 and 2 are closely related, and so we have chosen to deal with them together in our response. We thank the reviewer for highlighting this ambiguity in our reporting of the original ERPs. We have conducted a new analysis that brings our results in line with the mismatch negativity literature, and which should therefore alleviate these concerns. To briefly reiterate our earlier explanation of the original result, the waveform shape is largely driven by the filtering used and the presented stimulus. Traditionally, ERPs have been band-pass filtered so that simple peak analyses can be applied. We believe this approach is outdated, and that modern analytic approaches are sufficient to reveal the actual, rather than processed, brain response (and that these approaches are also less susceptible to experimenter degrees of freedom). However, we acknowledge these subtle issues aren’t obvious to those outside the field, and we have therefore included bandpass filtered results and conducted a more traditional peak analysis, so the results adhere better to the familiar pattern. This analysis showed a larger positivity (P1 component) to the alternate than the repeating stimuli. Critically, there was a significantly greater negativity (N1 component) for unexpected stimuli than for expected stimuli, in line with the widely reported visual mismatch negativity (Garrido, Sahani and Dolan, 2013; Näätänen et al., 2007; Saarinen et al., 1992).

Note that this analysis merely provides confirmation that the expectation manipulation elicited a surprise response in brain. It does not address our central question of how visual features – in this case orientation – are affected by expectancy. It is this aspect of predictive coding theory that we go on to investigate using multivariate forward encoding analyses.

3) Original point 5. I still don't understand why there is such strong off-diagonal decoding in Figure 7, given that the orientation of the second grating was randomized, as the authors state in the legend. In the orientation of the second grating was randomized, how is it possible to decode it when training on the first grating?

We believe the strong off-diagonal encoding of the first grating during presentation of the second grating reflects ongoing processing of the initial stimulus. It is possible to see prolonged encoding of the first grating even after presentation of the second because the orientations of the two gratings within each pair were uncorrelated. One interpretation of this effect is that it reflects an ongoing comparison of the orientations of the first and second stimuli, but we acknowledge that at present we do not have a definitive explanation for the off-diagonal encoding observed. We hope our findings will provoke readers to generate and test their own hypotheses with respect to this interesting and reliable effect.

4) In response to my original point #10, the authors explain that the EEG data were baseline corrected to the interval -100 to 0 ms before presentation of the second Gabor in the pair. Isn't this problematic, in a paradigm in which grating 1 can lead to a prediction about grating 2? If there is an expectation signal present just before the appearance of grating 2 (as is quite likely), this would 'contaminate' the baseline, and effects found elsewhere (before -100 ms or after 0 ms) might actually reflect a negative image of expectation effects present in the pre-stimulus 2 interval.

We carefully balanced the trials so there was an equal number of different orientations of the initial grating leading to each subsequent (unexpected) grating within a pair. For repeating blocks, the expectation was that the orientation would repeat; thus, the unexpected stimulus was any orientation apart from the expected one. Critically, the difference between the conditions only arises after presentation of the second stimulus in the pair, once again ensuring the baseline period is matched across conditions. As we trained across all trial types, the expectations contained within the baseline period averaged to zero, removing any potential for contamination. We have included a discussion of these issues in subsection “EEG acquisition and pre-processing” of the revised manuscript.

Reviewer #2:

I thank the authors for their revision, which addresses some of the earlier concerns.

There are a few remaining issues, addressing them would improve the manuscript further.

About the localization of the signal:

Central to predictive coding theories is the idea that sensory predictions which are generated higher up in the hierarchy modulate responses in lower levels of sensory cortices. This statement is much more specific than the more general idea that: recent experience establishes expectations in the brain, which manifest somewhere and somehow in the encoding of sensory input. In the current state of the manuscript, this important nuance seems to be missing. Given that predictive coding theories have very specific hierarchical predictions, the argument that the authors wanted to "limit experimenter degrees of freedom potentially introduced through the post-hoc selection of subsets of electrodes" is simply not valid here. This is not a trivial issue, because as long as we do not constrain a theory, it cannot be falsified.

I understand that the authors might not want to or might not be able to localize their effects of expectation, but that limits the extent to which these findings are instructive about predictive coding theory and more generally about how the brain processes unexpected sensory events. Even if the general orientation selectivity is mainly determined by the signal at occipital and parietal electrodes, there is no guarantee that any interaction between expectation and orientation selectivity originates from the same signal in these electrodes. For example, I wonder whether a potential explanation for the discrepancy with Kok et al.,'s work could be that their effects were localized to visual cortex? Perhaps in this study their expectation effects do not originate from in visual cortex, but for example only in frontal areas?

Without such information, the findings of this study reduce to: (a) that repetition suppression and expectation effects are dissociable (which already has been demonstrated by for example Todorovic and de Lange, 2012) and (b) expectation increases selectivity for (Gabor) stimuli somewhere in the brain for this particular task. The fact that this would be hard to reconcile with other findings in early visual cortex is then explained away by the argument that "as this is a relatively complex phenomenon it is likely to yield different outcomes when different levels of prediction are manipulated". This reasoning does not echo the idea of predictive coding as universal principle of cortical responses. A good theory should be able to make general predictions that are valid in a (relatively) broad range of situations, without requiring ad hoc explanations for different experiments.

The authors should be more explicit about these problems.

We agree with the reviewer that a good theory should make testable predictions. The current work aimed to provide a strong test of one aspect of predictive coding theory, namely, how expectations affect the neural representation of visual feature information. We believe it is important to test this key aspect of the theory because most evidence for predictive coding does not address this issue. As the reviewer correctly recognises, aspects of our results may not fit neatly with the current version of the theory as formulated by Rao and Ballard, (1999) or Friston, (2005). Instead, they provide an important constraint on future versions of the theory.

We also agree that the hierarchal aspect of the theory can only be tested by simultaneously measuring neuronal activity at multiple stages of the processing hierarchy. The requisite temporal and spatial resolution for such a test is likely only possible using invasive recordings across cortical areas in animal models. Here we have introduced a paradigm and data analytic approach which could by readily implemented in animal experiments. We chose to use EEG in our study because it is the most widely-used index of the mismatch negativity effect, this allowing us to compare our results with those reported across this large and longstanding literature.

We have substantially revised the Introduction, Results section and Discussion section of the manuscript to indicate more clearly where the results support (or do not support) predictive coding theory. We have also identified limitations of the current work and provided a more detailed consideration of how invasive recordings in animal models could further test the theory.

Finally, we would like to repeat a point we made in response to the Editors’ comments (above). Our aim in framing our work in terms of predictive coding should not be construed as endorsement of the theory. We remain agnostic as to whether predictive coding theory provides an optimal conceptualisation of the effects of repetition and expectation on neural responses and believe the results we report in the paper add value to the field quite independently of this influential framework.

Specific points:

"One mechanism by which the brain reduces its information processing load is to encode successive presentations of the same stimulus in a more efficient form, a process known as neural adaptation": The authors have added references for this sentence, but I think they still can't justify the statement. Right now, the sentence reads like the function of adaptation is a settled case. Given our current lack of understanding of the mechanisms underlying adaptation, any suggestions about its function remain highly speculative. I suggest you change this sentence to "Neural adaptation is one mechanism by which the brain might reduce its information…".

We have changed this sentence as suggested.

In their rebuttal the authors confirm that they find Expected > Unexpected in occipital-parietal electrodes (while Expected < Unexpected in central frontal electrodes). This is a finding that contradicts the hallmark of predictive coding theory, namely that neural response strength signals prediction error. The authors should address this discrepancy. On the other hand, the authors still use the term "prediction errors" in reference to their results. For example, in the Discussion section they say: "We found that prediction errors were associated with increased gain of stimulus representations…". Yet, as they acknowledge, they can't know where the change in stimulus presentations took place, while they only find prediction errors in a frontal cluster.

This same issue was raised by reviewer 1 (comments 1 and 2), and we refer the reviewer to our response to this point above.

Comment 5:

I understand that care should be taken for these comparisons, but still it is very valid to compare with the first stimulus. Take the field of single-unit animal studies of repetition suppression, would they not always show data/analyses for the first stimulus also? I think it is an important statistical control for various effects (as I mentioned in my original email).

We have now conducted this analysis as requested, noting the caveats with respect to stimulus confounds outlined in our response to the Editors (see their Point ii). To do this, we had to modify our analytic approach because the orientations presented in the first and second gratings were often not the same. We separated each trial into first- and second-grating epochs, and trained and tested the forward model on the orientations within each epoch. For each condition (i.e., Repeat, Alternate, Expected and Unexpected) we compared the response for the first and second gratings. The Author response image 1 shows the early (79 -185 ms) orientation-selective response to the grating for each condition. As the figure shows, the only marginally reliable difference in orientation selectivity emerged for the unexpected condition, such that the second grating elicited a larger (p =. 056) response than the first grating. None of the other comparisons were significant (p >. 2). As noted in our response to the Editors (their point ii), we have chosen not to include these results in the revised manuscript as they involve an inherent stimulus confound. If the reviewers and editors feel it is imperative we report the new results, however, we shall do so.

Reviewer #3:

I think the revised document is much more clear and transparent.

After reading the responses and details of Figure 5, I have a slight concern that it might suffer from some circularity.

The figure is very clear and the results are consistent. However, the time period for which these results are displayed can affect the trend.

Ideally one would want to see the classification accuracy for different "conditions" as a function of time and see how and when the expected/unexpected x repeat/alternate differ in terms of orientation information. However, I think a figure like Figure 5 is also fine provided that all the information is given in the Materials and methods section and the figure caption.

In the Materials and methods section you can also state if you are averaging in time and then computing the classification accuracy or vice versa.

I think the details of which Matlab function they used is unnecessary, but this is key.

Our aim was to make the analysis of the data shown in Figure 5 as transparent and straightforward as possible. To clarify our approach, the values used in the analysis were not chosen arbitrarily in terms of timing, but rather were determined by the data themselves. The value selected was the maximum classification accuracy from a wide time window (from 0-600 ms after stimulus presentation) and was exactly the same between all conditions for each participant. These values were then compared between the conditions of interest. Our goal in taking this approach was to accommodate the possibility of inter-individual differences in peak classification across participants without biasing the result to any condition. We also used the maximum classification accuracy to remove the time dimension so all four conditions could be simultaneously visualised and compared. We have now included a detailed description of how we derived the measure in the Results section and Materials and methods sections.

[Editors’ note: the author responses to the re-review follow.]

Your letter of appeal has been considered by a Senior Editor and a Reviewing editor, and we are prepared to consider a revised submission. However, I should point out that the reviewers have requested a new analysis and the reviewers and editors are all in agreement that our continued consideration of this MS would be contingent on the outcome of this analysis.

I paste below the comments from one of our reviewers (reviewer 1; reviewer 2 was broadly happy with the MS you resubmitted). There are 3 points. We agree that points 1 and 3 can be dealt with by acknowledging limitations to the data/paradigm in the discussion or elsewhere in the main text. Point 2 is more thorny and the reviewer requests a further analysis, because the paper seems to hinge principally on this finding. You have argued that baselining in this period is unbiased because of balancing of trial types in your design. The reviewer is worried (and we agree) that this is not an adequate response, because on expected trials, the baselining will remove trial-specific information that relates to the expectation of the stimulus and potentially affect the levels of encoding of expected vs. unexpected stimuli. To be crystal clear, I precis from our discussion:

"The interval the authors use as the baseline for all their analyses (as far as i can tell), is the interval between gratings 1 and 2, i.e. the time at which they can form an expectation about the upcoming stimulus. So, if there is a neural expectation signal in this interval, using this as baseline affects all (decoding) analyses. For instance, in their Figure 4B, decoding of expected gratings is worse than unexpected – could this be because for the expected trials they've subtracted a pre-stimulus expectation signal that was present in the baseline?"

The reviewer requests that you replicate this analysis baselining from a different period (e.g. before stimulus onset when there is no risk of contamination with expectation signals). Contingent on this analysis working (which, in the event that your explanations were correct, it certainly should), and on satisfactory addressing of points 1 and 3 with discussion of limitations, I think we would be happy to go forward with the paper. I think if an analysis of the sort suggested does not work, the reviewers might be skeptical about whether the claims in the paper stand up.

We thank the editors for this clearly worded summary of the residual issue concerning our selection of an appropriate baseline epoch in the reported decoding analyses. We have now undertaken the exact analysis requested by reviewer 1 and replicated the pattern that emerged from our original analysis. We have provided a full explanation of this new analysis in our detailed response to reviewer 1, below.

Reviewer 1:

I find the univariate EEG much easier to interpret now, and I applaud the authors for making the effort to improve this. However, I still have concerns about the decoding analyses.

1) With regard to the strong off-diagonal decoding, it seems the authors themselves do not fully understand this either. They suggest it may be to do with the ongoing processing of grating 1 during the processing of grating 2, but would such ongoing late processing really be expected to lead to almost as strong off-diagonal as diagonal decoding, as Figure 7 suggests? I also find the very strong decoding signals in the pre-grating1 testing time disturbing; why would there be decoding prior to presentation of the first grating? (I come back to this below as well.)

Thank you for raising this issue. As indicated by the reviewer, interpretation of the off-diagonal decoding result is not straightforward and lends itself to several plausible interpretations that are difficult to disentangle from one another. Consequently, to improve the clarity of the paper, we have removed this analysis entirely. Importantly, none of the main conclusions of our work rely on the outcome of this analysis, and the central message of the paper remains unchanged.

2) In response to my concern about the baseline procedure, i.e. baselining on the interval between the two gratings, the authors state that this is not a concern, since the trials were balanced "so there was an equal number of different orientations of the initial grating leading to each subsequent (unexpected) grating within a pair." I don't understand how this addresses my concerns. They say, "the difference between the conditions only arises after presentation of the second stimulus in the pair". While that may be true when comparing an expected repetition to an unexpected alternation, and an expected alternation to an unexpected repetition (i.e., in those comparisons, participants had the same expectation), it is *not* true when comparing e.g. an expected repetition to an unexpected repetition. To be clear, I am not raising a concern about the signal from grating 1 itself carrying over into the grating 2 period, I am concerned about an expectation signal, induced by grating 1 and the context (repetition vs. alternation block), being present in the interval between gratings 1 and 2, i.e. the baseline period.

To be specific, what I am concerned about is that there may be a neural signal in the interval between the two gratings that reflects the expected orientation. This does not need to be carry-over from the bottom-up signal caused by the first grating, it may be a genuine top-down expectation induced template of the expected orientation. If such a pre-second-stimulus expectation template exists, then in the 'Expected repetition' trials the current baselining procedure would lead to confounds, since this template is in the baseline period, and therefore a negative reflection of this template would be introduced throughout the rest of the trial timeline, i.e. during grating 1 and grating 2. This process would affect the different conditions differently, since in 'Unexpected repetition' and 'Expected alternation' trials participants cannot form an expectation of a specific trials, hence no expectation template would be expected to be present, and in the 'Unexpected alternation' trials a template might be present, but it doesn't match the presented orientation (of the second grating). Regardless of this specific confound, in more general terms baselining on a period during which participants form expectations in a subset of the trials seems problematic. The only way I would see of addressing this issue, is to baseline the trials on the EEG data prior to the first grating, when no expectation can be formed yet. (Though from their response to some of the other points, I take the authors as saying that the relationship between the orientation of the second grating on trial n and the first grating on trial n+1 is not perfectly controlled, in which case this approach would not be valid. And indeed, in Figure 7, there seems to be strong decoding signal prior to the presentation of grating 1, which would either be caused by some relationship between grating 2 on the previous trial and grating 1 on the current trial, or by the baselining concern I raise above.)

We understand the nature of the reviewer’s concern here. To address this issue, we employed a new baseline period that extended from -100 to 0 ms before the appearance of the first Gabor in a pair, as suggested by the reviewer, and recomputed the forward encoding model. We averaged the responses over the same early time interval over which the original analysis showed increased orientation selectivity. This confirmed the original pattern of results, such that orientation selectivity was significantly greater for unexpected versus expected stimuli (see Figure 4—figure supplement 1). Again, there was no effect on the width of the representation with expectation. It should also be noted that our original procedure of taking a baseline before the second Gabor is in line with previous work that used variants of the same paradigm (Todorovic and de Lange, 2012). We have included the results of this new analysis from subsection “Expectations increase orientation-selective information contained within patterns of EEG activity” of the revised manuscript (below).

“It might be argued that the particular baseline period we chose for the encoding analyses – namely from -100 to 0 ms before the onset of the second Gabor in each pair – biased the results by incorporating a purely top-down expectation template triggered by the orientation of the first Gabor (Kok et al., 2017). To rule out this possibility, we performed a further forward encoding analysis where we baselined the raw EEG data to the mean activity from -100 to 0 ms before the first Gabor in each pair. Critically, this control analysis involved a baseline period over which it was not possible to form a top-down expectation of the orientation of the second Gabor based on the orientation of the first. This analysis yielded the same pattern of results as the original analysis (Figure 4—figure supplement 1), such that the unexpected stimulus evoked significantly greater orientation selectivity than the expected stimulus (p =.02). Also in line with the original analyses, the width of the representation was not affected by expectation (p =.44), and there was no effect of repetition suppression on orientation selectivity (p =.64). We can thus be confident that the effect of expectation on orientation selectivity that we report here, based on our forward encoding analyses, is not an artefact of the baselining procedure.”

3) This is a more general point about the experiment design, that I didn't raise before as I only realised it fully in this round. Expected alternation trials were as follows: a certain orientation was presented as the first grating, and then participants could form the expectation that it would alternate, meaning that it was equally likely to be any other orientation. So that's a very unspecific expectation. By contrast, in a previous study orthogonalising expectation and repetition (Todorovic and De Lange), 'expected alternations' meant that participants had a 75% valid expectation of exactly which stimulus would appear. That is, in that study, participants had a 75% chance of predicting the exact stimulus coming up in both expected repetition and expected alternation trials, making those two conditions nicely symmetrical. In the current study however, expected alternation trials ('any orientation can come up, just probably not the one I've just seen') are not at all similar to expected repetition trials, and it seems a bit of a stretch to average these two conditions together and call them 'expected' orientations. In other words, I'm not sure how successful the current design is in orthogonalising expectation and repetition.

Thank you for highlighting this subtlety in the design, which we have now clarified in the revised manuscript. To be clear, our design was based on the original Summerfield et al. (2008) work, with Gabor orientation replacing faces while maintaining the same relationships in the alternating trials. Furthermore, the current design is consistent with later work examining the interaction between repetition suppression and expectation (Kaliukhovich and Vogel, 2010; Kovács, et al., 2013).

To briefly explain our choice of design, in the Todorovic and de Lange (2012) study only two or three auditory frequencies were used. This made it easier to present alternating stimuli with a predictable identity. By contrast, in our visual paradigm we used nine different grating orientations to ensure we could measure orientation selectivity at a reasonable level of resolution. It simply was not feasible to have every orientation uniquely predict another orientation in the alternating blocks (e.g., having a 0 degree Gabor predicting a 90 degree Gabor in one block, a zero degree Gabor predicting a 40 degree Gabor in another block, and so on for all possible combinations of stimuli). This would have led to a factorial explosion of conditions. Worse still, such an approach, even if feasible, could have complicated interpretation of the results, as the amount of visual adaptation is itself highly orientation dependent (Blackmore and Campbell, 1969; Zavitz et al., 2016). The rule in the current study was that the orientation of the first Gabor was repeated or alternated for the second Gabor. This rule can be equally violated for repeating and alternating orientations, meaning the conditions are still matched for expectation and repetition. We have now added a consideration of this issue, as requested, from subsection “Experimental Design” of the revised manuscript (below).

“The relationship between the pairs of orientations for the different expectation conditions was based on the original study (Summerfield et al., 2008), and on other studies (Kaliukhovich & Vogels, 2010; Kovács et al., 2013) that examined the interaction between repetition suppression and expectation. In the repeating condition, the orientation of the second Gabor is expected to be the same as the orientation of the first, whereas in the alternating condition the orientation of the second Gabor is expected to be different from that of the first. This relationship between the expected orientations of the stimuli in the alternating condition is slightly different to another modification of the paradigm which employed a more limited range of stimuli (Todorovic et al., 2011; Todorovic and de Lange, 2012). Specifically, the paradigm introduced by Todorovic and colleagues used two or three auditory tones of different frequencies. In the alternating condition, the expectation is that one tone will follow another (i.e. 1000 Hz and then 1032 Hz), then this is violated when a 1000 Hz tone is repeated. In this paradigm, an exact frequency is expected in the alternating condition, a design feature that differs from the paradigm used in the current work where there is no specific expectation of the orientation of the second Gabor based on the orientation of the first in the alternating condition. Instead the expectation in the alternating condition is that the orientation will change, and this can be violated by repeating the orientation. In this sense, there is no specific expectation about the second orientation in the alternating condition. Instead, the rule is about alternating or repeating the first orientation. We did not implement the Todorovic et al. paradigm because the combinatorial explosion of stimulus conditions needed to measure orientation selectivity (such that every orientation is predicted by another orientation). Future work could investigate how this subtle change in paradigm design affects the encoding of stimulus information.”

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Citations

Tang M, Smout C, Arabzadeh E, Mattingley J. 2018. Data from: Prediction Error and Repetition Suppression Have Distinct Effects on Neural Representations of Visual Information. Dryad Digital Repository. [DOI] [PMC free article] [PubMed]

Supplementary Materials

Transparent reporting form

elife-33123-transrepform.pdf^{(273.2KB, pdf)}

DOI: 10.7554/eLife.33123.011

Data Availability Statement

The EEG data have been deposited on Dryad 10.5061/dryad.3d7kq

The following dataset was generated:

[bib1] Adibi M, McDonald JS, Clifford CW, Arabzadeh E. Adaptation improves neural coding efficiency despite increasing correlations in variability. Journal of Neuroscience. 2013;33:2108–2120. doi: 10.1523/JNEUROSCI.3449-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib2] Auksztulewicz R, Friston K. Repetition suppression and its contextual determinants in predictive coding. Cortex. 2016;80:125–140. doi: 10.1016/j.cortex.2015.11.024. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib3] Barron HC, Garvert MM, Behrens TEJ. Repetition suppression: a means to index neural representations using BOLD? Philosophical Transactions of the Royal Society B: Biological Sciences. 2016;371:e20150355. doi: 10.1098/rstb.2015.0355. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib4] Bekinschtein TA, Dehaene S, Rohaut B, Tadel F, Cohen L, Naccache L. Neural signature of the conscious processing of auditory regularities. PNAS. 2009;106:1672–1677. doi: 10.1073/pnas.0809667106. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib5] Benucci A, Saleem AB, Carandini M. Adaptation maintains population homeostasis in primary visual cortex. Nature Neuroscience. 2013;16:724–729. doi: 10.1038/nn.3382. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib6] Bigdely-Shamlo N, Mullen T, Kothe C, Su K-M, Robbins KA. The PREP pipeline: standardized preprocessing for large-scale EEG analysis. Frontiers in Neuroinformatics. 2015;9:B153. doi: 10.3389/fninf.2015.00016. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib7] Brainard DH. The Psychophysics Toolbox. Spatial Vision. 1997;10:433–436. doi: 10.1163/156856897X00357. [DOI] [PubMed] [Google Scholar]

[bib8] Brouwer GJ, Heeger DJ. Decoding and reconstructing color from responses in human visual cortex. Journal of Neuroscience. 2009;29:13992–14003. doi: 10.1523/JNEUROSCI.3577-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib9] Buckner RL, Goodman J, Burock M, Rotte M, Koutstaal W, Schacter D, Rosen B, Dale AM. Functional-anatomic correlates of object priming in humans revealed by rapid presentation event-related fMRI. Neuron. 1998;20:285–296. doi: 10.1016/S0896-6273(00)80456-0. [DOI] [PubMed] [Google Scholar]

[bib10] Caharel S, d'Arripe O, Ramon M, Jacques C, Rossion B. Early adaptation to repeated unfamiliar faces across viewpoint changes in the right hemisphere: evidence from the N170 ERP component. Neuropsychologia. 2009;47:639–643. doi: 10.1016/j.neuropsychologia.2008.11.016. [DOI] [PubMed] [Google Scholar]

[bib11] Chaumon M, Bishop DV, Busch NA. A practical guide to the selection of independent components of the electroencephalogram for artifact correction. Journal of Neuroscience Methods. 2015;250:47–63. doi: 10.1016/j.jneumeth.2015.02.025. [DOI] [PubMed] [Google Scholar]

[bib12] Cichy RM, Pantazis D, Oliva A. Resolving human object recognition in space and time. Nature Neuroscience. 2014;17:455–462. doi: 10.1038/nn.3635. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib13] Cichy RM, Ramirez FM, Pantazis D. Can visual information encoded in cortical columns be decoded from magnetoencephalography data in humans? NeuroImage. 2015;121:193–204. doi: 10.1016/j.neuroimage.2015.07.011. [DOI] [PubMed] [Google Scholar]

[bib14] Clifford CW. Perceptual adaptation: motion parallels orientation. Trends in Cognitive Sciences. 2002;6:136–143. doi: 10.1016/S1364-6613(00)01856-8. [DOI] [PubMed] [Google Scholar]

[bib15] Clifford CW. The tilt illusion: phenomenology and functional implications. Vision Research. 2014;104:3–11. doi: 10.1016/j.visres.2014.06.009. [DOI] [PubMed] [Google Scholar]

[bib16] Cui Y, Wang YV, Park SJ, Demb JB, Butts DA. Divisive suppression explains high-precision firing and contrast adaptation in retinal ganglion cells. eLife. 2016;5:e19460. doi: 10.7554/eLife.19460. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib17] Dehaene S, Naccache L, Cohen L, Bihan DL, Mangin JF, Poline JB, Rivière D. Cerebral mechanisms of word masking and unconscious repetition priming. Nature Neuroscience. 2001;4:752–758. doi: 10.1038/89551. [DOI] [PubMed] [Google Scholar]

[bib18] Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods. 2004;134:9–21. doi: 10.1016/j.jneumeth.2003.10.009. [DOI] [PubMed] [Google Scholar]

[bib19] Dickinson JE, Almeida RA, Bell J, Badcock DR. Global shape aftereffects have a local substrate: A tilt aftereffect field. Journal of Vision. 2010;10:e5. doi: 10.1167/10.13.5. [DOI] [PubMed] [Google Scholar]

[bib20] Dickinson JE, Morgan SK, Tang MF, Badcock DR. Separate banks of information channels encode size and aspect ratio. Journal of Vision. 2017;17:27. doi: 10.1167/17.3.27. [DOI] [PubMed] [Google Scholar]

[bib21] Diederen KM, Spencer T, Vestergaard MD, Fletcher PC, Schultz W. Adaptive prediction error coding in the human midbrain and striatum facilitates behavioral adaptation and learning efficiency. Neuron. 2016;90:1127–1138. doi: 10.1016/j.neuron.2016.04.019. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib22] Ester EF, Sutterer DW, Serences JT, Awh E. Feature-Selective Attentional Modulations in Human Frontoparietal Cortex. Journal of Neuroscience. 2016;36:8188–8199. doi: 10.1523/JNEUROSCI.3935-15.2016. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib23] Fairhall AL, Lewen GD, Bialek W, de Ruyter Van Steveninck RR. Efficiency and ambiguity in an adaptive neural code. Nature. 2001;412:787–792. doi: 10.1038/35090500. [DOI] [PubMed] [Google Scholar]

[bib24] Felsen G, Shen YS, Yao H, Spor G, Li C, Dan Y. Dynamic modification of cortical orientation tuning mediated by recurrent connections. Neuron. 2002;36:945–954. doi: 10.1016/S0896-6273(02)01011-5. [DOI] [PubMed] [Google Scholar]

[bib25] Fiser A, Mahringer D, Oyibo HK, Petersen AV, Leinweber M, Keller GB. Experience-dependent spatial expectations in mouse visual cortex. Nature Neuroscience. 2016;19:1658–1664. doi: 10.1038/nn.4385. [DOI] [PubMed] [Google Scholar]

[bib26] Friston K. A theory of cortical responses. Philosophical Transactions of the Royal Society of London B Biological Sciences. 2005;360:815–836. doi: 10.1098/rstb.2005.1622. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib27] Garcia JO, Srinivasan R, Serences JT. Near-real-time feature-selective modulations in human cortex. Current Biology. 2013;23:515–522. doi: 10.1016/j.cub.2013.02.013. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib28] Garrido MI, Kilner JM, Stephan KE, Friston KJ. The mismatch negativity: a review of underlying mechanisms. Clinical Neurophysiology. 2009;120:453–463. doi: 10.1016/j.clinph.2008.11.029. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib29] Garrido MI, Sahani M, Dolan RJ. Outlier responses reflect sensitivity to statistical structure in the human brain. PLoS Computational Biology. 2013;9:e1002999. doi: 10.1371/journal.pcbi.1002999. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib30] Gotts SJ, Chow CC, Martin A. Repetition priming and repetition suppression: a case for enhanced efficiency through neural synchronization. Cognitive Neuroscience. 2012;3:227–237. doi: 10.1080/17588928.2012.670617. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib31] Grootswagers T, Wardle SG, Carlson TA. Decoding dynamic brain patterns from evoked responses: a tutorial on multivariate pattern analysis applied to time series neuroimaging data. Journal of Cognitive Neuroscience. 2017;29:677–697. doi: 10.1162/jocn_a_01068. [DOI] [PubMed] [Google Scholar]

[bib32] Gross CG, Schiller PH, Wells C, Gerstein GL. Single-unit activity in temporal association cortex of the monkey. Journal of Neurophysiology. 1967;30:833–843. doi: 10.1152/jn.1967.30.4.833. [DOI] [PubMed] [Google Scholar]

[bib33] Grotheer M, Kovács G. Repetition probability effects depend on prior experiences. Journal of Neuroscience. 2014;34:6640–6646. doi: 10.1523/JNEUROSCI.5326-13.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib34] Henson RN, Rugg MD. Neural response suppression, haemodynamic repetition effects, and behavioural priming. Neuropsychologia. 2003;41:263–270. doi: 10.1016/S0028-3932(02)00159-8. [DOI] [PubMed] [Google Scholar]

[bib35] Kaliukhovich DA, Vogels R. Stimulus repetition probability does not affect repetition suppression in macaque inferior temporal cortex. Cerebral Cortex. 2011;21:1547–1558. doi: 10.1093/cercor/bhq207. [DOI] [PubMed] [Google Scholar]

[bib36] Kamitani Y, Tong F. Decoding the visual and subjective contents of the human brain. Nature Neuroscience. 2005;8:679–685. doi: 10.1038/nn1444. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib37] Kaposvari P, Kumar S, Vogels R. Statistical learning signals in macaque inferior temporal cortex. Cerebral Cortex. 2018;28:250–266. doi: 10.1093/cercor/bhw374. [DOI] [PubMed] [Google Scholar]

[bib38] Kay KN, Naselaris T, Prenger RJ, Gallant JL. Identifying natural images from human brain activity. Nature. 2008;452:352–355. doi: 10.1038/nature06713. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib39] Keil A, Debener S, Gratton G, Junghöfer M, Kappenman ES, Luck SJ, Luu P, Miller GA, Yee CM. Committee report: publication guidelines and recommendations for studies using electroencephalography and magnetoencephalography. Psychophysiology. 2014;51:1–21. doi: 10.1111/psyp.12147. [DOI] [PubMed] [Google Scholar]

[bib40] Keller AJ, Houlton R, Kampa BM, Lesica NA, Mrsic-Flogel TD, Keller GB, Helmchen F. Stimulus relevance modulates contrast adaptation in visual cortex. eLife. 2017;6:e21589. doi: 10.7554/eLife.21589. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib41] King JR, Dehaene S. Characterizing the dynamics of mental representations: the temporal generalization method. Trends in Cognitive Sciences. 2014;18:203–210. doi: 10.1016/j.tics.2014.01.002. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib42] King JR, Gramfort A, Schurger A, Naccache L, Dehaene S. Two distinct dynamic modes subtend the detection of unexpected sounds. PLoS ONE. 2014;9:e85791. doi: 10.1371/journal.pone.0085791. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib43] King JR, Pescetelli N, Dehaene S. Brain mechanisms underlying the brief maintenance of seen and unseen sensory information. Neuron. 2016;92:1122–1134. doi: 10.1016/j.neuron.2016.10.051. [DOI] [PubMed] [Google Scholar]

[bib44] Kohn A, Movshon JA. Adaptation changes the direction tuning of macaque MT neurons. Nature Neuroscience. 2004;7:764–772. doi: 10.1038/nn1267. [DOI] [PubMed] [Google Scholar]

[bib45] Kok P, Jehee JF, de Lange FP. Less is more: expectation sharpens representations in the primary visual cortex. Neuron. 2012;75:265–270. doi: 10.1016/j.neuron.2012.04.034. [DOI] [PubMed] [Google Scholar]

[bib46] Kok P, Mostert P, de Lange FP. Prior expectations induce prestimulus sensory templates. PNAS. 2017;114:10473–10478. doi: 10.1073/pnas.1705652114. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib47] Kourtzi Z, Kanwisher N. Representation of perceived object shape by the human lateral occipital complex. Science. 2001;293:1506–1509. doi: 10.1126/science.1061133. [DOI] [PubMed] [Google Scholar]

[bib48] Kovács G, Kaiser D, Kaliukhovich DA, Vidnyánszky Z, Vogels R. Repetition probability does not affect fMRI repetition suppression for objects. Journal of Neuroscience. 2013;33:9805–9812. doi: 10.1523/JNEUROSCI.3423-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib49] Kumar S, Kaposvari P, Vogels R. Encoding of predictable and unpredictable stimuli by inferior temporal cortical neurons. Journal of Cognitive Neuroscience. 2017;29:1445–1454. doi: 10.1162/jocn_a_01135. [DOI] [PubMed] [Google Scholar]

[bib50] Kvale MN, Schreiner CE. Short-term adaptation of auditory receptive fields to dynamic stimuli. Journal of Neurophysiology. 2004;91:604–612. doi: 10.1152/jn.00484.2003. [DOI] [PubMed] [Google Scholar]

[bib51] Larsson J, Smith AT. fMRI repetition suppression: neuronal adaptation or stimulus expectation? Cerebral Cortex. 2012;22:567–576. doi: 10.1093/cercor/bhr119. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib52] Liu T, Larsson J, Carrasco M. Feature-based attention modulates orientation-selective responses in human visual cortex. Neuron. 2007;55:313–323. doi: 10.1016/j.neuron.2007.06.030. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib53] Maravall M, Petersen RS, Fairhall AL, Arabzadeh E, Diamond ME. Shifts in coding properties and maintenance of information transmission during adaptation in barrel cortex. PLoS Biology. 2007;5:e19. doi: 10.1371/journal.pbio.0050019. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib54] Marti S, King JR, Dehaene S. Time-resolved decoding of two processing chains during dual-task interference. Neuron. 2015;88:1297–1307. doi: 10.1016/j.neuron.2015.10.040. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib55] Maunsell JH, Treue S. Feature-based attention in visual cortex. Trends in Neurosciences. 2006;29:317–322. doi: 10.1016/j.tins.2006.04.001. [DOI] [PubMed] [Google Scholar]

[bib56] Maunsell JHR. Neuronal mechanisms of visual attention. Annual Review of Vision Science. 2015;1:373–391. doi: 10.1146/annurev-vision-082114-035431. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib57] Movshon JA, Lennie P. Pattern-selective adaptation in visual cortical neurones. Nature. 1979;278:850–852. doi: 10.1038/278850a0. [DOI] [PubMed] [Google Scholar]

[bib58] Myers NE, Rohenkohl G, Wyart V, Woolrich MW, Nobre AC, Stokes MG. Testing sensory evidence against mnemonic templates. eLife. 2015;4:e09000. doi: 10.7554/eLife.09000. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib59] Näätänen R, Paavilainen P, Rinne T, Alho K. The mismatch negativity (MMN) in basic research of central auditory processing: a review. Clinical Neurophysiology. 2007;118:2544–2590. doi: 10.1016/j.clinph.2007.04.026. [DOI] [PubMed] [Google Scholar]

[bib60] Oostenveld R, Praamstra P. The five percent electrode system for high-resolution EEG and ERP measurements. Clinical Neurophysiology. 2001;112:713–719. doi: 10.1016/S1388-2457(00)00527-7. [DOI] [PubMed] [Google Scholar]

[bib61] Paavilainen P. The mismatch-negativity (MMN) component of the auditory event-related potential to violations of abstract regularities: a review. International Journal of Psychophysiology. 2013;88:109–123. doi: 10.1016/j.ijpsycho.2013.03.015. [DOI] [PubMed] [Google Scholar]

[bib62] Patterson CA, Wissig SC, Kohn A. Distinct effects of brief and prolonged adaptation on orientation tuning in primary visual cortex. Journal of Neuroscience. 2013;33:532–543. doi: 10.1523/JNEUROSCI.3345-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib63] Pelli DG. The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision. 1997;10:437–442. doi: 10.1163/156856897X00366. [DOI] [PubMed] [Google Scholar]

[bib64] Rao RP, Ballard DH. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience. 1999;2:79–87. doi: 10.1038/4580. [DOI] [PubMed] [Google Scholar]

[bib65] Rasmussen RG, Schwartz A, Chase SM. Dynamic range adaptation in primary motor cortical populations. eLife. 2017;6:e21409. doi: 10.7554/eLife.21409. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib66] Rentzeperis I, Nikolaev AR, Kiper DC, van Leeuwen C. Relationship between neural response and adaptation selectivity to form and color: an ERP study. Frontiers in Human Neuroscience. 2012;6 doi: 10.3389/fnhum.2012.00089. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib67] Saarinen J, Paavilainen P, Schöger E, Tervaniemi M, Näätänen R. Representation of abstract attributes of auditory stimuli in the human brain. NeuroReport. 1992;3:1149–1151. doi: 10.1097/00001756-199212000-00030. [DOI] [PubMed] [Google Scholar]

[bib68] Salti M, Monto S, Charles L, King J-R, Parkkonen L, Dehaene S. Distinct cortical codes and temporal dynamics for conscious and unconscious percepts. eLife. 2015;4:e05652. doi: 10.7554/eLife.05652. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib69] Schacter DL, Buckner RL. Priming and the brain. Neuron. 1998;20:185–195. doi: 10.1016/S0896-6273(00)80448-1. [DOI] [PubMed] [Google Scholar]

[bib70] Simmons JP, Nelson LD, Simonsohn U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science. 2011;22:1359–1366. doi: 10.1177/0956797611417632. [DOI] [PubMed] [Google Scholar]

[bib71] Smirnakis SM, Berry MJ, Warland DK, Bialek W, Meister M. Adaptation of retinal processing to image contrast and spatial scale. Nature. 1997;386:69–73. doi: 10.1038/386069a0. [DOI] [PubMed] [Google Scholar]

[bib72] Spaak E, Watanabe K, Funahashi S, Stokes MG. Stable and dynamic coding for working memory in primate prefrontal cortex. The Journal of Neuroscience. 2017;37:6503–6516. doi: 10.1523/JNEUROSCI.3364-16.2017. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib73] Sprague TC, Serences JT. Attention modulates spatial priority maps in the human occipital, parietal and frontal cortices. Nature Neuroscience. 2013;16:1879–1887. doi: 10.1038/nn.3574. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib74] Stokes MG, Kusunoki M, Sigala N, Nili H, Gaffan D, Duncan J. Dynamic coding for cognitive control in prefrontal cortex. Neuron. 2013;78:364–375. doi: 10.1016/j.neuron.2013.01.039. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib75] Stokes MG, Wolff MJ, Spaak E. Decoding rich spatial information with high temporal resolution. Trends in Cognitive Sciences. 2015;19:636–638. doi: 10.1016/j.tics.2015.08.016. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib76] Summerfield C, Trittschuh EH, Monti JM, Mesulam MM, Egner T. Neural repetition suppression reflects fulfilled perceptual expectations. Nature Neuroscience. 2008;11:1004–1006. doi: 10.1038/nn.2163. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib77] Summerfield C, Wyart V, Johnen VM, de Gardelle V. Human scalp electroencephalography reveals that repetition suppression varies with expectation. Frontiers in Human Neuroscience. 2011;5:e67. doi: 10.3389/fnhum.2011.00067. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib78] Tang MF, Dickinson JE, Visser TA, Badcock DR. The broad orientation dependence of the motion streak aftereffect reveals interactions between form and motion neurons. Journal of Vision. 2015;15:4. doi: 10.1167/15.13.4. [DOI] [PubMed] [Google Scholar]

[bib79] Todorovic A, van Ede F, Maris E, de Lange FP. Prior expectation mediates neural adaptation to repeated sounds in the auditory cortex: an MEG study. Journal of Neuroscience. 2011;31:9118–9123. doi: 10.1523/JNEUROSCI.1425-11.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib80] Todorovic A, de Lange FP. Repetition suppression and expectation suppression are dissociable in time in early auditory evoked fields. Journal of Neuroscience. 2012;32:13389–13395. doi: 10.1523/JNEUROSCI.2227-12.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib81] Tootell RB, Reppas JB, Dale AM, Look RB, Sereno MI, Malach R, Brady TJ, Rosen BR. Visual motion aftereffect in human cortical area MT revealed by functional magnetic resonance imaging. Nature. 1995;375:139–141. doi: 10.1038/375139a0. [DOI] [PubMed] [Google Scholar]

[bib82] Tootell RB, Hadjikhani NK, Vanduffel W, Liu AK, Mendola JD, Sereno MI, Dale AM. Functional analysis of primary visual cortex (V1) in humans. PNAS. 1998;95:811–817. doi: 10.1073/pnas.95.3.811. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib83] Treue S, Martínez Trujillo JC. Feature-based attention influences motion processing gain in macaque visual cortex. Nature. 1999;399:575–579. doi: 10.1038/21176. [DOI] [PubMed] [Google Scholar]

[bib84] Ulanovsky N, Las L, Nelken I. Processing of low-probability sounds by cortical neurons. Nature Neuroscience. 2003;6:391–398. doi: 10.1038/nn1032. [DOI] [PubMed] [Google Scholar]

[bib85] Vinken K, Vogels R, Op de Beeck H. Recent visual experience shapes visual processing in rats through stimulus-specific adaptation and response enhancement. Current Biology. 2017;27:914–919. doi: 10.1016/j.cub.2017.02.024. [DOI] [PubMed] [Google Scholar]

[bib86] Wacongne C, Labyt E, van Wassenhove V, Bekinschtein T, Naccache L, Dehaene S. Evidence for a hierarchy of predictions and prediction errors in human cortex. PNAS. 2011;108:20754–20759. doi: 10.1073/pnas.1117807108. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib87] Wolff MJ, Jochim J, Akyürek EG, Stokes MG. Dynamic hidden states underlying working-memory-guided behavior. Nature Neuroscience. 2017;20:864–871. doi: 10.1038/nn.4546. [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Prediction error and repetition suppression have distinct effects on neural representations of visual information

Matthew F Tang

Cooper A Smout

Ehsan Arabzadeh

Jason B Mattingley

Roles

Abstract

Introduction

Results

Figure 1. Example stimulus displays and task design.

Video 1. Example of a stimulus sequence of Gabors in a typical alternating block.

Repetition suppression and prediction error affect the overall level of neural activity

Figure 2. Univariate EEG results for the effect of repetition suppression and expectation on the second stimulus in a pair.

Expectations increase orientation-selective information contained within patterns of EEG activity

Figure 3. Results of the forward encoding modelling for orientation-selectivity.

Figure 4. The effect of repetition suppression and expectation on orientation selectivity measured using forward encoding modelling.

Figure 4—figure supplement 1. The effect of a different baseline period (−100 to 0 ms before onset of the first Gabor) on orientation selectivity for the two main conditions.

Figure 5. Peak (naive Bayes) classification accuracy of the presented grating orientation for expected and unexpected conditions.

Expectation affects the temporal stability of stimulus representations

Figure 6. Cross-temporal generalisation of the forward encoding model based on grating orientations for the main effects of repetition suppression (upper panels) and expectation (lower panels).

Discussion

Materials and methods

Participants

Experimental setup

Visual task

EEG acquisition and pre-processing

Experimental design

Forward encoding modelling

Figure 7. A schematic of the forward-encoding approach applied to EEG activity.

Multivariate pattern analysis

Statistical testing

Acknowledgements

Funding Statement

Contributor Information

Funding Information

Additional information

Competing interests

Author contributions

Ethics

Additional files

Data availability

Reference

Decision letter

Roles

Author response

Author response image 1.

Associated Data

Data Citations

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases