Exploring the detection of associatively novel events using fMRI

Abstract

Identifying and evaluating events which are novel in a particular environment is crucially important for adaptive behavior. These events are often not just novel, as they typically violate expectations which may be formulated based on numerous features of our surroundings, one of which includes the ordinal structure (temporal order) of relevant stimuli. Events which violate such expectations, namely sequential deviants, constitute one category of associatively novel stimuli. The present event‐related fMRI study investigated the detection of sequential deviants presented within three types of equivalently organized, attended visual sequences which differed in stimulus dimensions relevant for defining the sequential structure (position, rhythm, and object identity). Presenting deviants within perceptual sequences defined by position and rhythm stimulus properties triggered comparable patterns of activations within the lateral parietal, premotor, and prefrontal regions. However, the activations identified in the context of position sequences showed a more dorsal distribution when compared to those in rhythm sequences. In contrast, detection of deviants within object sequences was supported by right‐lateralized parietal and temporal cortices. Thus, although the obtained results indicate similarities and partial overlap in activations triggered by specific pairs of deviants, differences in their processing were also revealed. This suggests that the general task context and specific stimulus features which define the deviant itself influence which brain regions within a widespread network incorporating lateral prefrontal, anterior premotor, and posterior (mainly lateral parietal) areas will become engaged in its processing. Hum Brain Mapp, 2011. © 2010 Wiley‐Liss, Inc.

INTRODUCTION

Although crucially relevant for adaptive behavior, detection of associative novelty characterized by the presentation of familiar items in novel spatial or temporal configurations [Kumaran and Maguire, 2007a] has not been consistently investigated. This can be attributed to a wide array of situations in which this phenomenon is encountered and to the nonuniform terminology used to describe it. For instance, it can be equally well described as “relational” or “associative,” while additional terms such as “sequential” may be used for describing one (temporal) type of such novelty. Furthermore, the terms “novelty” and “deviance” can be interchanged, as such events are not simply novel, but also violate expectations which can be formulated based on the learned associations between stimuli. Accepting this variable terminology, a comprehensive understanding of regular and violated associative processing can be found in the field of motor sequence processing [Huettel et al., 2002; Keele et al., 2003; Rüsseler and Rösler, 2000] which has recently been complemented by comparable findings from the perceptual domain [Schubotz and von Cramon, 2001, 2002c]. Studies investigating the detection of violations within both motor and perceptual sequences have shown a rather special status of processing sequential when compared to nonsequential deviants which is supported by the pronounced involvement of lateral prefrontal cortices [Bubic et al., 2009; Huettel et al., 2002].

Although it seems plausible to suggest that the level of prefrontal engagement in this context depends on specific task requirements, an alternative emphasizing an imperative role of prefrontal cortex in detecting sequential or even all associative deviants could also be proposed. Similarly, Kumaran and Maguire [2007a] have recently argued for the existence of a generative mechanism underlying the detection of all forms of associative novelty. Specifically, they suggested that this detection relies on a match‐mismatch comparison process supported by the hippocampus which is also in line with findings showing the involvement of this region and adjacent cortices in both regular and violated associative, sequence and general contextual processing [Bar et al., 2008; Kumaran and Maguire, 2009; Lisman and Redish, 2009; Schendan et al., 2003]. However, absence of such engagement in some forms of regular sequencing within the motor [cf., Keele et al., 2003] and other, e.g., perceptual, linguistic, or cognitive [Dominey, 2005; Schubotz, 2007] domains, as well as in processing sequential deviants [Bubic et al., 2009; Huettel et al., 2002] has also been previously reported. Although plausible given the differences in tasks and paradigms employed across different studies, such divergence in understanding regular and novel associative processing is still somewhat surprising. Aimed at addressing potential similarities and differences in processing different types within the same class of associatively novel events, the present event‐related fMRI study investigated the detection of sequential deviants introduced into perceptual sequences previously suggested to rely on internal models as implemented within the motor system [Schubotz, 2007]. By exploring the detection of sequential deviants introduced into three types of equivalently organized perceptual sequences, we investigated whether and how detecting such deviants depends on the stimulus features (position, rhythm, and object identity) relevant for defining the sequential structure.

MATERIALS AND METHODS

Participants

Thirty right‐handed, healthy male volunteers (mean age 26.7) participated in the study. All participants reported having normal or corrected‐to‐normal vision. Four participants were excluded from further analysis due to below‐chance level performance in the sequencing task and one due to movement during the experiment. All subsequent analysis was performed on the data from 25 participants. All participants gave informed consent for participating after being informed about potential risks and screened by the physician of the institution. The experimental standards were approved by the local ethics committee of the University of Leipzig. Collected data were handled anonymously.

Procedure

Participants were instructed and underwent a behavioral training session several days before the fMRI measurement in which they were trained to perform the three tasks until they reached a learning criterion of 75%. Prior to the main experiment on the day of the measurement, they were additionally presented with the instructions and a 5‐min behavioral training session which included all tasks. During the main experiment, participants were supine on the scanner bed with their index and middle fingers of the right hand positioned on the response buttons. To prevent postural adjustments, the participants' arms and hands were carefully stabilized by tape. In addition, arm, hand, and head motion was prevented by using form‐fitting cushions. To attenuate scanner noise, participants were provided with earplugs and headphones.

Stimuli and Task

The stimulus material used in this study included 12 different objects, each composed of a 25‐mm circle (0.14° of visual angle) and a slightly smaller geometrical form, either a square or a circle, placed in its centre (see Fig. 1). The colors of both geometrical forms could be red, yellow, or blue and they always differed between the two forms. Each stimulus display consisted of two identical objects presented on opposite locations of a virtual circle with a radius of 6 cm. A fixation cross was presented at the screen centre to facilitate constant visual fixation. Each stimulus was presented for either 300, 600, 900, 1,200, 1,500, or 1,800 ms. Responses were made by pressing the left or right key of a standard response button box with the index and middle finger of the right hand.

Figure 1.

Schematic examples of four tasks. All versions of the SPT trials started with a three‐stimulus sequential pattern followed by its three full repetitions or two repetitions and one violation (here only one full repetition and one violation marked by the reversal in the order of the 2nd and 3rd stimulus are shown). Participants' task was to indicate whether a sequential violation occurred within the trial or not. In the control, target detection task, participants monitored for the presence of occasional stimuli deviating from the remaining stimulus set (here the two objects are not presented at the exactly opposite locations on a virtual circle). The response was given at the end of each trial. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Three different versions of the sequencing (serial prediction task; SPT) and a control (target detection task; control) task of equal trial organization were presented in a mixed trial design. Each trial included the successive presentation of 12 stimuli without temporal gaps, preceded by a task cue with the duration of 400 ms and followed by a 1,500‐ms response period and performance feedback lasting for 400 ms. During all other periods in the experiment a fixation cross was presented at the center of the screen. Overall trial duration was 14 s and, to improve temporal resolution, each trial occurred at four different offset points (0, 500, 1,000, and 1,500 ms) in relation to fMRI data acquisition [Josephs et al., 1997]. During the course of the experiment the stimulus trials were interspersed with empty trials during which only a fixation cross was presented and no task was to be performed by the participants. Stimuli were presented using Presentation 11.7 (Neurobehavioral systems, San Francisco, CA).

In all three versions of SPT, the participants attended to the order of presented stimuli in an attempt to extract and subsequently predict a specific repetitive pattern contained within them. Each trial started with three stimuli defining a sequential pattern which was then fully repeated two times. The last part of the trial entailed either one additional full repetition or a violation of the original three‐stimulus pattern which was characterized by the reversal in the order of 1st and 2nd or 2nd and 3rd element of the original sequence. The task of the participant was to indicate, in a forced‐choice manner, whether the end of the trial entailed a violation or an ordered repetition of the original pattern. Sequential violations were presented in 50% of all trials and the participants were provided with feedback concerning the correctness of their response. Three versions of SPT differed in the stimulus dimension relevant for defining the sequential structure which corresponded to the stimulus dimensions (object identity, position, or rhythm) along which the stimuli were manipulated. In each version of SPT one dimension was task‐relevant and varied in an orderly manner in contrast to the two irrelevant, randomly varying dimensions. In the object serial prediction task (SPT‐O) participants attended to the stimulus identity which was defined by the color and form of the two objects contained in each stimulus. In the position serial prediction task (SPT‐P) participants attended to the position of the elementary forms on the virtual circle, while in the rhythm serial prediction task (SPT‐R) they attended to their temporal duration which formed a distinct rhythmic pattern.

Besides the three versions of SPT, the participants were presented with a control target detection task (control) which was organized in an equivalent fashion as SPT, but did not contain a repeating sequential pattern across any of the three stimulus dimensions. The participants were instructed to attend to these trials in order to detect the presence of occasional individual target stimuli which deviated from the remaining stimulus set in one of three possible ways: the two objects that constituted a stimulus were either of unequal duration, were not identical or were not presented at the exactly opposite locations on a virtual circle. The participants' task was to count such stimuli and respond whether an odd or even number of them was presented during the trial.

Across all trials in the experiment the order of stimuli was pseudo‐randomized. The probability of each stimulus and that of transitions between stimuli were balanced across different positions within the trial. To avoid any motor contributions to the tasks, participants' response was always required after the end of each sequence. The experiment included eight types of trials: ordered and violated object SPT trials, ordered and violated position SPT trials, ordered and violated rhythm SPT trials, control trials with a deviant and control trials without a deviant (see Fig. 1). Twenty one trials of each type were used which, together with the 15 empty trials, amounted to the total of 183 trials presented in the course of the experiment.

Data Acquisition

The experiment was carried out on a 3T scanner (Medspec S300, Bruker, Ettlingen) equipped with a standard bird cage coil. Immediately prior to the functional experiment, a set of two‐dimensional anatomical images was acquired for each participant using a MDEFT sequence (256 × 256 pixel matrix) [Norris, 2000; Ugurbil et al., 1993]. Additionally, to improve the localization of activation foci, high resolution whole‐brain images using a T1‐weighted three‐dimensional segmented MDEFT sequence were acquired for each participant in a separate session. This volume dataset with 160 slices and 1‐mm slice thickness was standardized to the Talairach stereotactic space [Talairach and Tournoux, 1988]. Functional images in‐plane with the anatomical images were acquired using a gradient‐echo echo planar imaging (EPI) sequence with an echo time (TE) of 30 ms, a flip angle of 90°, and a repetition time (TR) of 2,000 ms. Twenty two functional slices were acquired parallel to the bicommissural plane (AC‐PC) (thickness 4 mm, interslice gap 1 mm) covering the whole brain. The matrix acquired was 64 × 64 with a field of view of 192 mm, resulting in an in‐plane resolution of 3 mm × 3 mm. A total of 1,290 volumes were acquired.

Data Analysis

MR data processing was performed using the software package LIPSIA [Lohmann et al., 2001] which contains tools for preprocessing, coregistration, statistical evaluation, and visualization of fMRI data. To correct for the temporal offset between the slices acquired in one scan, a cubic‐spline‐interpolation was applied. A temporal high pass filter with a cut‐off frequency of 1/130 Hz was used for baseline correction, removing low‐frequency drifts in an fMRI time series (frequencies due to global signal changes). Spatial Gaussian smoothing was applied using a Gaussian filter with 5.65‐mm full width at half maximum (FWHM). To align the functional data slices with a 3D stereotactic coordinate system, a rigid linear registration with six degrees of freedom (three translational and three rotational parameters) was performed. The parameters were acquired on the basis on MDEFT and EPI‐T1 slices to achieve an optimal match between these slices and the individual 3D reference dataset. Each transformation matrix was subsequently transformed to a standard Talairach brain size [x = 135, y = 175, z = 120 mm; Talairach and Tournoux, 1988] by applying linear scaling. Finally, the normalized transformation matrices were applied to the acquired functional slices to align them with the stereotactic coordinate system. Transformation was performed using trilinear interpolation, thus generating data with a spatial resolution of 3 mm³.

The statistical evaluation was based on a least‐squares estimation using the general linear model for serially autocorrelated observations (random effects model). In the first stage, autocorrelation parameters were estimated from the least squares residuals using the Yule‐Walker equations and used to “whiten” the data and the design matrix. In the second stage, the linear model was reestimated using least‐squares on the whitened data to produce estimates of effects and their standard errors [Worsley et al., 2002]. Data were modeled using two design matrices. To explore the neural correlates of deviance detection, a design matrix was used which consisted of onset vectors with events time‐locked to the violations within conditions containing them and comparable positions within nonviolated trials, with one additional vector for responses and one for the remaining stimulation periods of no interest, including the trials that were incorrectly responded to. To explore regular sequence processing, a design matrix was used with events time‐locked to the presentation of the first stimulus within each sequence and two additional vectors identical to the ones in the previously described matrix. Within both types of matrices the events related to each sequence type were modeled with the same duration. The design matrices were generated using a synthetic hemodynamic response function [Friston et al., 1998; Josephs et al., 1997] and, in case of the one used for modeling sequential violations, its first derivative. Contrast images, namely estimates of the raw‐score differences between specified conditions, were generated for each participant. Contrast images which simultaneously compared one type of a trial with two other trial types (e.g., position deviants contrasted with object and rhythm deviants) were calculated using contrast vectors of a form c = [2 −1 −1]. Specifically, the values in the contrast vector were set to “2” for one target trial (e.g., position deviant) and to “−1” for the two trials against which the target trial was contrasted (e.g., both object deviant and rhythm deviant). Single‐participant contrast images were entered into a second level random effects analysis for each of the contrasts. The group analysis consisted of one‐sample t‐tests across the contrast images of all participants that indicated whether observed differences between conditions were significantly different from zero (z > 3.09, P < 0.001, uncorrected) [Holmes and Friston, 1998]. To correct for false‐positive activations, the results were corrected using cluster‐size and cluster‐value thresholds obtained by Monte Carlo simulations (P < 0.005, corrected). For this purpose, all activated clusters were first identified using the threshold of z = 2.56. Next, the significantly activated clusters were selected at the predefined significance level of P = 0.005, corrected for multiple comparisons. In addition, effect sizes as indexed by the Cohen's d index [Cohen, 1992] were calculated and reported for each of the identified activations. Next, conjunction analyses of the calculated contrasts were performed in order to identify common regions supporting the conditions of interest. This was performed by identifying all voxels which exceeded the prespecified significant levels within each of the reported contrast maps entering the analysis. Therefore, the output maps from the conjunction analysis show the overlap between the contrasts of interest which corresponds to a logical “and” of the mentioned contrasts [Nichols et al., 2005]. Finally, a region‐of‐interest (ROI) analysis consisting of one‐sample t‐tests on ROIs selected from independent analysis across a group of contrast images was performed.

RESULTS

Behavioral Performance

Behavioral performance was assessed by calculating participants' sensitivity index d‐prime according to the signal detection theory [Green and Swets, 1966]. The obtained d‐prime indices were 2.53 ± 0.72 for SPT‐O, 3.10 ± 0.70 for SPT‐P, 2.57 ± 0.68 for SPT‐R, and 3.11 ± 0.85 for control. A repeated‐measures ANOVA with a four‐level factor task (SPT‐O, SPT‐P, SPT‐R, control) showed that the sensitivity significantly differed between the four tasks (F(3, 72) = 7.58, P < 0.001) such that, as revealed by additional pair‐wise comparisons, d‐prime was higher in SPT‐P in comparison with SPT‐O (P = 0.003) and SPT‐R (P = 0.027). Similarly, the sensitivity was higher in the control task in comparison with SPT‐O (P = 0.001) and SPT‐R (P = 0.0049). Participants' sensitivity was equivalent in SPT‐P and control (P = 1.0) as well as SPT‐O and SPT‐R (P = 1.0) tasks. Analysis of participants' response criteria revealed that in SPT‐R they maintained a generally conservative response criterion (c = 0.33 ± 0.30, t(24) = 5.55, P < 0.001) while in the other tasks no response bias was identified (SPT‐O: c = 0.08 ± 0.22, t(24) = 1.82, P = 0.081; SPT‐P: c = −0.01 ± 0.21, t(24) = 0.27, P = 0.789; control: c = −0.02 ± 0.18, t(24) = 0.63, P = 0.534). Given that the participants' responses were delayed and occurred at different time periods following the critical event in the trial, response times were not used as an additional measure of behavioral performance.

MRI DATA

Neural Correlates of Detecting Different Types of Sequential Deviants

Brain areas which were activated by the presentation of sequential deviants in the three types of SPT were revealed through the comparison of violated and ordered sequence trials (contrasts: violated object sequence vs. ordered object sequence; violated position sequence vs. ordered position sequence; violated rhythm sequence vs. ordered rhythm sequence; Table I, Fig. 2). As results, presentation of deviants in SPT‐O triggered only posterior activations in the right hemisphere encompassing the right inferior parietal lobule (IPL) and middle temporal gyrus (MTG). Bilateral IPL was activated in processing deviants within SPT‐R and SPT‐P where the precuneus was also activated. Additionally, in SPT‐P bilateral dorsal and superior ventral premotor cortex and middle frontal gyrus (MFG) activations were triggered by the presence of sequential deviants, while in SPT‐R more ventral activations along the right inferior frontal gyrus (IFG) were revealed. Given that no frontal activations were revealed by the contrast violated object sequence vs. ordered object sequence, it was hypothesized that this could be attributed to a more pronounced frontal contribution to processing all (both ordered and violated) SPT‐O when compared with SPT‐P and SPT‐R trials. Therefore, a direct comparison between ordered object sequence vs. ordered position and rhythm sequences was calculated and masked with the contrast comparing ordered SPT‐O sequences vs. control. This analysis indeed showed preferential engagement of frontal cortices (x = −41, y = 4, z = 30, max z = 5.81, d = 3.63; x = 37, y = 28, z = 33, max z = 4.36, d = 2.36; x = 28, y = 46, z = 12, max z = 3.81, d = 2.30), among other regions (x = 28, y = −62, z = 42, max z = 5.02, d = 3.13; x = −29, y = −65, z = 42, max z = 6.39, d = 4.18; x = −2, y = 13, z = 45, max z = 4.60, d = 2.74; x = 4, y = −62, z = 42, max z = 3.82, d = 1.82; x = −8, y = −11, z = 6, max z = 4.25, d = 2.13; x = 37, y = −65, z = −15, max z = 5.34, d = 2.90; x = −38, y = −62, z = −9, max z = 6.05, d = 3.89; x = −20, y = −74, z = 6, max z = 4.41, d = 2.02; x = 7, y = −71, z = −21, max z = 5.06, d = 2.93) in SPT‐O. In summary, the obtained results indicate that deviants embedded into sequences defined by position and rhythm stimulus properties mainly activated posterior prefrontal, premotor, and parietal cortices such that the activations elicited by position deviants were distributed more dorsally when compared to those elicited by rhythm deviants. In contrast, violations of object sequences elicited only a parieto‐temporal activation which might partially reflect the fact that ordered SPT‐O sequences generally engaged the prefrontal cortex to a higher degree than SPT‐P and SPT‐R ordered trials, making them very similar to violated SPT‐O trials in this respect.

Table I.

Anatomical brain area, hemisphere location, Talairach coordinates (x,y,z), maximal z‐score, size of significant activations and effect size (Cohen's d)

Anatomy	Hem	Talairach coordinates			z	mm3	d
		x	y	z
Violated vs. ordered object sequence
IPL (39/40)	R	55	−50	33	4.30	2,862	2.26
	R	58	−50	21	4.16		2.09
MTG (21)	R	58	−38	−3	4.27		2.13
Violated vs. ordered position sequence
PMC (6)	R	25	13	54	4.16	2,970	2.08
MFG (9)	R	52	16	30	3.48		1.64
PMC (6)	L	−35	4	39	3.79	1,350	1.84
MFG (6/8)	L	−44	10	45	3.97		1.96
PCU (7)	R	1	−65	60	3.96	2,862	1.95
IPL (39/40)	R	43	−41	51	3.90	1,458	1.91
	L	−44	−50	51	4.39	2,673	2.25
Violated vs. ordered rhythm sequence
IFG (45/47)	R	52	22	6	4.35	3,888	2.29
PrCG/IFG (6/44)	R	40	13	24	3.81		1.85
IPL (39/40)	R	55	−38	39	4.26	1,944	2.15
	L	−53	−50	39	3.81	1,404	1.85

Note: BA: Brodmann area; IFG: inferior frontal gyrus; IPL: inferior parietal lobule; MFG: middle frontal gyrus; MTG: middle temporal gyrus; PCU: precuneus; PMC: premotor cortex; PrCG: precentral gyrus.

Figure 2.

A: Brain correlates of detecting sequential deviants in SPT‐O (violated vs. ordered object sequence). Shown is the right hemisphere from parasaggital section (x = 60) and a set of axial images (in the first row: z = −10, 0, 10, 20; in the second row: z = 30, 40, 50, 60). B: Brain correlates of detecting sequential deviants in SPT‐P (violated vs. ordered position sequence). From left to right: left hemisphere from parasaggital section (x = −44), right hemisphere from parasaggital section (x = 38) and a set of axial images (in the first row: z = −10, 0, 10, 20; in the second row: z = 30, 40, 50, 60). C: Brain correlates of detecting sequential deviants in SPT‐R (violated vs. ordered rhythm sequence). From left to right: left hemisphere from parasaggital section (x = −55), right hemisphere from parasaggital section (x = 53) and a set of axial images (in the first row: z = −10, 0, 10, 20; in the second row: z = 30, 40, 50, 60). Group‐averaged statistical maps are superimposed onto a brain of one participant from the experiment which was scaled to the standard Talairach brain size [Talairach and Tournoux, 1988]. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Similarities in Processing Different Types of Sequential Deviants

To identify similarities in processing different types of sequential deviants, a conjunction analysis was performed on the three previously described contrasts (violated object sequence vs. ordered object sequence, violated position sequence vs. ordered position sequence, and violated rhythm sequence vs. ordered rhythm sequence). This analysis did not reveal any common activation. Thus, additional conjunctions of three pairs of individual contrasts were performed. First, a conjunction between pairs of contrasts violated object sequence vs. ordered object sequence and violated rhythm sequence vs. ordered rhythm sequence revealed a common activation in the right IPL (x = 55, y = −44, z = 33, max z = 3.72). Given that this portion of the right IPL activation was not revealed in the contrast violated position sequence vs. ordered position sequence, a post‐hoc region‐of‐interest (ROI) analysis independent from the contrast of interest was performed on this contrast. For this purpose, the right IPL coordinates identified in the conjunction of contrasts violated object sequence vs. ordered object sequence and violated rhythm sequence vs. ordered rhythm sequence were used to limit the volume of search. The obtained results indicated that this region was also activated for position deviants (t = 2.18, P = 0.02), but not to a degree high enough to be revealed using whole‐brain analysis. Second, a conjunction analysis between contrasts violated position sequence vs. ordered position sequence and violated rhythm sequence vs. ordered rhythm sequence showed an overlap in the left IPL (x = −53, y = −44, z = 39, max z = 3.43). Given that this region was not revealed in the contrast violated object sequence vs. ordered object sequence, a post‐hoc ROI analysis independent from the contrast of interest was also performed on this contrast. For this purpose, the coordinates of the left IPL identified in the conjunction of contrasts violated position sequence vs. ordered position sequence and violated rhythm sequence vs. ordered rhythm sequence were used to limit the volume of search. This analysis indicated a trend towards an activation in case of object deviants which, however, failed to reach statistical significance (t = 1.66, P = 0.06). Third, a conjunction between pairs of contrasts violated object sequence vs. ordered object sequence and violated position sequence vs. ordered position sequence revealed no common activations. To summarize, right and, to a smaller degree, left IPL were engaged in processing all three deviant types.

Differences in Processing Different Types of Sequential Deviants

In addition to comparing violated sequences with their respective ordered counterparts, direct contrasts of each type of violated sequences with the other two types of violations were calculated (violated object sequence vs. violated position and rhythm sequences; violated position sequence vs. violated object and rhythm sequences; violated rhythm sequence vs. violated object and position sequences). The obtained results indicated that deviants introduced into SPT‐O triggered stronger activations in the occipital and inferior temporal cortex including the bilateral fusiform gyrus than the two other deviant types. On the other hand, deviants introduced into SPT‐P were associated with activations within bilateral posterior temporal cortices while those presented in SPT‐R with activations within the IFG. In addition, all three types of sequential deviants preferentially activated different portions of the parietal lobe (Table II).

Table II.

Anatomical brain area, hemisphere location, Talairach coordinates (x,y,z), maximal z‐score, size of significant activations and effect size (Cohen's d)

Anatomy	Hem	Talairach coordinates			z	mm3	d
		x	y	z
Violated object vs. violated position and rhythm sequences
SPL (7)	R	31	−62	45	4.60	2,457	2.41
	L	−26	−71	42	4.18	1,782	2.09
PCU (7)	L	−5	−53	39	4.43	1,485	2.28
OGm (18)	R	22	−95	9	4.54	1,593	2.36
FG (36/37)	R	25	−47	−15	4.41	1,161	2.27
	L	−29	−47	−15	4.70	1,674	2.50
FG/OGm (18/19)	L	−26	−80	−6	4.13	1,323	2.06
CE	R	7	−68	−18	3.95	1,755	1.94
Violated position vs. violated object and rhythm sequences
SPL/PCU (7)	R	19	−47	63	5.27	14,121	3.00
	L	−26	−44	63	5.19	8,478	2.92
MTG (21)	R	43	−62	3	4.71	4,401	2.51
	L	−47	−65	3	4.56	1,593	2.38
Violated rhythm vs. violated object and position sequences
IFG/INS (44/45/47)	R	46	22	0	5.56	16,335	3.29
	L	−41	19	6	5.13	9,666	2.87
IPL (39/40)	R	55	−38	42	4.46	1,539	2.30

Note: CE: cerebellum; FG: fusiform gyrus; INS: insula; OGm: middle occipital gyrus, SPL: superior parietal lobule. For other abbreviations, see Table I.

Next, to identify regions which can be considered both as a correlate of deviance detection (as indicated by their stronger involvement in violated when compared to ordered sequences) and as showing preferential involvement in detecting one specific deviant type (as indicated by their stronger activation for detecting one when compared to other deviant types), conjunctions between the above reported contrasts and the respective contrasts comparing violated and ordered sequences were calculated. First, conjunction between contrasts violated position sequence vs. ordered position sequence and violated position sequence vs. violated object and rhythm sequences revealed a common activation in the precuneus (x = 7, y = −59, z = 48, max z = 3.69). Second, in the conjunction between contrasts violated rhythm sequence vs. ordered rhythm sequence and violated rhythm sequence vs. violated object and position sequences, activations within the right IPL (x = 55, y = −38, z = 39, max z = 4.09) and IFG (x = 52, y = 22, z = 6, max z = 4.41) were identified. Finally, no activations were revealed in the conjunction of violated object sequence vs. ordered object sequence and violated object sequence vs. violated position and rhythm sequences. In summary, with respect to differences in processing different types of violations, precuneus was preferentially engaged in processing position in contrast to the right IPL and IFG which were specifically engaged in detecting rhythm deviants. Additionally, preferential engagement of brain regions involved in processing particular feature of relevance (posterior occipital and temporal cortex for object, middle temporal gyrus for position and inferior parietal and frontal cortices for rhythm properties) was also identified when directly contrasting different deviant types.

DISCUSSION

The present study investigated the process of detecting sequential deviants within three types of perceptual sequences differing in the stimulus property relevant for defining the repeating sequential pattern. Such events represent one form of associative novelty characterized by a novel temporal arrangement (stimulus order) within the sequence. Although all deviant types within the experiment were highly similar and presented within perceptual sequences of equivalent organization, they nevertheless evoked somewhat distinct patterns of activation. These results suggest that processing associative novelty does not always rely on the same brain structures, but depends on the properties of such events and the context in which they are encountered.

Neural Correlates of Detecting Different Types of Sequential Deviants

To interpret the obtained results, the characteristics of stimuli employed within the serial prediction task (SPT) need to be considered in more detail. As previously described, these were abstract and composed of two elementary figures whose position, identity or presentation duration defined the stimulus dimension relevant for determining the sequential structure. They were shown within sequences whose processing typically engages the premotor and connecting parietal regions [Schubotz and von Cramon, 2002a, 2002b, 2002c, 2003] and is suggested to rely on forward models defined by styles of transformations of the object or body part they describe [Schubotz et al., 2008]. It has also been previously argued that the type of processing as evoked in this context is by nature predictive [Schubotz, 2007] and includes constant comparisons between the expected and realized stimuli (match‐mismatch comparison process).

Presenting deviants within sequences defined by position and rhythm stimulus properties (position and rhythm deviants) triggered comparable increases of activations within the lateral parietal, premotor, and prefrontal regions when trials containing deviants were compared to their ordered counterparts. Although these activations showed a restricted overlap within the left inferior parietal lobule (IPL), dissociation between them was also found: activations within the position SPT (SPT‐P) were distributed more dorsally and posteriorly in contrast to those from the rhythm SPT (SPT‐R) which were located more ventrally and anteriorly within both the parietal and frontal cortex. In contrast, detecting object deviants, namely violations within the object SPT (SPT‐O), elicited a widespread parieto‐temporal activation which was not accompanied by additional engagement of frontal areas. To a certain degree, the involvement of brain regions in detecting sequential deviants within SPT‐P and SPT‐R corresponds to the mapping which was previously identified in processing ordered sequences defined by spatial or rhythmical stimulus properties. Specifically, it has previously been shown that predictions based on spatial stimulus properties activate the dorsal part of the premotor cortex (PMC) in contrast to those based on rhythm properties which activate inferiormost portion of the ventral PMC [Schubotz and von Cramon, 2001; Schubotz et al., 2003, 2008]. This distribution of activations can be roughly related to findings showing that the PMC contains a movement or body representation comparable to the one contained in the primary motor cortex [Buccino et al., 2001; Corfield et al., 1999; Hamzei et al., 2002; O'Driscoll et al., 1995]. Specifically, the dorsal PMC which is more engaged in processing spatial sequences is also associated with preparing reaching movements in contrast to inferiormost portions of the ventral PMC which show a preference for rhythmic properties and are involved in preparing actions related to vocal and articulatory control [cf., Schubotz, 2004]. Interestingly, activations triggered by the presentation of sequential deviants within SPT‐P and SPT‐R did not only show a comparable differentiation across the dorsal‐ventral dimension, but also a shift toward the more anterior prefrontal regions in comparison to more posterior and premotor regions typically engaged in ordered sequencing.

When discussing the obtained results, more general findings showing the engagement of the dorsal PMC (especially its more anterior parts together with the frontal eye fields) in attentional processing [Bledowski et al., 2004a; Boussaoud, 2001; Chouinard and Paus, 2006] also need to be taken into account. Although the participants were constantly attending to spatial stimulus properties in SPT‐P, it is plausible to assume that the presentation of a deviant triggered an increase in their attentional engagement and more careful monitoring of the final stimuli which could be informative for making their decision regarding the compromised sequential order. In a comparable fashion, detection of violations pertaining to the rhythmical structure of a sequence required more focused reassessing within this stimulus property, triggering an increase of activation within the posterior inferior frontal gyrus (IFG). The coinvolvement of IPL in this context is plausible taking into account findings showing that this region is, together with the inferiormost portion of the ventral premotor cortex, involved in generating temporal expectations [Coull and Nobre, 2008]. The overall pattern of results pertaining to spatial and rhythm deviants is in line with the findings from Marois et al. [2000] who have shown partial preferential activation of dorsal brain regions in detecting spatial oddball stimuli. In contrast, events which violated the rhythmical trial structure lead to the involvement of Broca's area which has previously been involved in setting temporal expectations [Coull and Nobre, 2008] as well as music processing [Maess et al., 2001; Patel, 2003]. Unlike the deviants within SPT‐P and SPT‐R, events which violated expectations related to the object identity elicited only a right‐lateralized activation in the inferior parietal and temporal cortices, encompassing the temporo‐parietal junction (TPJ) which has, among other contexts, previously been related to detection of novel events and attentional reorienting [Corbetta and Shulman, 2002; Corbetta et al., 2008; Mitchell, 2008]. Kiehl et al. [2001] have previously suggested that a strong activation of posterior brain regions in detecting rare events may reflect a need for more visuo‐spatial processing supporting object recognition and spatial attention. Since participants in the present experiment had to attend to detailed stimulus features to verify the presence of an object deviant, such an analysis was very likely to be required in the present context. A lack of prefrontal activation in detecting this deviant type may be related to the fact that the demands in ordered and violated object sequences are more mutually similar than in other sequence types. Specifically, as shown through the direct comparison of object and the two other types of sequences, even ordered SPT‐O trials required more prefrontal engagement when compared to SPT‐P and SPT‐R trials. This may reflect the fact that in this task, unlike in SPT‐P and SPT‐R which afford continuous transformations between stimuli, participants needed to remember the exact identity of each stimulus which promoted more prefrontal‐dependent strategies such as, e.g., linguistic rule descriptions of the sequential structure. Interestingly, extended prefrontal involvement in detecting sequential deviants occurring in sequences defined by stimulus size, a dimension which also affords continuous transformations [Bubic et al., 2009], may speak in favor of this hypothesis. In summary, detecting different types of deviants embedded into perceptual sequences activated a widespread network incorporating different portions of the posterior, mainly parietal cortex coupled with, in case of position and rhythm deviants, lateral prefrontal and premotor cortices. The lack of more anterior activations in case of object deviants may be related to the fact that processing all, both ordered and violated, sequences defined by object properties is generally more dependent on the prefrontal cortices, resulting in marginal relative differences between the two trial types.

Comparing the Detection of Different Types of Associatively Novel Events

A conjunction analysis which was calculated in order to explore the similarities in processing different types of violations revealed no common activations across all three different deviant types. Instead, a portion of the left IPL was identified as relevant for processing deviants introduced into SPT‐P and SPT‐R, while its right hemisphere counterpart supported the processing of deviants within SPT‐O and SPT‐R sequences. Additional ROI analysis, however, also indicated significant contributions of the right IPL in processing position deviants, while the contribution of the left IPL in processing object deviants showed a trend which did not reach statistical significance. Taken together, these results indicate that the right and, to a lesser degree, left IPL supported the detection of different types of sequential deviants. Although clearly localized in the parietal lobule, these activations can be considered as belonging to or closely neighboring the TPJ. As mentioned earlier, contributions of this region to novelty or deviance processing have previously been widely reported, mainly in the context of the oddball paradigm [Bledowski et al., 2004b; Downar et al., 2000; Kiehl et al., 2001; Stevens et al., 2000]. Although its function is typically suggested to reflect behavioral relevance of such events [Corbetta et al., 2008], it has also been suggested that it is not behavioral relevance, but events' salience which determines the activation of this region [Downar et al., 2002]. Although the results of the present study may not distinguish between these two hypotheses, they provide evidence of this region's contribution in detecting not just oddball stimuli, but also sequential deviants which represent one category of associatively novel events.

With respect to the differences in processing the three types of sequential deviants, indirect evidence for a clear differentiation comes from nonoverlapping patterns of activations identified by comparing violated trials of the three sequence types with their ordered counterparts. The results from these analyses indicate that, although the detection of different types of sequential deviants may principally be related to a network incorporating lateral prefrontal, anterior premotor, and posterior, mainly parietal areas, specific task requirements and relevant stimulus features may strongly influence the relative contributions of these regions. While in some contexts different components of the network may show a similar degree of activation (e.g., SPT‐P, SPT‐R), in some others more posterior (SPT‐O) or more anterior [Bubic et al., 2009] areas may dominate the activation pattern. However, as indicated by the results of the present study, only a subset of regions which were identified as relevant for detecting sequential deviants by contrasting violated and ordered sequences were also significantly more activated when directly comparing the sequences containing different deviant types. These included a portion of the precuneus in case of position and, in case of rhythm deviants, the right inferior parietal lobule and the right inferior frontal gyrus. Thus, only these regions can indeed be considered as being preferentially involved in processing the mentioned types of deviants.

This does not, however, imply that the brain regions involved in processing different types of violations do not mutually differ to a high degree, as is suggested by the differences identified through direct comparisons of trials containing different deviant types. Naturally, the activations identified in this analysis differed from those revealed by contrasting violated and ordered sequences which is not surprising given that the factor of interest, namely the violation effect, was cancelled out in these direct comparisons. Instead, this analysis was sensitive to the specific stimulus features defining the violations which are reflected in the identified brain activations. Specifically, occipital and inferior temporal regions typically involved in visual recognition [Grill‐Spector, 2003; Tyler et al., 2004] showed stronger activation for detecting deviants within SPT‐O, while detecting violations of the rhythmical structure preferentially engaged inferior frontal regions involved in rhythmic and music processing [Maess et al., 2001; Patel, 2003]. Finally, detection of position deviants elicited stronger activation of posterior middle temporal cortices potentially overlapping with regions specialized for motion processing [Tootell et al., 1996], an area which has previously been identified in detecting stimuli presented in novel locations [Marois et al., 2000]. In addition, different portions of the parietal lobule were preferentially involved in processing different types of sequential deviants.

Taken together, the results of the present study indicate both similarities and differences in detecting different types of associatively novel events. With respect to similarities, lateral inferior parietal cortex, especially in the right hemisphere, showed involvement in processing all three types of deviants. Detecting different deviants was, however, also marked by distinct patterns of activations. Among these, the precuneus was preferentially engaged in processing position in contrast to the right IPL and IFG which were more engaged in processing rhythm when compared to other deviant types. Direct comparisons between different deviant types additionally indicated preferential engagement of brain regions involved in processing particular feature of relevance (posterior occipital and temporal cortex for object, middle temporal gyrus for position and inferior parietal and frontal cortices for rhythm properties) in detecting different deviant types. When these results are compared to the previously reported findings related to the detection of associative novelty in other contexts, a rather divergent picture emerges. For example, it has previously been convincingly shown that hippocampus may be crucial in detecting associatively novel events and supporting match‐mismatch comparison process required for this detection [Kumaran and Maguire, 2006, 2007b, 2009]. Interestingly, however, although a similar process of constant comparison between predicted and realized stimuli is also suggested to constitute the basis of deviant detection in the present study, hippocampus was not activated in detecting any type of sequential deviants. Although the results obtained in this study do not in any way undermine previous findings showing the relevance of this region in associative novelty, they raise the question about the generalizability of potential novelty detection mechanisms across all domains. Obviously, cognitive domain as well as specific task requirements, especially the participants' attentional involvement, number of sequence repetitions (one‐trial learning or repeated exposure to the pattern), type of employed stimuli or the timescale of stimulus presentation might in this context be equally important as the associative nature of presented deviants.

CONCLUSION

The present study investigated the process of detecting one category of associatively novel events, namely sequential deviants presented within three types of perceptual sequences differing in the stimulus property which defined the sequential pattern. Although highly similar and introduced into sequences of equivalent organization, the detection of deviants within different sequence types engaged somewhat different brain regions. These results illustrate that processing associatively novel events does not always rely on the same brain structures, but depends on the properties of such events and the context in which they are encountered. Importantly, some of the regions involved in detecting associatively novel events, e.g., the TPJ, probably subserve more general cognitive processes which are not specific to the detection of associatively novel events. This suggests a need for caution when generalizing across all types of sequential, or especially the broader class of relational deviants which makes the demanding task of identifying the generative mechanisms of associative novelty even more challenging than previously envisioned.