Medial Prefrontal Cortex Predicts and Evaluates the Timing of Action Outcomes

Abstract

The medial prefrontal cortex (mPFC) is active in conditions of performance monitoring including error commission and response conflict, but the mechanisms underlying these effects remain in dispute. Recent work suggests that mPFC learns to predict the value of actions, and that error effects represent a discrepancy between actual and expected outcomes of an action. In general, expectation signals regarding the outcome of an action may have a temporal structure, given that outcomes are expected at specific times. Nonetheless, it is unknown whether and how mPFC predicts the timing as well as the valence of expected action outcomes. Here we show with fMRI that otherwise correct feedback elicits apparent error-related activity in mPFC when delivered later than expected, suggesting that mPFC predicts not only the valence but also the timing of expected outcomes of an action. Results of a model-based analysis of fMRI data suggested that regions in the caudal cingulate zone, dorsal mPFC, and dorsal anterior cingulate cortex were jointly responsive to unexpectedly delayed feedback and negative feedback outcomes. These results suggest that regions in anterior cingulate and mPFC may be more broadly responsive to outcome prediction errors, signaling violations of both predicted outcome valence and predicted outcome timing, and the results further constrain theories of performance monitoring and cognitive control pertaining to these regions.

Introduction

The evaluation of expectations against observed outcomes can identify discrepancies that may in turn drive learning in a complex, dynamic environment. Indeed, it has been suggested that this process provides the mechanism by which performance monitoring and reinforcement learning are instantiated in humans and other primates (Holroyd and Coles, 2002; Rescorla and Wagner, 1972; Sutton and Barto, 1990). Structures along the medial wall of the frontal lobe have recently emerged as potential loci of processes requisite to expectation evaluation and subsequent behavioral adjustments (Rushworth et al., 2004), especially with regard to the outcomes of chosen actions (Walton et al., 2004). However, while there is substantial evidence that medial prefrontal cortex (mPFC) and anterior cingulate cortex (ACC) subserve performance monitoring and related functions, the details of evaluative mechanisms at play remain largely unknown. In particular, it is likely that evaluative mechanisms address both the expected valence and expected timing of impending events but the importance of the latter domain has not yet been thoroughly investigated.

The putative role of ACC and surrounding mPFC in the formation and evaluation of expectations has been explored through numerous studies addressing both outcome anticipation and evaluation. Error signals, in particular, may indicate a discrepancy between actual and expected outcomes (Scheffers and Coles, 2000), but see (Botvinick et al., 2001; Holroyd et al., 2005). The ACC and mPFC respond to aversive outcomes including errors (Carter et al., 1998; Dehaene et al., 1994; Luu et al., 2000; Mathalon et al., 2003; Niki and Watanabe, 1979; Wang et al., 2005), negative feedback (Holroyd and Coles, 2002; Miltner et al., 1997; Ullsperger et al., 2007), and pain (Peyron et al., 2000; Talbot et al., 1991). Moreover, the magnitude of the ACC/mPFC response to aversive outcomes reflects the degree to which actual and expected outcomes differ (Brown and Braver, 2005), suggesting that these regions signal the occurrence of an outcome that is worse than expected. More generally, ACC/mPFC may track action outcomes over time to adjust expectations of future events (Alexander and Brown, 2009; Behrens et al., 2007; Botvinick, 2007; Christopoulos et al., 2009; Croxson et al., 2009; Holroyd and Coles, 2002; Kennerley et al., 2009). Discrepancies in outcome valence or timing may provide a training signal to adjust future outcome predictions (Alexander and Brown, In Press).

Much interest in ACC/mPFC function has focused on the observation that, in primates, this region receives a preponderance of cortical projections emanating from the mesencephalic dopamine system (Berger et al., 1991; Gaspar et al., 1989; Williams and Goldman-Rakic, 1998). The ACC in turn projects reciprocally to the dopaminergic cells via the striosomes of the striatum (Eblen and Graybiel, 1995). The role of midbrain dopamine nuclei in reinforcement learning and related processes (Schultz, 1998) has already been well established (see Berridge and Robinson, 1998 for review). Importantly, activation (or deactivation) of midbrain dopamine neurons has been observed in many contexts in which ACC/mPFC responsivity has also been noted. Similar to findings in the ACC/mPFC, midbrain dopamine neurons have been demonstrated to signal both predicted positive (Ljungberg et al., 1992; Schultz et al., 1993) and negative outcomes (Mirenowicz and Schultz, 1996) and have been suggested to signal the occurrence of outcomes that are both better and worse than those predicted (Fiorillo et al., 2003; Schultz et al., 1997).

Importantly, midbrain dopamine neurons have also demonstrated sensitivity to the timing of expected outcomes, although the source of these timed predictions remains to be determined (Brown et al., 1999). Rewards presented earlier than expected result in a spike in the midbrain dopamine signal upon reward reception with no corresponding change in firing at the expected time of reward. Meanwhile, rewards presented later than expected result in suppression of the dopamine signal at the latency of expected reward and a phasic increase in firing upon later reward reception (Hollerman and Schultz, 1998). When reward timing is uncertain, suppression of the dopamine signal both precedes and follows reward presentation, whereupon a coinciding phasic increase in dopamine firing has been reported to occur (Schultz et al., 1993). Modulation of midbrain dopamine firing patterns by both outcome timing and outcome valence suggests inclusion of both types of information in expectations about future events. Given the interaction between dopamine signals and ACC/mPFC, such results also suggest that the ACC/mPFC may be similarly sensitive to violations of expected outcome timing and that detection of such violations may result in heretofore overlooked cognitive control consequences. It is possible that the source of these timed expectations is the mPFC. The hypothesis that timed expectation signals in mPFC influence dopamine signaling stands in contrast to the theory that the direction of causality goes the other way, i.e. that dopaminergic pauses drive mPFC activity (Holroyd and Coles, 2002). In either case, the prior question is whether mPFC is sensitive to the timing of expected outcomes.

Evidence of medial prefrontal sensitivity to the timing of expected reward is currently lacking. However, one study by Amador and colleagues (2000) describes a pattern of anticipatory firing in “reward predicting” primate supplementary eye field (SEF) neurons during reward anticipation, which may constitute a timed prediction of reward. When reward delivery occurs at the expected latency, an abrupt cessation of firing is observed in this subset of anticipatory SEF neurons. However, when the expected reward is omitted, the anticipatory activity increases even further. The net result is greater activation for reward omissions than rewards, which casts the well-known error effects in mPFC as reflecting an unmet prior expectation of reward. Further, the pattern of firing observed in anticipatory SEF neurons suggests prediction of reward at a particular, learned latency, effectively represented in the latency of peak firing. (A similar pattern of anticipatory firing has also been reported in midbrain dopamine neurons when reward delivery at a learned latency is uncertain (Fiorillo et al., 2003)).

These findings suggest a simple model in which temporally structured excitation representing expected events is suppressed by inhibition as a result of actual occurrences (Figure 1). In both SEF and midbrain dopamine neurons, reward anticipation is associated with an increase in neural firing, but reward delivery is associated with suppression of the anticipatory signal. When the valence, magnitude, and timing of reward are as anticipated, the excitatory outcome anticipation signal and inhibitory outcome evaluation signal should exactly cancel each other. However, mismatch between expected and actual events could result in a discrepancy between these signals that may be monitored to indicate the occurrence of prediction error (see Figure 1). In general, there may be multiple possible outcomes of a given situation, whether appetitive or aversive, each expected with a corresponding probability at a particular time.

Figure 1.

Proposed model of discrepancy evaluation for temporally-structured predictions. In the model, mPFC is excited by a temporally-structured signal (“expectation”) that predicts events as the anticipated outcome of actions. When the predicted event occurs, an inhibitory signal (“outcome”) suppresses the prediction signal. Thus, when an expected event does not occur with the expected timing, the Discrepancy Evaluator becomes active to signal a mismatch between expected and actual events.

The present study aims to answer the questions of whether and how ACC/mPFC activity is sensitive to the timing, and not just the valence, of anticipated feedback during both anticipatory and evaluative intervals. Based on the theory depicted in Figure 1, we hypothesize that timed prediction signals may be found during the anticipatory interval, followed by signal suppression upon outcome occurrence. The net magnitude of signal during the evaluative interval is expected to reflect the degree of discrepancy between anticipated and actual outcome timing (i.e. the temporal prediction error signal). In the present study, expectations regarding the timing of performance feedback were manipulated such that both accurate temporal predictions and temporal prediction errors could be assessed. If the temporal prediction error signal directly corresponds with firing patterns reported for midbrain dopamine neurons, the ACC/mPFC should be responsive to feedback presented at unexpected latencies.

In order to specifically isolate the effect of violations of temporal expectation, the valence of feedback was predictable and only correct feedback trials were included in analysis of temporal expectancy effects. The timing of feedback alone was varied, so that the contributions of processes related to cues and motor responses were similar and therefore not confounded in the contrasts of different feedback times. Finally, given recent evidence that the latency of feedback may affect relative feedback valence, in and of itself (Bromberg-Martin and Hikosaka, 2009), the present study sought to differentiate the neural response to violations of expected valence and timing through systematic manipulation of expected timing. Because delay discounting and sustained effortful processing may impart a negative valence to late feedback, both early and late feedback were presented in contexts in which they were expected, unexpected, or equally likely. In effect, valence related effects of early or late feedback could be successfully distinguished from effects specific to violations of temporal expectancy. Both standard and model-based (O'Doherty et al., 2007) approaches to analysis were utilized to differentiate anticipatory and evaluative activation within the studied time intervals and expectancy conditions.

Materials and Methods

Participants

Twenty-three subjects (eleven female) were recruited for participation in the present study through fliers posted in the community. Subjects were between the ages of eighteen and forty (mean = 23.8, SD = 7.6) and were right-handed. All subjects were free of contraindications to fMRI research including pregnancy, metallic implants in the body, a history of epilepsy or claustrophobia, and body weight exceeding 440 pounds. Subjects earned $25 for each hour of participation in imaging procedures and an additional $15 for completion of psychometric inventories. All study procedures were approved by the Human Subjects Division of the Indiana University Office of Research Administration and each subject provided informed consent in accordance with its policies.

Data from one subject was excluded from analysis due to a mean response time exceeding two standard deviations of the group mean. Exclusion of this individual did not significantly alter group demographic composition.

Task Description and Procedures

A variant of the two-choice prediction task, originally described by Paulus (1997), was developed for the purpose of the present study. Before the start of the task, subjects were informed that they would see a man waiting in front of his house for a friend arriving by car at the beginning of each trial. Subjects were instructed to predict whether the car would arrive on the left or right side of the house by pressing a corresponding button and were informed that the car would usually appear in the same location for several trials in a row. In order to maximize prediction accuracy, subjects were instructed to predict the side of the house at which the car had previously arrived. The location of the car switched every ten trials on average, inducing an error rate approximating 10%.

Each trial began with a cartoon cue depicting a man standing in front of a house for 2 seconds. Subjects predicted arrival of the car on the left side of the house by pressing a button under the left index finger and predicted arrival on the right side of the house by pressing a button under the right index finger. From the time of cue onset, subjects had up to 3 seconds to respond. For each response, both response time and prediction accuracy data were recorded. A blank screen delay interval followed cue offset and was subsequently replaced by a feedback presentation. Feedback consisted of a cartoon depicting the location of the car (left or right of house), the predicted location as indicated by the position of the man (left or right of house), and either the word “Correct” or the word “Error,” in accordance with prediction accuracy. The duration of delay and feedback events was manipulated to create early and late latency feedback conditions. Early feedback trials had a 1 second delay interval, followed by presentation of the feedback image for 2.5 seconds. Respective durations of these events were reversed for late feedback trials such that the delay lasted 2.5 seconds and feedback lasted 1 second (see Figure 2). In effect, the total duration of early and late feedback trials was equivalent despite delay of feedback by an additional 1.5 seconds in the late relative to the early condition. The intertrial interval was randomly set to either 2, 4, or 6 seconds with an exponential distribution (Dale, 1999) to allow estimation of the event-related BOLD responses.

Figure 2.

Two-choice prediction task. Participants engaged in a modified version of the two-choice prediction task (Paulus, 2005). A cue stimulus depicting a man waiting in front of a house was presented at the onset of each trial. The participant made a leftward or rightward response to predict the upcoming location of an arriving car. Participants were instructed that the car would generally arrive on the same side of the house for several consecutive trials so predictions were to be based on prior trial outcomes. Feedback indicating the actual location of the car was presented after a short or long delay. The proportion of short and long delay trials was manipulated across blocks to induce expectation of early feedback (ExpectEarly), late feedback (ExpectLate), or either early or late feedback (ExpectEither). On a given trial, actual feedback latency could be congruent or incongruent with expected feedback latency.

Each subject completed 30 practice trials in the scanner prior to functional image acquisition. Six blocks of 60 experimental trials were subsequently completed during acquisition of functional imaging sequences, with approximately 30 seconds of rest between blocks. The proportion of early and late feedback trials was varied across blocks in order to establish three discrete expectation conditions, with regard to feedback latency. Two of the six blocks contained 80% early feedback and 20% late feedback trials (‘ExpectEarly’ condition), 2 contained equal proportions of early and late feedback trials (‘ExpectEither’ condition), and 2 contained 20% early feedback and 80% late (‘ExpectLate’ condition). Like block conditions were always presented consecutively with the specific order of presentation counterbalanced across subjects.

MRI Acquisition

Acquisition of imaging data was performed using a 3.0 T Siemens Magnetom Trio whole body system equipped with an 8-channel head coil. For the functional imaging, thirty-three, 3 mm, axial slices tilted 30° toward the coronal plane were acquired for each volume using an EPI pulse sequence with interleaved slice acquisition, slice gap of 1 mm, TR of 2000 ms, TE of 25 ms, flip angle of 70°, 64 × 64 acquisition matrix, and 220 × 220 mm field of view. This allowed whole brain coverage including the cerebellum. During each of the six task blocks, 240 volumes were acquired, the first of which was discarded as the magnetic field reached equilibrium. During image acquisition, task stimuli were presented using E-Prime software (Psychology Software Tools; Pittsburgh, PA) and projected onto a screen inside the scanner bore where participants could view them and make responses on an fMRI-compatible manual response panel, placed under both hands. High-resolution T1-weighted structural scans were also acquired from each subject to allow co-registration between the functional images and a standard atlas.

fMRI Analysis

AFNI’s 3dDespike utility (http://afni.nimh.nih.gov/afni) was used to reduce spike artifacts in the functional imaging dataset through interpolation of the underlying signal. All subsequent preprocessing and analyses were conducted using Statistical Parametric Mapping software (SPM5; Wellcome Department of Cognitive Neurology, UK), implemented on a Matlab platform (Matlab 7.8; Math Works, Natick, MA). For each subject, differences in the acquisition time of functional image slices were corrected using sync-interpolation (Oppenheim et al., 1999) and head movements were corrected using a least squares approach and a rigid body spatial transformation with 3 translation parameters and 3 rotation parameters. Distortion and movement-by-susceptibility artifacts in functional images were corrected using the SPM5 FieldMap toolbox (Andersson et al., 2001). Corrected functional data were then co-registered to individual, high-resolution structural images and segmented into grey and white-matter probability maps. These segmented images were used to calculate parameters for normalization to the standard MNI template image. Normalization parameters were then applied to functional images such that data for each participant was converted to standard MNI space. Normalized functional data were then spatially smoothed using an 8-mm full-width/half-maximum isotropic Gaussian kernel and scaled to a global mean intensity of 100. A 1/128 Hz highpass filter was applied to voxel timecourses to attenuate low frequency noise.

Single subject fixed effect analysis was performed with 6 types of events corresponding to the three expectation conditions (‘ExpectEarly’, ‘ExpectEither’, and ‘ExpectLate’) crossed with the two feedback latency conditions (‘GetEarly’ and ‘GetLate’). Together, these 6 event types represented all correctly predicted trials completed during the task. Two additional events represented trials on which negative feedback was delivered and trials for which no response was recorded. Events for each regressor were modeled by convolving the time of the behavioral response on each trial with the canonical hemodynamic response function. Six motion regressors representing the 3 directions of translational movement and 3° of rotational movement were also included as confounds for participants who exhibited movement in excess of 3 mm during the recording session or who moved more than 0.5 mm between acquired volumes. Two participants met criteria for inclusion of motion regressors.

Parameters for each regressor (beta weights) were estimated separately for each block using the general linear model. Linear contrasts were specified to discern the effects of early versus late feedback in each expectation condition. The contrast ExpectEarlyGetLate-ExpectEarlyGetEarly was defined to isolate regions responsive to unexpected late feedback (‘Unexpected Late’) while its converse (ExpectEarlyGetEarly-ExpectEarlyGetLate) identified regions responsive to the expected occurrence of early feedback (‘Expected Early’). Similarly, the contrast ExpectLateGetEarly-ExpectLateGetLate was defined to isolate regions responsive to unexpected occurrence of early feedback (‘Unexpected Early’) with the converse contrast (ExpectLateGetLate-ExpectLateGetEarly) exposing regions responsive to the expected occurrence of late feedback (‘Expected Late’). The contrasts ExpectEitherGetEarly-ExpectEitherGetLate and ExpectEitherGetLate-ExpectEitherGetEarly were defined to identify regions responsive to early (‘Neutral Early’) versus late (‘Neutral Late’) feedback, respectively, when feedback timing was uncertain. Given the predicted similarity between neural responses signaling violations of temporal expectancy and those signaling violations of expected outcome valence, the contrast Incorrect-Correct was also defined to ascertain effects of ‘Negative Feedback’.

Model-based fMRI analysis

In order to better distinguish activation associated with predictive and evaluative functions during both early and late feedback intervals, and to evaluate the results in the model-based framework of Figure 1, a second general linear model was implemented to estimate beta weights for ten model-based regressors. One main advantage of this model-based approach is that it allows us to directly look for brain regions that may respond in direct proportion to the probability of early feedback in different blocks, which is less feasible with the classical GLM approach. Similarly, regressing out the probability effects allows us to look for regions showing main effects of temporal expectancies and outcomes, collapsed across the probability manipulations. Separate model-based regressors were generated to represent the probability (p) and main (m) effect of feedback during the early (1) vs. the late (2) periods in the trial, for trials when feedback actually occurred early (EARLY) or late (LATE). In effect, early feedback trials were modeled using four independent regressors. The mEARLY₁ regressor was set to 1 at the latency of early feedback when early feedback was presented, thus representing early feedback occurrence. A second regressor, mEARLY₂, was set to 1 at the latency of late feedback when early feedback was presented, representing non-occurrence of late feedback during these trials. Copies of these regressors were parametrically modulated by the mean-centered probability of the feedback timing. Specifically, the regressors pEARLY₁ and pEARLY₂ were included to represent the a priori probability of feedback presentation at early and late latencies for those trials in which feedback occurred at the early latency. Within the ExpectEarly condition, pEARLY₁ was set to 0.8 and pEARLY₂ was set to 0.2. These values were reversed for the ExpectLate condition and were both set to 0.5 for the ExpectEither condition. Analogous main effect (mLATE₁, mLATE₂) and probability (pLATE₁, pLATE₂) regressors were generated for late feedback trials. Importantly, because probability regressor values were set irrespective of actual feedback latency, probability regressor values representing early and late trials were governed by the same criteria within each expectation condition. Probability regressors were normalized such that the mean probability regressor value was zero. Two additional regressors were generated to represent the main effect of cue presentation and the occurrence of negative feedback. All model-based regressors were then convolved with the canonical hemodynamic response function prior to being entered into the general linear model. Beta weights of model-based regressors were estimated for the entire acquisition session of 6 runs. The fact that the early and late regressors are placed so closely to each other in time might raise concerns about how well the activations can be regressed onto early vs. late sources. This issue is addressed by the results, which show consistently distinct effects that load on the early vs. late period regressors (see Results), and which are consistent with complementary findings from the conventional GLM analysis.

Several contrasts were defined on the basis of model-based regressors to identify expectation- and evaluation-related activity at early and late feedback latencies. The contrast mLATE₁- mEARLY₁ was specified to isolate regions that were more responsive to feedback non-occurrence than to feedback occurrence during the early feedback latency. Similarly, the contrast mEARLY₂- mLATE₂ was defined to identify regions that were more responsive to feedback nonoccurrence than to feedback occurrence during the late feedback latency. The converse of each main effect contrast was also computed to identify regions that were differentially responsive to feedback occurrence, relative to feedback non-occurrence, at each latency. Contrasts were similarly determined for model-based probability regressors. The contrast pEARLY₁-pLATE₁ was specified to identify regions with a stronger condition-specific expectation signal at the latency of early feedback on early relative to late feedback trials. Given that the preponderance of early trials occurred during the ExpectEarly condition, this contrast was expected to reveal regions integral to generating a timed, condition-specific early expectation signal. Similarly, the contrast pLATE₂-pEARLY₂ was defined to identify regions involved in generation of a timed, condition-specific late expectation signal. The converse of each probability contrast was also defined to isolate areas that represent the probability of early feedback on late feedback trials and vice versa.

Statistical Analysis

All contrasts were estimated for single subjects and then entered into a second-level analysis with subject as a random effect factor, using SPM5. One-sample t-tests were computed for the whole brain to identify regions with significant activation across subjects for each contrast. Significant regions were defined as clusters of eight or more voxels, significant at p < 0.001 (uncorrected), that survived cluster level correction for multiple comparisons at p-values reflecting a whole brain search volume. Due to widespread cortical activation associated with the model-based contrast, mLATE₁-mEARLY₁ (in agreement with the model predictions of Figure 1), a more stringent criterion requiring survival of voxel-by-voxel family-wise error correction at p < 0.05 was applied to identify key regions of interest (ROIs). When applicable, one way, repeated measures ANOVA was utilized to probe differential signal change across contrasts within ROIs.

Given our hypothesis that some mPFC regions would be jointly responsive to negative feedback and unexpectedly timed correct feedback, conjunction analyses were conducted for key model-based contrasts. Significant clusters identified for the Negative Feedback – Positive Feedback contrast (see Supplementary Table S3) were applied to results of mLATE₁-mEARLY₁ and mEARLY₂-mLATE₂ model-based contrasts as a small volume search mask. The mLATE₁-mEARLY₁ and mEARLY₂-mLATE₂ contrasts were expected to isolate regions responsive to nonoccurrence of correct feedback during early and late feedback latencies, respectively. According to the proposed model, increased activation following feedback nonoccurrence may reflect a temporal prediction error signal when feedback fails to occur at the predicted time. By extension, significant clusters identified within the mLATE₁-mEARLY₁ search mask should represent brain areas responsive to both negative and unexpectedly delayed feedback. Similarly, significant clusters within the mEARLY₂-mLATE₂ search mask are believed to represent regions jointly responsive to negative and unexpectedly early feedback. An analogous approach was utilized to identify regions responsive to both unexpectedly early and unexpectedly late feedback. For this conjunction analysis, significant clusters for the mEARLY₂-mLATE₂ contrast were defined as small volume search mask and applied to results of the mLATE₁-mEARLY₁ contrast.

Dependent measures for behavioral analysis were response time (RT) and accuracy. Two-way, repeated measures analysis of variance (ANOVA) was employed to explore differences in behavioral measures across expectation and feedback latency conditions. Post-hoc, paired t-tests were subsequently conducted to clarify ANOVA results as necessary.

Results

Behavioral Performance

A 3 (Expectation Condition) × 2 (Feedback Latency) repeated measures ANOVA revealed a significant main effect of expectation condition on trial RT (F(2,42) = 4.10, p = 0.024). Post-hoc pairwise comparisons indicated that RTs were significantly faster for the ExpectLate (M = 692 ms, SD = 169) condition than for the ExpectEarly (M = 745 ms, SD = 213; t(21)= 2.90, p = 0.009 (two-tailed)) and ExpectEither (M = 736 ms, SD = 176; t(21)= 3.05, p = 0.006 (two-tailed)) conditions. We speculate that the shorter response times may reflect frustration with generally longer feedback times. In general, delayed reward is less rewarding at the behavioral (Myerson and Green, 1995) and neural levels (Bromberg-Martin and Hikosaka, 2009), and lack of reward may indicate a need to change strategy or otherwise exert more effort to achieve the desired reward (Bush et al., 2002; Modirrousta and Fellows, 2008). An analogous ANOVA of trial error rates (ERs) returned no evidence of a significant main effect of either factor or interaction between factors. In addition, one sample t-tests verified that error rates observed across ExpectLate (M = 0.12, SD = 0.03), ExpectEarly (M = 0.12, SD = 0.04), and ExpectEither (M = 0.11, SD = 0.02) conditions did not significantly differ from the projected 10%.

Neuroimaging data - Conventional analysis

Late – Early Contrasts

The model of Figure 1 suggests that unexpectedly delayed feedback should lead to activation at the time when early feedback would otherwise occur. As a test of this prediction, the results of linear contrasts comparing early and late feedback trials within each expectation condition are presented in Table 1. Clusters of eight or more voxels (significant at p < 0.001 (uncorrected)) were identified as ROIs if found to be significant at the 0.05 α-level, following cluster level correction. As predicted, unexpected occurrence of late feedback in the ExpectEarly condition resulted in significant activation of a region in caudal cingulate zone (CCZ; see Fan et al., 2008; Picard and Strick, 2001 for clarification of nomenclature) area 24 (shown in yellow in Figure 3(a); peak voxel at −6, −6, 48), in comparison with the ExpectEarlyGetEarly condition (ExpectEarlyGetLate – ExpectEarlyGetEarly; z = 4.01, p = 0.001, cluster-corrected). Relative to early feedback, late feedback was also associated with significant medial prefrontal activation under conditions of uncertain feedback latency (i.e. ExpectEither) in which the ExpectEitherGetLate – ExpectEitherGetEarly comparison revealed significant activation of a cluster in dorsomedial prefrontal cortex (dmPFC) area 9 (shown in blue in Figure 3(a); peak voxel at −4, 42, 26; z = 5.02, p < 0.001, cluster-corrected). In the ExpectLate condition, by contrast, only a single cluster in left middle frontral gyrus (peak voxel at −16, −16, 58) exhibited significantly enhanced activation to late, relative to early feedback (ExpectLateGetLate – ExpectLateGetEarly; z = 4.01, p = 0.045, cluster-corrected).

Table 1.

Significant results of standard fMRI analysis contrasting early and late feedback within expectancy conditions.

Contrast		Peak coords.			Peak	Cluster	Cluster
Anatomical area		x	y	z	Z-score	p-value	size
ExpectEarlyGetLate - ExpectEarlyGetEarly (Unexpected Late)
Caudal cingulate zone (BA 24)		−6	−6	48	4.01	0.001	187
ExpectEitherGetLate - ExpectEitherGetEarly (Neutral Late)
Superior temporal gyrus (BA 22)	L	−44	−14	−8	5.38	0.000	316
Dorsomedial prefrontal cortex (BA 9)		−4	42	26	5.02	0.000	737
Cerebellum: posterior lobe (uvula)	L	−16	−78	−34	4.79	0.000	289
Medial globus pallidus	R	20	−10	−8	4.59	0.000	169
Middle temporal gyrus	L	−52	−38	−8	4.55	0.028	105
Middle temporal gyrus (BA 21)	R	58	−30	−4	4.22	0.000	425
Supramarginal gyrus (BA 40)	L	−54	−48	26	3.79	0.019	114
ExpectLateGetLate - ExpectLateGetEarly (Expected Late)
Middle frontal gyrus (subgyral)	L	−16	−16	58	4.01	0.045	82
ExpectLateGetEarly - ExpectLateGetLate (Unexpected Early)
Fusiform gyrus (BA 18)	L	−20	−82	−12	6.25	0.000	10907
ExpectEitherGetEarly - ExpectEitherGetLate(Neutral Early)
Middle occipital gyrus (BA 19)	L	−32	−84	0	6.35	0.000	9786
ExpectEarlyGetEarly - ExpectEarlyGetLate (Expected Early)
Fusiform gyrus (BA 18)	R	18	−80	−8	5.99	0.000	8982

Figure 3.

Results of conventional fMRI analysis: Regions of interest and corresponding mean percent signal change by contrast. (A.) Regions of interest were identified in CCZ (BA24) and dmPFC (BA9) for late feedback trials in the ExpectEarly and ExpectEither conditions, respectively. An additional region in dACC (BA32) was found to exhibit significantly enhanced activation on incorrect relative to correct feedback trials. Presentation of feedback at the early latency was associated with enhanced visual cortex activation relative to late feedback, irrespective of expectation condition. (B.) Mean percent signal change (vertical axis) was calculated for each contrast in each region of interest. Regions identified in CCZ (BA24) and dACC (BA32) appeared to exhibit selective responsivity to unexpected late feedback and negative feedback, respectively. Significant variation in mean percent signal change across contrast conditions was not observed for the dmPFC (BA9) or visual cortex regions of interest.

ROIs identified in CCZ and dmPFC demonstrated selective responsivity to unexpected late feedback, but it remains to be determined whether these effects are specific to a subset of the ExpectEarly, ExpectEither, or ExpectLate feedback probability blocks. Also, given that regions may be jointly responsive to both unexpected feedback timing and unexpected feedback valence, possible activation of CCZ and dmPFC on negative feedback trials should also be explored. A one-way ANOVA was implemented to determine whether mean percent signal change in CCZ and dmPFC ROIs varied significantly across four contrast conditions: ExpectEarlyGetLate-ExpectEarlyGetEarly, ExpectEitherGetLate-ExpectEitherGetEarly, ExpectLateGetLate-ExpectLateGetEarly, and Incorrect-Correct (see Figure 3(b)). The CCZ region of interest demonstrated significant variation in mean percent signal change across the specified contrast conditions (F(3,63) = 6.46, p = 0.001), primarily reflecting a larger effect of unexpected late feedback (Figure 3(b)). Caution is warranted in interpreting this effect though, because the unexpected late contrast was used to identify the region in the first place, so it is not surprising that this effect is the strongest in the region. The mean percent signal change did not exhibit significant variation across contrast conditions in dmPFC (F(3,63) = 1.56, p = 0.207).

Given the model of Figure 1, we expected to find a timecourse of activation such that an early anticipatory signal is opposed by subsequent feedback occurrence, and this prediction is largely borne out by the observed BOLD timecourses. The average fMRI timecourses for early and late feedback trials within each expectancy condition are plotted for CCZ in Figure 4(a) with the average negative feedback timecourse included for comparison. Anticipation of early feedback was found to yield a larger early excitation, with signal suppression in CCZ following feedback occurrence. Suppression was most robust upon presentation of early feedback, regardless of expectation condition, but was also considerable following the occurrence of expected late feedback. Signal suppression in CCZ was found to be weaker upon occurrence of unexpected or uncertain late feedback. Of note, unmet expectations of early feedback seemed to yield larger CCZ activity in proportion to the probability of early feedback. This finding is explored more directly in the model-based GLM analysis below. A similar pattern of results was identified in average fMRI timecourses estimated for dmPFC (see Figure 4(b)). Again, robust signal suppression followed the occurrence of early feedback with reduced suppression for trials in which late feedback was presented. However, in this region, the degree of suppression following late feedback was not similarly mediated by expectation condition, with all late feedback conditions exhibiting similar levels of signal suppression.

Figure 4.

Average fMRI timecourses for medial prefrontal and cingulate regions of interest by feedback latency and expectation condition, aligned on response time. Data points are spaced by one TR, i.e. 2 secs. (A.) Average fMRI timecourses for the region identified in CCZ (BA24) reveal robust response suppression on early feedback trials and attenuated suppression on late feedback trials. Timecourses reveal evidence of signal modulation by expectation condition on late feedback trials wherein increased attenuation of signal suppression coincided with unexpectedness of late feedback. (B.) The region identified in dmPFC (BA9) also exhibited increased signal suppression on early relative to late feedback trials. However, in contrast with results for CCZ, the degree of signal suppression did not appear to be mediated by expectation condition. (C.) Average fMRI timecourses for the dACC (BA32) region of interest reveal a robust positive signal for negative feedback trials, signal suppression for early feedback trials, and no net signal change from baseline for late feedback trials.

Early – Late Contrasts

Unlike late-early feedback contrast results, early-late feedback contrasts did not expose any regions of significant activation within cingulate or medial prefrontal cortices. Instead, within each expectancy condition, presentation of early feedback resulted in significant activation of visual cortex relative to presentation of late feedback (see Table 1, Figure 3(c)). Mean percent MR signal change was not found to vary significantly across expectation conditions in these regions.

Negative Feedback

The contrast incorrect – correct was utilized to isolate regions responsive to negative feedback. Regions of significant activation comprised a network including insula (area 13), dorsal ACC (dACC) area 32, ventral posterior cingulate cortex (vPCC) area 30, and inferior frontal gyrus (see Supplementary Table S1); all regions associated with error-related activation elsewhere in the literature (Braet et al., 2009; Chevrier et al., 2007; Hester et al., 2004; Li et al., 2008; Magno et al., 2006; Menon et al., 2001). The region identified in dACC (peak voxel at 4, 17, 44) lay just anterior to the region in CCZ identified with unexpected occurrence of late feedback (shown in red in Figure 3(a); z = 4.85, p < 0.001, cluster corrected). One-way ANOVA demonstrated significant variation in dACC signal change across the four contrast conditions explored for CCZ and dmPFC: ExpectEarlyGetLate-ExpectEarlyGetEarly, ExpectEitherGetLate-ExpectEitherGetEarly, ExpectLateGetLate-ExpectLateGetEarly, and Incorrect-Correct (F(3,63) = 5.22, p = 0.014; see Figure 3(b)). Examination of average fMRI event timecourses for this region demonstrated robust activation upon presentation of negative feedback. However, similar to timecourses derived for CCZ and dmPFC, robust signal suppression occurred upon presentation of early correct feedback. Presentation of late feedback resulted in no net suppression or activation and did not appear to be mediated by expectation condition (Figure 4(c)). A significant difference was additionally noted between timecourses associated with early and late negative feedback events at TR=5 within the timecourse estimate (t(21)= 5.20, p < 0.001 (two-tailed)). Late negative feedback resulted in greater signal change in dACC than early negative feedback; possibly reflecting a greater violation of expectancy when feedback is both later than expected and of an unexpected valence (see Figure 4(c)). Significant differences between early and late negative feedback timecourses were also noted in CCZ at TR=4 (t(21)= 3.88, p = 0.001 (two-tailed)) and dmPFC at TR=4 (t(21)= 2.26, p = 0.035 (two-tailed). Consistent with effects noted for early and late correct feedback events in these regions, early negative feedback was associated with greater signal suppression than late negative feedback (see Figure 4(a & b).

Learning Effects

Given that participants were not informed of the feedback latency manipulation, task experience may critically affect the formation and evaluation of temporally-structured expectations. In order to explore this hypothesis, each of the previously reported contrasts was evaluated for learning effects by comparing activation across the first and second blocks of each expectancy condition (in the case of Late-Early and Early-Late contrasts) or activation across the first and last two blocks of the task (in the case of the incorrect-correct feedback contrast). A single region in right supramarginal gyrus (area 40; peak voxel at 44, −48, 40; z = 3.91, p = 0.010, cluster corrected) was found to exhibit enhanced activation in the second ExpectEarly block, relative to the first, for the ExpectEarlyGetLate-ExpectEarlyGetEarly contrast. No significant learning effects were identified for other key contrasts. The absence of significant leaning effects more generally and in the ACC/mPFC specifically, however, is not entirely surprising. It is possible that learning occurs fairly rapidly, even within the first block of each expectancy condition, a finding that is consistent with earlier work (see e.g. Brown & Braver, 2005, Figure 3). In addition, the number of trials included in each block constitutes a relatively small sample size, making single block estimates less reliable than those based on the entire experimental session.

Neuroimaging data - Model-based analysis

Main Effect Contrasts

The results of model-based main effect contrasts are presented in Table 2. Clusters of eight or more voxels (significant at p < 0.001 (uncorrected)) that passed whole-brain cluster and/or family-wise error correction at p < 0.05 were identified as regions of significant activation. Family-wise error correction criteria were specifically applied in identification of ROIs for the contrast, mLATE₁-mEARLY₁, due to particularly robust activation associated with this comparison. The model-based regressor contrast, mLATE₁-mEARLY₁, reflects the neural response at the time of early feedback when feedback is delayed. This contrast revealed widespread activation across frontal, temporal, and parietal lobes reflecting delayed feedback (shown in blue in Figure 5). Importantly, this contrast specifically revealed significant activation of bilateral CCZ (area 24; z = 6.19, p < 0.001, Bonferroni corrected and z = 5.62, p = 0.002, Bonferroni corrected, for the left region (peak voxel at −6, −8, 48) and right region (peak voxel at 8, −8, 50), respectively), encompassing the CCZ ROI defined for the ExpectEarlyGetLate-ExpectEarlyGetEarly contrast, as might be expected. In addition, an ROI in dmPFC (area 9; peak voxel at −4, 38, 40) was identified (z = 5.70, p = 0.001, Bonferroni corrected), similar to that defined for ExpectEitherGetLate-ExpectEitherGetEarly. Somewhat surprisingly, there was a particularly strong effect in right premotor and especially right primary motor cortex, which was not found in the corresponding conventional analysis. Nonetheless, the findings are otherwise substantially consistent between standard and model-based analyses in that the model-based contrast, mLATE₁-mEARLY₁, represents the main effect of early feedback nonoccurrence at the early feedback latency. Also consistent with results of the standard analysis, the converse contrast, mEARLY₁-mLATE₁ primarily resulted in activation of visual cortex, with additional significant regions in the bilateral thalamus and right precuneus (area 7). Overall, the consistent findings suggest that the early and late period model-based regressors were able to effectively capture the BOLD responses, despite their close temporal spacing.

Table 2.

Significant results of model-based fMRI analysis contrasting main effect regressors representing early and late feedback trials.

Contrast		Peak coords.
					Peak	Cluster	Cluster	FWE
Anatomical area		X	y	z	Z-score	p-value	size	p-value
mLATE1 – mEARLY1 (FWE p < 0.05)
Primary motor cortex (BA 4)	R	28	−30	54	6.29	0.000	52	0.000
Caudal cingulate zone (BA 24)		−6	−8	48	6.19	0.000	216	0.000
Superior temporal gyrus	R	50	−42	8	6.02	0.000	43	0.000
Middle temporal gyrus (BA 21)	R	60	−20	−10	5.98	0.000	108	0.000
Supramarginal gyrus (BA 40)	L	−58	−48	24	5.89	0.000	76	0.000
Inferior frontal gyrus (BA 47)	L	−48	40	−12	5.75	0.000	14	0.001
Dorsomedial prefrontal cortex (BA 9)		−4	38	40	5.70	0.000	58	0.001
Caudal cingulate zone (BA 24)		8	−8	50	5.62	0.000	29	0.002
Precuneus (BA 7)		8	−56	32	5.48	0.000	22	0.004
Premotor Cortex (BA 6)	R	40	−8	40	5.42	0.000	10	0.006
Middle frontal gyrus (BA 6)	R	22	−16	62	5.37	0.000	25	0.007
Middle temporal gyrus (subgyral)	L	−50	−42	−8	5.26	0.000	11	0.013
Caudate (extra-nuclear)	R	22	−42	8	5.26	0.000	12	0.013
Inferior parietal lobule	R	62	−40	24	5.24	0.000	11	0.015
mEARLY1 – mLATE1
Lingual gyrus (BA 18)	R	20	−84	−2	6.50	0.000	15815	0.000
Thalamus	L	−22	−30	−2	6.10	0.000	174	0.000
Thalamus	R	24	−28	4	5.93	0.000	218	0.000
Precuneus (BA 7)	R	28	−58	54	4.32	0.007	105	0.636
mEARLY2 – mLATE2
Insula (extra-nuclear)	L	−32	−28	−4	5.04	0.000	717	0.042
Middle temporal gyrus	L	−52	−46	−12	5.96	0.000	247	0.065
Dorsal PCC (BA 31)		6	−54	26	4.84	0.000	1170	0.118
Parahippocampal gyrus	R	26	−16	−20	4.68	0.000	379	0.231
Precuneus (BA 39)	L	−44	−78	34	4.63	0.000	692	0.274
Medial frontopolar cortex (BA 10)	L	−12	56	−2	4.48	0.000	203	0.434
Cerebellum: posterior lobe (tuber)	R	42	−70	−40	4.45	0.000	195	0.471
Fusiform gyrus (BA 20)	R	56	−6	−30	4.44	0.000	89	0.475
Inferior temporal gyrus (BA 21)	L	−56	−14	−20	4.34	0.000	171	0.613
Angular gyrus	R	48	−66	34	4.21	0.000	228	0.771
Middle frontal gyrus (BA 6)	L	−30	8	52	4.13	0.000	303	0.850
Lateral globus pallidus	L	−18	−6	2	4.12	0.010	100	0.865
Posterior medial frontal cortex	R	10	−20	56	4.08	0.007	106	0.898
Medial frontopolar cortex (BA 10)	R	10	60	12	4.08	0.022	85	0.899
Putamen	R	26	−20	8	3.99	0.002	129	0.949
Sublobar (extra-nuclear)	L	−8	−28	20	4.69	0.006	108	0.992
mLATE2 – mEARLY2
Superior occipital gyrus (BA 19)	L	−28	−82	24	4.90	0.000	834	0.087
Lingual gyrus (BA 18)	L	−24	−76	−10	4.85	0.000	3335	0.111
Middle occipital gyrus (BA 19)	R	28	−82	18	4.66	0.000	411	0.249
Superior parietal lobule (BA 7)	L	−30	−58	50	4.29	0.009	102	0.975
Inferior frontal gyrus (BA 9)	R	46	6	30	4.28	0.000	428	0.689
Inferior parietal lobule (BA 40)	R	32	−58	48	4.12	0.001	153	0.864

Figure 5.

Regions of interest identified through model-based canonical contrasts. The model-based contrast, mLATE₁ – mEARLY₁ (blue), representing nonoccurrence of feedback at the early feedback latency, revealed significant activation of bilateral CCZ (BA24) and medial dmPFC (BA9). In contrast, non-occurrence of feedback at the late feedback latency was associated with enhanced responsivity of dPCC (BA31) and anterior mPFC (including BA10), relative to late feedback occurrence (mEARLY₂ – mLATE₂, cyan). Meanwhile, the model-based contrast of Negative Feedback – Positive Feedback (red) revealed enhanced activation of vPCC (BA23) and dACC (BA32).

The third model-based contrast, mEARLY₂-mLATE₂, represented the main effect of feedback nonoccurrence at the late feedback latency on trials in which feedback was presented at the early latency. Similar to results reported for mLATE₁-mEARLY₁, this contrast exposed activation across frontal, temporal, and parietal cortices (shown in cyan in Figure 5). Of particular interest were regions identified in the dorsal PCC (dPCC; area 31; peak voxel at 6, −54, 26; z = 4.84, p < 0.001, cluster corrected) and medial frontopolar cortex (including area 10; peak voxel at 10, 60, 12; z = 4.08, p = 0.022, cluster corrected). While a similar pattern of activation was not obtained for analogous standard analysis contrasts (e.g. ExpectLateGetEarly-ExpectLateGetLate), these results may expose additional regions responsive to violations of temporal expectancy. However, because early feedback occurrence signals the likely non-occurrence of feedback at the late latency, evaluative processes during this interval may qualitatively differ from those implicated in the detection of early omissions. Once again, the converse contrast, mLATE₂-mEARLY₂ primarily revealed regions of significant activation in visual cortex, consistent with representation of late feedback occurrence.

Probability Contrasts

Results of probability contrasts are summarized in Supplementary Table S2. A region of significant activation was identified in the medial anterior lobe of the cerebellum for the contrast pLATE₁ – pEARLY₁ (see Supplementary Figure S1(b)). The converse contrasts revealed two significant clusters in left middle frontal gyrus. The contrast, pEARLY₂-pLATE₂, was associated with significant activation in cuneus (area 19) and postcentral gyrus (area 3) while its converse exposed significant activation in superior frontal gyrus (BA 8). Surprisingly, despite the apparent relationship in Figure 4 between the block-wise probability of early feedback and the CCZ activation when feedback was unexpectedly late, there probability regressors failed to show a significant effect in the CCZ region.

Negative Feedback

The contrast between the model-based negative feedback regressor and the average of correct trial mEARLY₁ and mLATE₂ regressors (i.e. Negative Feedback – Positive Feedback) was also computed. In concert with standard analysis results, significant regions of activation were identified in the vPCC (area 23; peak voxel at 4, −26, 28; z = 5.77, p < 0.001, cluster corrected), dACC (area 32; peak voxel at −4, 6, 48; z = 4.91, p < 0.001, cluster corrected), lingual gyrus (peak voxel at 12, −86, 0; z = 6.93, p < 0.001, cluster corrected), and inferior frontal gyrus (area 47; peak voxel at −32, 22, −6; z = 5.63, p < 0.001, cluster corrected). Additional regions in inferior parietal lobule (area 40), premotor cortex (area 6), middle frontal gyrus (area 6), and thalamus also proved to be significant for this contrast (see Supplementary Table S3).

Conjunction Analysis

Error signals might reflect the omission of expected reward, mediated by a common mechanism that detects both unexpectedly delayed and omitted reward. To explore this possibility further, the model-based contrasts of delayed (mLATE1 - mEARLY₁) and error feedback (Negative Feedback – Positive Feedback) were entered into a conjunction analysis to explore regions jointly responsive to the nonoccurrence of early feedback and the presentation of negative feedback (see Methods). As predicted, results of this conjunction analysis revealed a region of significant joint activity in CCZ (area 24; peak voxel at −6, 2, 52; z = 4.96, p < 0.001, cluster corrected), extending into dmPFC (area 9) and dACC (area 32). Significant joint activation was also identified in bilateral insula (area 13), left supramarginal gyrus (area 40), right inferior parietal lobule (area 40), right premotor cortex (area 6), and left middle frontal gyrus (area 6). These results are summarized in Figure 6 (upper panel) and Supplementary Table S4.

Figure 6.

Results of conjunction analysis identifying regions of common responsivity across model-based contrasts. Regions in CCZ (area 24), dACC (area 32), and dmPFC (BA9) were found to exhibit enhanced activation following both the non-occurrence of early feedback (mLATE₁ – mEARLY₁) and the presentation of negative feedback (Negative Feedback – Positive Feedback; red). Regions in dPCC (area 31), posterior medial frontal cortex (area 6), and frontopolar area (area 10) exhibited joint responsivity to feedback non-occurrence during both early and late feedback periods (conjunction of mLATE₁ – mEARLY₁ and mEARLY₂ – mLATE₂; cyan).

Importantly, a similar conjunction analysis with the contrasts of late feedback nonoccurrence (i.e. late latency activation on early feedback trials; mEARLY₂ – mLATE₂) and error identified no regions exhibiting common responsivity. In effect, while a frontoparietal network of regions, jointly responsive to early feedback omission and negative feedback was identified, no regions were significantly responsive to both negative feedback and late feedback nonoccurrence. This is consistent with an interpretation of mPFC error effects as reflecting the unexpected omission of correct feedback at the expected time, as in Figure 1 (Jessup et al., In Press). Importantly, in agreement with the model proposed by Holroyd and Coles (2002), these results also suggest that error signals in the ACC/mPFC occur upon the earliest indication that expectations have been violated. In the current study, both the omission of expected early feedback and the presentation of error feedback represent the earliest available indicators of expectancy violation and may thus result in an mPFC-mediated error signal. In contrast, the presentation of unexpectedly early feedback indicates that later feedback omission is to be expected, presumably negating an error signal at this later latency.

One possibility consistent with the model of Figure 1 is that some regions might be jointly sensitive to the absence of feedback, whether at the time of early or late feedback. To test this, a third conjunction analysis was computed to identify regions that were jointly active under conditions of early and late feedback nonoccurrence (i.e. for the contrast mLATE₁ – mEARLY₁, masked by significant clusters of the contrast mEARLY₂ – mLATE₂). Consistent with results for late feedback nonoccurrence, this conjunction analysis revealed three regions of significant joint activation along the midline in dPCC (area 31; peak voxel at 8, −56, 32; z = 5.48, p < 0.001, cluster corrected), right posterior medial frontal cortex (area 6; peak voxel at 12, −20, 54; z = 5.77, p < 0.001, cluster corrected), and medial frontopolar cortex (area 10; peak voxel at 12, 58, 14; z = 4.93, p = 0.002, cluster corrected). Significant joint activation was also noted for regions in bilateral angular gyrus (area 39), left middle temporal gyrus, left inferior temporal gyrus (area 21), right parahippocampal gyrus, right fusiform gyrus (area 20), left insula (BA 13), left medial globus pallidus, and right dorsal striatum. These regions may also underlie processing of temporal expectations, though through mechanisms equally relevant to evaluation of unexpected early and late feedback. The results of this conjunction analysis are summarized in Figure 6 (lower panel) and Supplementary Table S4.

Discussion

In the present study both standard event-related and model-based approaches were utilized to explore brain regions involved in the formation and evaluation of temporally structured outcome predictions. It was hypothesized that regions in medial prefrontal and cingulate cortices would demonstrate selective responsivity to discrepancies between excitatory expectation signals and inhibitory outcome evaluation signals. Both anticipatory and evaluative signals are expected to reflect temporal features of outcomes such that ACC/mPFC will signal discrepancies in the predicted and actual timing of events. The results of both standard and model-based approaches were largely consistent with this hypothesis. Standard event-related contrasts identified a region of CCZ that was differentially responsive to unexpected late feedback. Similarly, a region in dmPFC demonstrated significant responsivity to late feedback in the ExpectEither condition. Further exploration through model-based contrasts suggested that both CCZ and dmPFC were responsive to nonoccurrence of early feedback, though it was unclear that these effects were significantly modulated by the probability of early feedback. However, it is important to note that a standard contrast targeting regions responsive to expected late feedback was not associated with medial prefrontal or cingulate cortex activity. These results suggest that the model-based findings for early feedback omission were largely driven by unexpectedly late feedback trials. Standard and model-based results also converged in showing negative feedback effects in a broad network of error sensitive regions including dACC.

Occurrences and Non-occurrences

Standard event-related early minus late feedback contrasts exclusively revealed visual cortex activation and did not show modulation of activity by expectation. However, model-based contrasts revealed enhanced activity on early-feedback trials relative to late-feedback trials at the late feedback latency in numerous regions outside of visual cortex. At the latency of late feedback, feedback nonoccurrence resulted in increased activation relative to feedback occurrence in a network of regions including dPCC (area 31), medial frontopolar cortex (area 10), insula (area 13), and the posterior lobe of the cerebellum. Failure of standard event-related contrasts to reveal significant activation of these regions may be explained by the increased temporal precision of the model-based analytic approach. However, given that the model-based contrast in question did not differentiate trials on the basis of the probability of early feedback, findings regarding the role of these regions in signaling the anticipated probability of feedback timing were inconclusive.

Unexpected timing

Conjunction analysis revealed that dPCC (area 31), posterior medial frontal cortex (area 6), and frontopolar area (area 10) were more responsive to feedback nonoccurrence than to feedback occurrence at both early and late feedback latencies. This is striking as it constitutes activation due to the absence rather than the presence of stimuli and is consistent with learned expectations of the timing of an outcome. Moreover, regions identified are not merely responsive to the putative negative valence of delayed feedback (Bromberg-Martin and Hikosaka, 2009), since they respond to unexpectedly early feedback, and may instead play a role in evaluating temporally-framed expectations against external events. In particular, the area of posterior medial frontal cortex (area 6) identified may represent another target of midbrain dopamine projections conveying outcome valence and timing information. In addition, posterior cingulate cortex (BA 31) has recently been associated with prospective imagination and planning (Vann et al., 2009), suggesting a possible role of this region in outcome anticipation and outcome-informed behavioral adaptation. Indeed, given that no regions of common activation were identified for late feedback nonoccurrence and negative feedback, it is possible that the network of regions identified for the conjunction of early and late feedback nonoccurrence are primarily involved in expectation revision rather than error detection per se.

Error effects

A separate conjunction analysis identified several regions that were jointly responsive to negative feedback and the nonoccurrence of early feedback, consistent with the interpretation of error effects as reflecting the absence of expected reward. Importantly, CCZ (area 24), dACC (area 32), and dmPFC (area 9) passed significance for this conjunction analysis, indicating similar responsivity in both negative feedback and delayed feedback contexts. However, as suggested above, common responsivity across contexts of negative and delayed feedback could reflect either a prediction error signal sensitive to violations of valence and of timing or a signal reflecting valence evaluation exclusively (wherein delayed feedback constitutes a negative outcome). Given this ambiguity, evidence suggesting modulation of medial prefrontal and cingulate signals by feedback timing expectations is particularly informative. The region in CCZ identified through the ExpectEarlyGetLate – ExpectEarlyGetEarly contrast was selectively responsive to late feedback when early feedback was expected. This result was further borne out in average fMRI timecourses plotted for this region. The amplitudes of timecourses associated with late feedback were graded with respect to feedback timing expectations with a greater probability of late feedback predicting greater signal suppression upon feedback presentation. Nonetheless, this result must be treated with caution as the model-based analysis did not reveal a significant effect of probability. In the same vein, a region in dmPFC (area 9) was identified for the contrast ExpectEitherGetLate-ExpectEitherGetEarly. While differential responsivity across expectation conditions was less pronounced in this region, it still demonstrated a selective response under conditions of uncertain feedback timing. In summary, CCZ (area 24) and dmPFC (area 9) demonstrated selective sensitivity to unexpected and uncertain late feedback. Neither region demonstrated a significant response to expected late feedback or to early feedback, regardless of expectation condition. These findings suggest that activation of CCZ and dmPFC may reflect a temporal prediction error signal and that ACC/mPFC regions found to be jointly responsive to negative and delayed feedback may be similarly implicated in generation of such a signal. Nonetheless, negative feedback and temporal expectancy violation signals may still drive distinct cognitive effects, as negative feedback should drive a change in task strategy, but temporal expectancy violations should not. Future studies might include designs in which temporal expectancy violations should in fact drive a change in task strategy, so that negative feedback and temporal expectancy violation effects can be compared more directly.

Cerebellar cognitive function

Several standard and model-based contrasts revealed significant activation of the cerebellum, which may reflect a cerebellar role in cognitive rather than motor function. In particular, bilateral posterior lobe activation was identified for the ExpectEitherGetLate-ExpectEitherGetEarly standard event-related contrast (see Supplementary Figure S1(a.)) and medial anterior lobe activation was demonstrated for the pLATE₁ – pEARLY₁ model-based contrast (see Supplementary Figure S1(b.)). Recent evidence has promoted expansion of the cerebellum’s ascribed role in motor planning and coordination to include various cognitive and affective functions (Middleton and Strick, 1997, 1998, 2001; Ramnani and Miall, 2001; Schmahmann and Sherman, 1998; Timmann et al., 2009). In particular, the cerebellum has recently emerged as the potential locus of mechanisms supporting time perception and prospective modeling processes, necessary to associative learning and higher order cognitive functioning (Bueti et al., 2008; Ivry and Spencer, 2004; O'Reilly et al., 2008; Rhodes and Bullock, 2002). Our results are consistent with a role for the cerebellum in representing the latencies of predicted outcomes at the level of cognitive function, in the same way that the cerebellum may time the latencies of movement signals at the level of motor control (Fiala et al., 1996).

Implications for theories of performance monitoring in mPFC

The present results inform current theories of performance monitoring in mPFC. The well-known error effects in mPFC have previously been interpreted as reflecting a comparison of actual vs. intended outcomes (Ito et al., 2003; Scheffers and Coles, 2000), or of conflict monitoring (Yeung et al., 2004). More recent work has suggested that mPFC compares actual vs. expected rather than actual vs. intended outcomes (Jessup et al., In Press), and that mPFC both predicts the outcome of actions and evaluates those predictions against actual outcomes (Alexander and Brown, 2008, In Press). In the present task, feedback occurs well after response, so it is unclear how response conflict could account for the observed timing effects at the time of expected feedback. The results suggest that mPFC signals the expected outcomes of an action, and not only with respect to valence, but also with regard to the timing of expected outcomes. Furthermore, these temporally-structured expectation signals may activate mPFC, but this activity is suppressed by signals reflecting the actual outcome that occurs as expected and when expected. Thus, mPFC may detect moment-by-moment discrepancies between actual and expected outcomes (Alexander and Brown, 2008, In Press).

Limitations and future work

Overall, the results are largely consistent with a model of temporally structured outcome prediction and evaluation, but several related questions remain. While the model-based analysis included regressors representing probable feedback timing to parse out anticipatory signals, few regions of interest emerged from related probability contrasts (i.e. pLATE₁ – pEARLY₁, pEARLY₁ – pLATE₁, pLATE₂ – pEARLY₂, and pEARLY₂ – pLATE₂). The absence of robust, coherent findings with regard to temporal expectation signal probability may be due to several factors. Firstly, it is possible that altering the probability of early versus late feedback did not constitute an adequately salient manipulation. Participants were not informed of the manipulation and did not stand to reap an explicit benefit by engaging in temporally accurate feedback anticipation. Strategies to increase effortful expectation of feedback may be utilized in future studies to better characterize anticipatory signals. Future model-bases analyses may also attempt to represent gradual learning of expected outcome timing through experience with each task block to improve the quality of results.

Another limitation of the current design is the close temporal proximity of anticipatory and evaluative intervals. Feedback anticipation is expected to begin upon response commission and continue until the latency at which feedback is expected and/or delivered, but this interval was kept relatively short by design, and no catch trials without feedback were included, in order to optimize the number of trials within a given run. While the model-based analytic approach was utilized to better separate anticipatory and evaluative events, the poor temporal precision of fMRI makes it possible that the model-based regressors with events barely 1.5 seconds apart may have been difficult to estimate. Nonetheless, in support of the model-based analysis design, the analysis revealed significant differences at the latencies of early vs. late feedback. Future research into the formation and evaluation of temporally-framed expectations may benefit from the enhanced temporal precision afforded by electrophysiological techniques.

Given that interest in evaluative mechanisms stems largely from cognitive control and decision-making consequences attributed to evaluative signals, future research must also address the behavioral consequences of temporal prediction error detection. Evaluative signals associated with ACC/mPFC have been associated with adaptive shifts in response timing (Gehring et al., 1993) and response strategy selection (Cohen and Ranganath, 2007; Hewig et al., 2009; Paulus et al., 2002), as well as in attention and inhibitory control (Durston et al., 2003; Hester et al., 2009). If ACC/mPFC also relay temporal prediction error signals, it is possible that such signals would also be found to inform ongoing behavior. In the present study design, the non-occurrence of early feedback necessarily informs the subject that feedback will occur late, and conversely, early feedback informs the subject that feedback will not occur later. Thus, information at the time of early feedback may lead to a cognitive updating of expectations regarding the time of late feedback. This updating process may be cognitively distinct from detecting the occurrence or lack there of at the time of early feedback, but a limitation of our design is that these detection and expectation updating processes cannot be distinguished. In particular, we cannot rule out that some of the observed temporal expectancy violation effects may reflect an underlying expectation update instead of temporal expectancy violation per se. Future research may address whether ACC/mPFC mediated detection of temporal prediction errors affects ongoing expectations, task performance, learning, and decision-making within the context of more demanding behavioral paradigms.

Research Highlights

• Anterior cingulate and medial prefrontal cortex predict and evaluate action outcomes

• These regions predict not only the nature but also the timing of action outcomes

• The regions may compute a difference between timed expectations and actual outcomes

• Error and unexpected feedback timing effects may reflect a common mechanism