The role of the nucleus accumbens in knowing when to respond

Abstract

While knowing what to expect is important, it is equally important to know when to expect it and to respond accordingly. This is apparent even in simple Pavlovian training situations in which animals learn to respond more strongly closer to reward delivery. Here we report that the nucleus accumbens core, an area well-positioned to represent information about the timing of impending rewards, plays a critical role in this timing function.

Timing can be everything. Coming to Thanksgiving dinner a day early is sure to annoy your host and will likely earn you some cooking duties if you stick around, and stopping at an intersection at the first sight of yellow may well get you hurt. Even animals understand the importance of timing. This was demonstrated by Pavlov, who famously showed that dogs will learn to salivate (and do all sorts of other things) upon presentation of a cue predicting a food reward (Pavlov 1927). However, less well-appreciated is that the dogs also understood when to salivate. This was evident in the temporal pattern of the response, which started with cue onset but increased thereafter until the reward was actually delivered. Thus, the dogs understood not only what was coming (something that should be salivated over) but when it would be delivered (at the end of the cue) and were able to translate that knowledge to govern when to exhibit the appropriate response. Rats trained that a cue predicts food also learn to grasp the temporal relationship and group their responding accordingly (Vogel et al. 2003).

This effect, termed “inhibition of delay” by Pavlov, is part of a strong consensus among learning theorists that animals represent even the simplest of associations as a rich framework of independent elements (Rescorla 1988). For example, a rat exposed repeatedly to a light followed by food learns to associate the light with a response to obtain food. It also learns to evoke a representation of the food—the specific food—when the light is presented. Further, it comes to endow the light itself with some general value. Knowing when to respond is one more element of this rich associative framework.

However, while many of the informational elements reflecting what is known have been associated with neural circuits (Cardinal et al. 2002), the areas mediating the temporal control of responding have not been clearly identified. One hypothesis is that striatal medium spiny neurons (MSNs) encode time intervals from seconds to minutes (Meck et al. 2008). MSNs are the predominant cell type in the nucleus accumbens core (NAc). There is conflicting evidence for the role of NAc in timing, as instrumental performance in peak interval tasks is not affected by either lesions to NAc or dopaminergic dennervation of NAc (Meck 2006; Galtress and Kirkpatrick 2010). However, the role of NAc in Pavlovian timing has never been tested.

NAc receives strong inputs from areas known to signal information about time and impending outcomes (McDonald 1991; Voorn et al. 2004) and exhibits neurons that respond to Pavlovian cues with escalating firing related to the hedonic value of the impending outcome (Schultz et al. 1992). Here we tested this question by training accumbens lesioned rats in a simple Pavlovian conditioning task, using procedures designed to optimize the development of the delay of inhibition effect (please see Supplemental Methods for details).

Briefly, rats with sham (n = 8) or neurotoxic (n = 7; Fig. 1) lesions of NAc were trained for 12 sessions to associate either a tone or a white noise with a food pellet reward or nonreward. The cue paired with reward was counterbalanced within each group, and the cues were 30 sec long with reward delivered only at the end, in order to encourage the rats to time their responses. As reported previously, rats should spend an increasing amount of time in the food cup during the last part of the 30-sec CS+ in the latter half of training sessions (Delamater and Holland 2008). This is unlike procedures in prior studies of NAc, which typically report no effects of lesions on Pavlovian conditioning (Corbit et al. 2001; Hall et al. 2001).

Figure 1.

Neurotoxic lesions of NAc. Individual traces showing the extent of each neurotoxic lesion (n = 7).

As illustrated in Figure 2, this approach was effective; sham controls showed good conditioning to the cue paired with reward (CS+) across sessions and conditioned responding increased toward the end of this cue, especially after the initial few training sessions. Notably, this increase was not observed in lesioned rats; though they showed no statistical difference in their level of overall level of responding to the CS+ compared to controls, they failed to exhibit any change in responding during the 30-sec cue.

Figure 2.

Effect of NAc lesions on Pavlovian responding during 30-sec cues. Lines show time spent in the food cup during the three 10-sec periods of the CS+ (top panel) and CS− (bottom panel) for sham controls and lesioned rats during each of 12 conditioning sessions. Insets in top panel show average responding during each of the three cue periods during sessions 1–6 vs. 7–12. Bars indicate SEM. *, P < 0.05.

These impressions were confirmed by statistical analyses. While a three-factor ANOVA (group × session × 10-sec cue period) showed that responding to the neutral cue (CS−) did not differ between groups (Fs < 1.3, Ps > 0.2), analysis of responding to the CS+ revealed a significant main effect of session (F_(11,143) = 8.95, P < 0.01) and significant interactions between session and cue period (F_(22,286) = 2.46, P < 0.01), cue period and group (F_(2,26) = 3.52, P < 0.05), and session, cue period, and group (F_(22,286) = 2.39, P < 0.05). No other effects were significant (Fs < 2.8, Ps > 0.1).

The effect of lesions was even more striking when the data were analyzed separately for the first six training sessions, when learning was occurring, vs. the last six sessions, when the overall increase in responding to the CS+ had largely stabilized (Fig. 2, insets). An ANOVA of responding in the first six sessions revealed no main effects nor any interactions with group (Fs < 2.2, Ps > 0.15), and while there was a main effect of period (F_(2,26) = 5.41, P < 0.05), this reflected a significant decline rather than an increase in responding during the CS+ (P < 0.05). In contrast, an ANOVA of responding to the CS+ in last six sessions revealed a nearly significant interaction between group and period (F_(2,26) = 4.08, P = 0.056) and a highly significant interaction between group, period, and session (F_(10,130) = 2.8, P < 0.01). Further post-hoc testing showed that responding increased significantly during the CS+ in sham controls (P < 0.05) but not in lesioned rats (Ps > 0.6).

These results suggest that a neural circuit including NAc mediates the timing of Pavlovian responses first reported by Pavlov nearly 100 years ago (Pavlov 1927); NAc-lesioned rats exhibited nearly normal levels of responding to the Pavlovian cue overall, but showed a complete loss of the normal temporal pattern of that responding. In subsequent testing, responding during extinction and also in a conditioned reinforcement task was unaffected in the lesioned rats (see Supplemental Results for more details). This suggests that the effects of lesions on responding were not due to a general effect of lesions on the acquisition of incentive motivational properties by the CS+. Rats were also equally motivated to retrieve the food pellets following termination of the CS+; both sham and NAc-lesioned rats responded at similar rates and spend similar amount of time in the food cup upon CS+ termination (see Supplemental Results for more details). This suggests that NAc lesions did not affect the general motivational value of either the US or the CS. Similarly, it is also unlikely that NAc lesions affected performance due to a general role in timing per se; at least two studies have found no role for NAc in the precise timing of instrumental responding (Meck 2006; Galtress and Kirkpatrick 2010). Instead, we would suggest that NAc-lesioned rats were impaired in timing the delivery of reward in a Pavlovian setting because they were unable to appropriately associate the end of the CS+ with the higher value of impending reward. This may reflect an inability to endow the end of the CS+ with higher incentive value. However, it seems more likely that this role is secondary to a general role in signaling information about the impending outcome, including its approximate time of delivery relative to the CS+.

Such a role in signaling information about impending outcomes is consistent with recent reports implicating NAc (and dopamine specifically) in changes in Pavlovian responding after reinforcer devaluation (Lex and Hauber 2010; Singh et al. 2010). Indeed NAc receives strong input from orbitofrontal and amygdalar regions (McDonald 1991; Voorn et al. 2004) that are critical for Pavlovian behaviors guided by information about impending outcomes (Hatfield et al. 1996; Malkova et al. 1997; Gallagher et al. 1999; Izquierdo et al. 2004; Machado and Bachevalier 2007). NAc could extract from these sources temporal information related to impending outcomes and their value, perhaps without including information about the outcomes' specific sensory features. Such a model might explain the variable involvement of NAc in a variety of Pavlovian tasks (see Supplemental Discussion for more details). In this regard, it is notable that a reanalysis of previously acquired data from these two regions showed that inactivation of orbitofrontal cortex (and ventral tegmental area) but not amygdala disrupted proper timing of Pavlovian responding in a similar setting (see Supplemental Results for more details). Future behavioral and neurophysiological studies employing longer duration cues and other techniques that emphasize the role of timing will be necessary to more fully delineate this circuit and to test whether timing and other Pavlovian information is dissociable on a neural level.