Abstract
The development of alcoholism may involve a shift from goal-directed to habitual drinking. These action control systems are distinct in the dorsal striatum, with the dorsomedial striatum (DMS) important for goal-directed behavior and the dorsolateral striatum (DLS) required for habit formation. Goal-directed behavior can be modeled in rats with a fixed ratio (FR) reinforcement schedule, while a variable interval (VI) schedule promotes habitual behavior (e.g., insensitivity to contingency degradation). Using extracellular recordings from chronically implanted electrodes, we investigated how DMS and DLS neurons encoded lever-press responses and conditioned cues during operant alcohol self-administration in these two models. In rats self-administering 10% alcohol on a FR schedule, the DMS neuronal population showed increased firing at the onset of start-of-session stimuli. During self-administration, the most prominent phasic firing patterns in the DMS occurred at the time of reinforcement and reinforcement-associated cues, while the most prominent phasic activity in the DLS surrounded the lever response. Neural recordings from an additional cohort of rats trained on a VI schedule revealed a similar pattern of results; however, phasic changes in firing were smaller and differences between the medial and lateral dorsal striatum were less marked. In summary, the DMS and DLS exhibited overlapping but specialized phasic firing patterns: DMS excitations were typically time-locked to reinforcement, while DLS excitations were generally associated with lever responses. Furthermore, the regional specificities and magnitudes of phasic firing differed between reinforcement schedules, which may reflect differences in behavioral flexibility, reward expectancy and the action sequences required to procure reinforcement.
Keywords: dorsal striatum, extracellular electrophysiology, alcohol self-administration, habit
Introduction
Drug addictions, including alcoholism, are commonly defined by compulsive use despite negative consequences resulting from that use. The drug is initially sought for its rewarding properties; thus, drug-seeking is originally goal-directed (Balleine and Dickinson, 1998; Belin et al., 2009). Later, drug-seeking may transition to a habit that is outcome-independent and persistently elicited by alcohol-associated cues (Adams and Dickinson, 1981; Everitt and Robbins, 2005). Therefore, one aspect of addiction may be maladaptive learning that accompanies a shift from response-outcome representations to habitual, stimulus-response processes as the drug-seeking behavior becomes engrained (Everitt et al., 2001; Hyman, 2005).
The dorsal striatum supports action control, and behavioral reliance on this region differs between goal- and habit-like behavior (Yin and Knowlton, 2006). The dorsomedial striatum (DMS in rodent, caudate in primates) receives input from associative cortices (Alexander and Crutcher, 1990) and is required for goal-directed behavior. Specifically, DMS lesions impair goal-directed reward seeking and promote outcome-independent habitual behavior in rats (Yin et al., 2005; Corbit and Janak, 2010). Conversely, habitual behavior is thought to be dependent on the dorsolateral striatum (DLS in rodents, putamen in primates), which receives input from sensorimotor cortices (Alexander and Crutcher, 1990). For example, DLS lesions can prevent stimulus-response learning and habitual behavior (Yin et al., 2004). Moreover, operant responding for alcohol is sensitive to DMS manipulation early in training but is interrupted by DLS, but not DMS, manipulation after extended training (Corbit et al., 2012).
Medium spiny neurons (MSNs) encode information about conditioned cues and behavioral responses through phasic fluctuations in their firing rates (Carelli, 2002). DMS neuronal activity correlates with conditioned stimuli (White and Rebec, 1993; Kimchi and Laubach, 2009), while MSNs in the DLS can encode specific motor actions (West et al., 1990; White and Rebec, 1993; Rolls, 1994). However, few studies have directly compared DMS and DLS activity during operant tasks and, to our knowledge, no studies have monitored MSNs in the dorsal striatum during alcohol self-administration.
To address this gap in knowledge, we used extracellular electrophysiology to record DMS and DLS neuronal activity in rats trained to self-administer 10% alcohol under a fixed-ratio (FR) schedule of reinforcement that tends to produce goal-directed behavior (Dickinson, 1985; Yin et al., 2006). Recordings were also made in a second group of rats trained on a variable-interval (VI) reinforcement schedule that produces more persistent and habit-like operant behavior (Yin et al., 2006). Our data support the hypothesis that, consistent with the respective cortical inputs, neuronal firing patterns reflect alcohol-predictive cues in a greater proportion of DMS neurons, while more DLS neurons encode response initiation. Additionally, we predicted that associative DMS activity would predominate during goal-directed behavior, while response-related DLS activity would predominate during habitual behavior. However, the habit-inducing VI model produced greater overlap in neuronal firing patterns between the DMS and DLS, including more DLS post-reinforcement excitations than were observed in FR-trained rats. Moreover, DMS activations triggered by alcohol-associated cues tended to be farther posterior in VI-trained rats.
Methods
Subjects
Male Long-Evans rats (250-300g) were purchased from Charles River (Raleigh, NC, USA) or Harlan (Indianapolis, IN, USA) and individually housed under a 12h light/dark cycle. Except for the initial 5 days of operant training, rats received food and water ad libitum. Experimental procedures were performed in accordance with the NIH Guide for the Care and Use of Laboratory Animals and approved by the Institutional Animal Care and Use Committee of the University of North Carolina.
Behavioral Training
Experiment 1 & 2: General alcohol self-administration procedures
All rats were trained in daily sessions, Monday - Friday, in custom-built Plexiglas operant chambers in sound-attenuating cabinets (MedAssociates, St. Albans, VT, USA) as previously described (Robinson and Carelli, 2008). Briefly, each chamber contained a house light, two levers, two cue lights located above the levers, and two fluid-dispensing cups. Sessions began with the illumination of the house light followed 30s later by extension of the levers into the operant chamber. The first three sessions lasted up to 3h, and all subsequent training sessions were shortened to 30min. To facilitate alcohol self-administration, a sucrose-fading procedure was implemented over the first 20 sessions of training (Hay et al., 2013). Thereafter, alcohol deliveries were limited to a maximum of 25 in a session, after which point the session ended (levers retracted and house light extinguished).
Experiment 1: Fixed-ratio reinforcement schedule
Rats were initially trained on an FR1 schedule (1 lever-press response = 1 fluid delivery) with both levers reinforced, followed by FR3 schedule sessions, as previously described (Hay et al., 2013). After the third session, responses on one lever each session (either right or left) triggered fluid delivery, while the other lever was inactive (responses were recorded but had no consequences). At each reinforced response, 0.1mL of fluid was dispensed into the cup adjacent to the activated lever, and the following events occurred simultaneously and lasted for a 5s period referred to as the ‘time-out’: the cue light above the lever was illuminated, the house light was extinguished, and the levers were retracted. The reinforcement schedule increased to FR5 by the tenth session. In combination with the 25 alcohol delivery limit, alternation of the active lever each session between left and right prevented overtraining. The time-out period (with lever retraction) was gradually extended to 12s between session 20 and the commencement of electrophysiological recording.
Experiment 2: Variable-interval reinforcement schedule
A second group of rats was trained with sucrose-fading conditions as in Experiment 1, but with different criteria for fluid delivery (Hay et al., 2013). In Experiment 2, the location of the reinforced lever remained fixed throughout training (counterbalanced across animals). In this experiment there was no time-out period; thus, the levers remained extended throughout the session, and upon reinforcement the house light deactivation and cue light illumination always continued for 3.5s. In the first training session, fluid delivery and cue-light illumination occurred on a random-time 60s schedule. The rat was then trained on an FR1 schedule for 1-2 sessions before beginning on a VI7 schedule (VI7: after a variable interval with an average duration of 7s had elapsed, 1 response = 1 fluid delivery). Next, sessions were shortened to 30min and the reinforcement interval lengthened to 30s (VI30) by the 7th session.
Surgery
Surgery was performed after at least 6 weeks of training. Rats were anaesthetized with isoflurane (5% induction, 2% maintenance) and implanted with 16 stainless-steel, Teflon-coated electrodes (NB Labs, Denison, TX, USA; see Robinson and Carelli, 2008). Electrodes were 50µm in diameter and arranged ~0.5mm apart on two 1×8 arrays in an anterior-posterior orientation. One array was aimed at the DMS (0.2 – 2.2mm anterior, 1.7mm lateral, 4.5mm ventral from bregma) and the second at the contralateral DLS (0.2 – 2.2mm anterior, 3.4mm lateral, 4.5mm ventral from bregma), with sides counterbalanced across rats. Rats were monitored after surgery, given 15mg/kg ibuprofen daily for 3 days and allowed a week to recover before returning to the operant chambers.
Electrophysiology
After surgery, rats were habituated to the flexible tether that connected the electrode arrays to the headstage assembly. Training sessions continued in operant chambers equipped for electrophysiological recordings until operant behavior recovered to at least 17 reinforcements in a session (typically 5-6 days); the next session was the electrophysiological recording day. During all sessions on the tether, the chamber remained dark for 15min before session initiation, allowing the experimenter to select a differential reference and discriminate cells from background noise on the microwires. Neuronal activity was recorded using a multichannel acquisition processor (MAP system with SortClient software; Plexon, Inc., Dallas, TX, USA) while video was recorded from an overhead camera. Timestamps from the MAP system to the video and from the MedAssociates software to the MAP system were used to temporally align electrophysiological recordings with behavioral events.
Cell sorting was finalized after the experiment with Offline Sorter software (Plexon, Inc.). Automated clustering based on template analyses and principle component analyses were manually adjusted, guided by signal-to-noise measurements made during data collection (Robinson and Carelli, 2008). Signal-to-noise ratios ≥2, distinct principle component analysis clusters (determined during offline sorting), and physiological characteristics consistent with MSNs (i.e., ≤0.1% of spikes with interspike intervals <1ms and average firing rates <10Hz; Kish et al., 1999; Kimchi et al., 2009) were required for inclusion of neurons in analyses.
Satiety-specific devaluation and contingency degradation testing
Once all electrophysiological experiments were complete, rats were returned to the original training chambers for additional untethered self-administration sessions. In rats that maintained stable lever-press responding after electrophysiological recording, a satiety-specific devaluation test was conducted to assess behavioral flexibility (Hammond, 1980; Yin et al., 2006). To acutely devalue the alcohol reinforcer, rats were given 1h access to 10% alcohol in the home cage to induce satiety for that solution. Lever-press responding was then measured for 10min in the operant chamber under extinction conditions (no consequences of lever presses). To control for drinking a bolus of liquid before the session, rats were given 1h access to 2% maltodextrin (w/v) before an identical extinction test on a separate day (balanced order, 15mL maximum). The two devaluation test days were separated by 2-3 days of maintenance training on the standard FR5 or VI30 reinforcement schedules.
As a second test of behavioral flexibility, contingency degradation training was used to determine the persistence of behavior after complete disruption of action-outcome contingencies (Colwill and Rescorla, 1986; Balleine and Dickinson, 1998). First, a 10min extinction test was performed to assess a baseline level of responding. After two additional standard operant self-administration sessions, rats underwent three sessions of contingency-degradation training in which the reinforcing solution was delivered into the cup on a random time 30s schedule for 40min, resulting in 51±3 alcohol deliveries. The day after degradation training, the effect of contingency degradation on responding was tested in a 10min extinction session. Reward seeking was compared in the pre- and post-training extinction sessions.
Histology
Rats were anesthetized with ≥1.5g/kg of urethane before 10µA current was applied for 5s to each stainless-steel wire, producing an iron deposit for determination of electrode placement. Rats were perfused, and brains were sectioned and stained as previously described (Robinson and Carelli, 2008).
Data analysis
Description of the recording session
Operant session events are presented as mean ± S.E.M. In rare cases where not all delivered alcohol was consumed, alcohol consumption was calculated from the amount delivered and the amount remaining in the cup at the end of the session. To compare detection of cells in DMS and DLS, the number of MSN cells/wire in each rat was compared by Mann-Whitney U test (MWU; Sigma Plot, Systat Software Inc, San Jose, CA).
Neuronal firing at single events
The average baseline firing rate and coefficient of variance in the 60s before initiation of the operant session were calculated in 0.5s bins. Changes in firing rate at the presentation of cues signaling the start of the operant session (house-light illumination, initial lever extension) were determined by comparison of neuronal firing in the 0.5s bin after the cue to the previous 60s (0.5s signal: 60s baseline firing rate ratio; S:B). The baseline firing rate, coefficient of variance, and signal-to-baseline ratios were compared across regions (DMS, DLS) by MWU.
Perievent histograms of firing rates were created in NeuroExplorer software (Nex Technologies, Littleton, MA), and population analyses were completed using custom-written programs in MATLAB (MathWorks, Inc., Natick, MA, USA). To illustrate the activity of the population of neurons in the DMS versus DLS, the average firing rates of all neurons in each region were aligned and smoothed with a moving average of 250ms in 50ms steps. Because baseline firing rates varied among neurons, the firing rates of each neuron around intra-session events were normalized before analysis by dividing by the average firing rate across the whole session. This normalization better represented changes in neuronal response magnitude. For further analysis, neurons were split into anterior and posterior positions (1.2 – 2.2mm and 0.2 – 1.2mm relative to bregma, respectively).
Neuronal firing around repeated intra-session events
Spike rates from each neuron around the times of lever responses and cue events were averaged across trials before population analyses were conducted as for single events. There were typically many more non-reinforced than reinforced lever responses under the VI30 reinforcement schedule; thus, to facilitate comparison of neuronal activity, we selected 25 non-reinforced responses that were evenly distributed throughout the session for these analyses.
The firing activity of individual neurons around events that occurred multiple times within each session was classified by calculating z-scores of phasic frequency changes from baseline. For lever-response events, the average number of spikes in two target windows—the 0.5s before and the 0.5s after each event—was compared to a nearby 2s window that was designated as baseline. In Experiment 1 (FR5 model), the baseline was 2-4s before the 1st response and 8-10s after the 5th response in each series, to compare changes before and after these responses to a baseline outside of the action sequence. In Experiment 2 (VI30 model), baselines were 2-4s before non-reinforced and reinforced responses; as lever responses did not typically occur in bouts, this baseline rarely overlapped with behavioral responding. Neurons with z-scores between −2 and 2 were classified as non-phasic (NP). Those with significant z-scores (-2 > z > 2) were classified by the epoch and direction of greatest change as pre-excitatory (PreEx), pre-inhibitory (PreIn), post-excitatory (PostEx), or post-inhibitory (PostIn). In Experiment 1, we also analyzed activity after the lever extension terminating each time-out as a cue of alcohol availability. For this event, the baseline was set as the 2s immediately preceding lever extension, the 0.5s window after the event was analyzed, and cells were classified as PostEx or PostIn.
Behavioral tests
Lever responding during satiety-specific devaluation and contingency-degradation extinction tests was compared within-subjects using a paired t-test. Responding during contingency-degradation training was compared using 1-way RM ANOVA with the Tukey method for multiple comparisons (Sigma Plot).
Results
Experiment 1: Alcohol self-administration with fixed-ratio reinforcement
Twenty-four rats underwent surgery, and 14 completed the protocol for Experiment 1. Rats were trained in 36.5±1.5 sessions to self-administer 10% alcohol on a FR5-reinforcement schedule. On the electrophysiological recording day, rats responded on the active lever 110±4 times, receiving 22±1 alcohol deliveries; inactive lever responses occurred 37±7 times. The average total alcohol consumption was 0.5±0.02g/kg, similar to doses previously reported for a 30min session (Rassnick et al., 1992; Hodge et al., 1997; Robinson and Carelli, 2008). We recorded 101 neurons that were confirmed to be in the DMS or DLS (Figure 1) with firing rates ≤10Hz. Rates of detection of these presumed MSNs were similar in the two regions: 0.54±0.10 cells/wire in the DMS and 0.52±0.08 cells/wire in the DLS (MWU statistic =96.5, P>0.05).
Examples of self-administration behavior and MSN firing patterns from a single FR5-trained rat are shown in Figure 2. The FR5 schedule produced a fixed contingency between the number of lever responses and reinforcer deliveries (Figure 2A). The biphasic distribution of inter-response intervals (IRIs, Figure 2B) demonstrates the fast IRIs exhibited within the 5-response sequence as well as the longer IRIs imposed by the 12s time-out period. The mean IRI on the active lever for the FR5 rat shown here was 9.3±1.0s, and across all FR5-trained rats mean IRI was 8.7±0.8s. Firing rates of each cell were examined by aligning action potentials around operant events, such as reinforcement at the 5th lever response of a sequence, shown here (Figure 2C). In these examples, blue triangles indicate lever extension, pink diamonds indicate the 1st lever response, and green triangles indicate the 5th response and reinforcement (alcohol delivery and associated cues). This DMS neuron exhibited increased firing immediately following alcohol delivery. In contrast, the DLS neuron displayed higher firing rates during lever responding than at reinforcement. For this rat, the mean latency from alcohol delivery to start of drinking was 1.2±0.1s and drinking duration was 4.9±0.3s, which corresponded with periods of low firing frequencies in the representative DMS and DLS example cells and a rebound in firing in the DLS neuron at drinking cessation.
Neuronal population activity in DMS versus DLS during FR5 sessions
We first analyzed differences in the basal firing rates and coefficients of variance of DMS and DLS neurons at the start of the session in the 60s before house-light illumination: basal firing rates were 2.1±0.2 in the DMS and 2.4±0.3 in the DLS (MWU statistic =1174, P>0.05), while the coefficient of variance was 7.9±0.5 in the DMS and 7.9±0.4 in the DLS (MWU statistic =1201, P>0.05). Therefore, no significant baseline firing differences were found between these regions.
Next, we compared neuronal activation to conditioned cues signaling the start of the session by plotting the average normalized firing rate of all cells in the DMS and DLS. These population plots showed that the average DMS firing rate increased 4.1-fold compared to the whole session firing rate within 0.25s of the house-light illumination (Figure 3A). Similarly, DMS firing exhibited a brief 3.6-fold increase relative to the whole session firing rate immediately after the first lever extension (Figure 3B), demonstrating sensitivity to cues of session initiation that were independent of behavior. Comparison of the firing rate in the 0.5s after house-light illumination to the 60s basal firing rate (described above) revealed a significantly greater signal-to-baseline ratio in the DMS (3.4±0.9) compared to the DLS (1.0±0.2; MWU statistic =965, P<0.05). Similarly, after the initial lever extension, the signal-to-baseline ratio was significantly larger in the DMS versus the DLS (2.2±0.3 and 1.3±0.2, respectively; MWU statistic =980, P<0.05).
We additionally examined events that occurred repeatedly during the self-administration session, including the lever extension cue (12s after each fluid delivery), the 1st lever response in the 5-press sequence, and the combined action, cue and alcohol delivery around the 5th response. When neuronal activity was aligned to all lever extensions in the session, a brief 1.8-fold increase in DMS spike frequency was observed in the population (Figure 3C). Around the 1st response, however, the DMS showed no apparent change in the population firing rate, while firing increased 2-fold following reinforcement (Figures 3D & 3E). This phasic neuronal activation to the reinforced press was most prominent in the first half of the trials in the session (Supplemental Figure 1). Similar to the first press activation, no changes were observed in the DMS around inactive lever presses (data not shown). Thus, DMS neurons were most active at alcohol-associated cues of availability and delivery rather than initiation of alcohol-seeking behavior.
Neuronal activity in the DLS differed from that of the DMS around these events. Lever extensions evoked a 1.6-fold elevation in firing rate with a more prolonged DLS activation than was observed in the DMS; this excitation often encompassed the 1st lever response, as the median interval between the lever extension and the 1st response was 2.3s (Figure 3C). The 1st response itself was associated with a higher amplitude 2.5-fold increase in firing frequency in the DLS that peaked before the lever-press response (Figure 3D). Moreover, the DLS excitation after the 1st response persisted during the 5-response sequence but not after alcohol delivery (Figure 3E); indeed, the median interval between the 1st and 5th responses was 2.5s. Pre-response DLS excitations (2-fold increase in firing rates) were also observed before the few inactive lever responses (data not shown). When the first half of trials was compared to the second, there was some increase in excitation across the session (Supplemental Figure 1). Thus, the predominant response of MSNs in the DLS was a pre-response excitation. Finally, both DMS and DLS demonstrated decreased firing rates after the fluid delivery, extending previous findings of inhibition of MSNs in the nucleus accumbens during reward consumption (Taha and Fields, 2005; Taha and Fields, 2006; Krause et al., 2010) to the dorsal striatum.
As the electrode arrays were positioned in anterior-to-posterior rows, we compared population activity in those neurons anterior and those posterior to 1.2mm bregma (Figures 3F–3J). This analysis revealed that both anterior and posterior DMS neurons contributed to the DMS activation after house-light illumination, while anterior DMS neurons showed the predominant population changes in firing frequency after the 1st lever extension and the 5th lever response. In contrast, the doubling of DLS firing rates after repeated intra-session lever extensions seen in the entire DLS population was driven selectively by posterior DLS neurons. Similarly, the pre-response excitations around both the 1st and 5th lever responses were primarily driven by posterior neurons.
Firing patterns of individual neurons in DMS versus DLS around FR5 intra-session events
To determine the proportion of MSNs in each striatal region that exhibited particular phasic firing patterns, we classified individual neurons by their firing activity at repeated intra-session events: lever extension, 1st response and 5th response. Specifically, z-scores were used to compare normalized firing rates in the 0.5s bin after lever extension, before lever response, or after lever response to a 2s baseline depicted by the shaded area in Figure 4, and neuronal activity of the phasically active cells was plotted (nonphasic cells were excluded for clarity). The distribution of neurons across each category of neuronal activity (PreEx: pre-excitatory; PreIn: Pre-inhibitory; PostEx: post-excitatory; PostIn: post-inhibitory; NP: non-phasic) and the proportion of non-phasic neurons are displayed in pie charts on each graph. When we examined changes in firing after intra-session lever extensions, we found only PostEx phasic patterns in the DMS, comprising 28% (15/53) of the recorded neurons (Figure 4A). Similar to the DMS, 30% of DLS neurons displayed significantly different firing frequencies after the lever extension, although only 17% (8/48) of DLS neurons were PostEx while 13% (6/48) were PostIn (Figure 4B).
Around the 1st lever response of the 5-response sequence, 44% of DMS neurons (23/53) and 65% of DLS neurons (31/48) exhibited significant changes in firing rates. All firing patterns were observed in the DMS and DLS at the 1st response, but the regions differed in the proportions of neurons demonstrating each category of neuronal activity. Consistent with the population frequency plots, excitations were prominent in the DLS, where 21% of cells (10/48) exhibited a brief, 5-fold PreEx firing activity and 27% of cells (13/48) showed a more sustained PostEx pattern (Figure 4D). In contrast, the predominant phasic activity in the DMS was PostIn (11/53 neurons, or 21%; Figure 4C). Nevertheless, a subset of DMS neurons (6/53) exhibited PostEx activity that showed similar timing and amplitude to the DLS PostEx pattern. Overall, the DLS had a higher proportion of phasically active cells around the 1st response and these cells showed excitations time-locked to the lever response event, while the DMS had fewer phasically active cells and these tended to exhbit inhibitions that were less closely time-locked to the action.
When aligned to the 5th response, which initiated cue onset and alcohol delivery, we found that 67% of DMS and 64% of DLS cells demonstrated significantly altered firing rates. The highest magnitude of frequency change observed after categorization in either region was the PostEx activity after the 5th response in the DMS, with 12 of 53 cells (23%) reaching on average 5.6 times their whole-session firing rate (Figure 4E). This change was brief and time-locked to the reinforced lever-press response. In contrast, the predominant neuronal firing pattern in the DLS was a prolonged, 3- to 4-fold PreEx pattern exhibited by 17 of 48 neurons (35%; Figure 4F). This DLS excitation appeared to be a continuation of the firing activity that began at lever-response initiation and continued through the 5-response sequence; indeed, 24 of the 29 DLS neurons that were significantly excited ±0.5s around the 5th response also showed significant excitations ±0.5s around the 1st response. However, a small subset of DLS 5th response PostEx neurons (4/48) displayed brief excitations whose timing matched the DMS PostEx neurons, although with diminished amplitude.
Thus, while similar specific firing patterns were observed in the DMS and DLS, these regions differed in the proportions of neurons displaying these patterns. The DMS exhibited less phasic activity around the 1st lever response but distinct excitation after the 5th response and lever extension, while the DLS excitations appeared to persist throughout the action sequence.
Experiment 2: Alcohol self-administration with variable interval reinforcement
In Experiment 2, 21 rats underwent surgery and 16 rats completed successful electrophysiological recordings. These rats were trained to self-administer 10% alcohol over 35.3±1.5 weeks on a VI30-reinforcement schedule. On the electrophysiological recording day, rats responded on the active lever 117±16 times for 22±1 alcohol reinforcements, resulting in average total alcohol consumption of 0.5±0.02g/kg; inactive lever responses occurred 1±1 times. During these sessions, we recorded 102 neurons confirmed to be in the DMS or DLS with firing rates ≤10Hz (Figure 1). Detection rates were 0.61±0.10 presumed MSNs/wire in the DMS and 0.42±0.07 MSNs/wire in the DLS (MWU statistic =86, P>0.05), for a total of 61 DMS and 41 DLS neurons.
Examples of self-administration behavior and MSN firing patterns from a single VI30-trained rat are shown in Figure 5. Notably, alcohol delivery in the VI30 reinforcement schedule is less contingent on the rate of lever responding, as illustrated by the divergence of the cumulative activity plots in Figure 5A. The IRI for this rat was 9.7±1.1s, while the mean IRI in Experiment 2 was 11.3±0.7s. Compared to the FR5 schedule, the VI30-reinforcement schedule generated slower lever-press behavior, with a smoother IRI distribution (Figure 5B). Neuronal firing rates were examined by aligning action potentials around operant events, such as the firing around the reinforced response in the representative cells in Figure 5C. The DMS neuron displayed here demonstrated increased firing rates after reinforced responses, while the predominant change in the DLS cell was an excitation before reinforced lever-press responses. Both cells were less active during drinking periods, and the DMS cell exhibited a rebound excitation after drinking, which was initiated 0.8±0.03s after alcohol delivery and was sustained for the following 6.4±0.3s during the session shown here.
Population neuronal activity in DMS versus DLS during VI30 sessions
To analyze whether there were differences in the basal firing of each region, we compared spike frequency in the 60s before house-light illumination at the start of the session. Before normalization, the basal firing rate was 2.9±0.3 in the DMS and 2.3±0.2 in the DLS (MWU statistic =1131, P>0.05). The coefficient of variance was 5.1±0.4 in the DMS and 5.6±0.7 in the DLS (MWU statistic =1129, P>0.05). There were no significant differences in the basal firing or coefficient of variance between the DMS and DLS.
We expected that lever-press responses and the appearance of cues and reinforcers would be encoded by fluctuations in the firing rates of MSNs, but that there would be less of this phasic activity in the DMS of VI30-trained rats compared to FR5-trained rats. Therefore, we compared the population activity in the DMS and DLS around session-initiation cues, as well as non-reinforced and reinforced responses. Specifically, we analyzed average, normalized neuronal firing rates by aligning spike timing to either session-initiation cues, reinforced responses or 25 non-reinforced responses that were evenly distributed throughout the session. (Repeated lever extensions were not present in this model due to the lack of a time-out period.) Figures 6A & 6B illustrate that population activity in the DMS increased to around three times its whole-session firing rate after the session-initiation cues of house light illumination and initial lever extension, while the DLS showed a more modest increase to both cues that was less time-locked. However, no significant difference in the signal-to-baseline ratio of these regions was discovered when the ratio of the firing rate in the 0.5s after either event was compared to the 60s basal firing rate (1.3±0.2 in the DMS and 1.2±0.3 in the DLS at the house light, MWU statistic =1112, P>0.05; 1.5±0.5 in the DMS and 1.4±0.3 in the DLS at the 1st lever; MWU statistic =1141, P>0.05).
Around lever-press responses, a brief 1.8-fold increase over the baseline firing rate was observed in the DMS immediately after reinforced responses but not after non-reinforced responses, consistent with an association of neuronal activity to cues and alcohol delivery rather than lever responses per se (Figures 6C & 6D). The DLS showed a modest ramping of firing rate leading up to either type of lever response that peaked at approximately 1.5-fold increase over baseline before returning to basal rates. Inactive lever presses were too infrequent for analysis. Firing rate changes were larger in the first half of trials in the DMS, with no consistent change in DLS activation (Supplemental Figure 2). Finally, the population activities of both the DMS and DLS were diminished in the seconds following alcohol delivery, consistent with drinking-associated inhibition, as observed in Experiment 1. Overall, phasic firing patterns in both DMS and DLS neuronal populations at all events were smaller in amplitude than those in FR5-trained rats.
Dividing the neurons along the anterior-posterior axis (as in Experiment 1) revealed a higher amplitude response in the posterior DMS at the house-light cue and in the anterior DMS at the 1st lever extension (Figures 6E & 6F). Moreover, while the delayed excitation to house-light illumination was expressed across the DLS, the broad excitations that spanned ±1s around the initial lever extension were driven by posterior DLS neurons. At reinforced responses, DMS excitations were predominantly in the posterior cells (Figure 6H). The posterior DLS cells also exhibited brief increases in firing rates after reinforced responses, although this activity was delayed by 0.5s relative to the DMS excitation, similar to the DLS response to the house-light cue. However, the anterior-posterior analysis did not reveal sub-regional variation in the discharge activity at non-reinforced responses or during drinking (Figures 6G & 6H).
Individual neuronal firing patterns around VI30 lever response
To assess the distribution of phasic firing patters at reinforced versus non-reinforced responses, we categorized individual cells by their firing activity (Figure 7). Again, all firing patterns were observed in both DMS and DLS after non-reinforced and reinforced responses, but their proportions varied by region at each event. At non-reinforced responses, 55% of DMS cells exhibited significant changes in firing rates, including PreEx (10/61 cells), PostEx (13/61 cells), and PostIn (9/61 cells). In the DLS, the total proportion of phasically active cells was also 55%, but the predominant activity pattern was PreEx at 29% (12/41 cells), consistent with population activity. The largest amplitude of firing frequency change at non-reinforced responses in the DMS was a 2-fold change in PostEx cells, while the DLS peak phasic firing activity within 0.5s of the lever response was a 2.4-fold increase in PreEx cells.
More neurons exhibited significant changes in discharge rates around the reinforced responses as compared to non-reinforced responses: 65% of DMS and 68% of DLS cells. In the DMS (Figure 7C), 34% of neurons (21/61) were classified as PostEx and exhibited a 3.4-fold increase in firing rate. Interestingly, similar proportions of all other categories of neuronal activity were seen in the DMS around non-reinforced as reinforced responses, suggesting these DMS neurons encoded both lever-press responding and reinforcement-associated events in the VI30 model. In the DLS (Figure 7D), firing patterns included PreEx (9/41), PostEx (7/41) and PostIn (10/41). The PostEx activity in the DLS was less robust than in the DMS, with half the percentage of neurons classified as PostEx, but with similar amplitude and timing. Thus, lever responses and reinforcement were encoded in both regions, albeit with variable activity patterns. Again, the predominant pre-response DLS excitation and post-cue excitation in the DMS were smaller in magnitude in VI30-trained versus FR5-trained rats.
Satiety-specific devaluation and contingency degradation testing
Once all electrophysiological recordings were completed, satiety-specific devaluation and contingency degradation were used on rats with stable lever-press behavior to test whether behavior was goal-directed and dependent on action-outcome associations or habit-like and controlled by stimulus-response associations. First, satiety-specific devaluation of 10% alcohol tested whether alcohol-seeking behavior (i.e., lever responses) was reduced by 1h of home-cage access to 10% alcohol compared to access to a control fluid (2% maltodextrin). If pre-exposure and satiety for alcohol resulted in less lever responding during extinction versus pre-exposure to the control solution, the rat was considered goal-directed and sensitive to changes in reward value. Rats consumed 6.6±0.5mL of alcohol (4.62±0.35kcal; for a dose of 1.3±0.1g/kg) or 8.4±0.7mL (0.67±0.056kcal) of maltodextrin before a 10min extinction session (no cues or alcohol deliveries). Lever responses were compared between the two extinction sessions (Table 1): paired t-tests demonstrated that FR5-trained rats decreased responding by 43% in the extinction session after alcohol pre-access compared to maltodextrin pre-access (t12=2.58, P<0.05). Similarly, VI30-trained rats decreased pressing after devaluation by 34% (t14=2.52, P<0.05). Thus, rats under both training schedules demonstrated alcohol-seeking behavior that was sensitive to satiety-specific devaluation.
Table 1.
Satiety-specific devaluation test | Experiment 1 FR5, n=13 |
Experiment 2 VI30, n=15 |
---|---|---|
2% maltodextrin | 105±19 | 40±5 |
10% ethanol | 59±13a | 26±5a |
Contingency degradation training |
Experiment 1 FR5, n=12 |
Experiment 2 VI30, n=12 |
Day 1 | 134±34 | 182±28 |
Day 2 | 78±18 | 147±34 |
Day 3 | 58±16 b | 93±22 b |
Contingency degradation test |
Experiment 1 FR5, n=12 |
Experiment 2 VI30, n=12 |
Pre-degradation | 104±18 | 46±8 |
Post-degradation | 28±5 c | 35±6 |
significant effect of pre-access solution on extinction, paired t-test, P<0.05
significant effect of training day, 1-way RM ANOVA, P<0.05
significant effect of degradation training on extinction, paired t-test, P<0.05
The outcome of the satiety-specific devaluation test may have been affected by transferring the rats back to their training chambers (disrupting a habitual response), greater caloric content of the alcohol versus the control solution, or by the intoxicating effects of alcohol (slowing lever-press behavior). Thus, we used an additional test to distinguish goal-directed from habit-like behavior – contingency degradation training – that tested behavior in the absence of alcohol. We compared the number of responses during brief extinction sessions before versus after three sessions of contingency degradation training (fluid deliveries were made on a random time 30s schedule, independent of lever responses). Fewer lever responses in the post-training session would indicate goal-directed behavior that was sensitive to changes in reward contingency. Both FR5-trained and VI30-trained rats decreased responding during the contingency degradation sessions when feedback was provided (Table 1; 1-way RM ANOVA, FR5: F2,22=4.28, P<0.05; VI30: F2,22=7.85, P<0.01). This decrease in response rate persisted into the post-training test only in FR5-trained rats, demonstrating sensitivity to prior conditioning (t11=4.28, P<0.001). VI30-trained rats showed no significant effect of degradation training on alcohol seeking during extinction (t11=1.95, P>0.05). VI30 rats responded more slowly than FR5 rats in all conditions; while this difference is well known and expected (Dickinson, 1985; Hilario et al., 2007; Mangieri et al, 2012; Hay et al., 2013), a floor effect cannot be ruled out. Nevertheless, contingency degradation indicated that behavior remained goal-directed in FR5-trained rats after all self-administration sessions, as expected (Yin, 2006), while VI30-trained rats demonstrated less flexible operant behavior, specifically in response to changing action-outcome contingencies.
Discussion
Maladaptive stimulus-response learning resulting in habit formation likely contributes to the persistent drinking and susceptibility to relapse that characterize alcoholism (Everitt and Robbins, 2005). Previous research suggests that alcohol self-administration in rats can become habitual and resistant to changes in reward value, and alcohol exposure may facilitate habit formation (Dickinson et al., 2002; Corbit et al., 2012; Mangieri et al., 2012). However, the specificity of neuronal activity in dorsal striatum engaged by alcohol self-administration had not previously been investigated. Thus, the present study recorded neuronal activity in DMS and DLS in rats trained on one of two operant reinforcement schedules that produce distinct behavioral patterns (response sequences versus single responses) and differences in behavioral flexibility (differential sensitivity to contingency degradation). A major finding was that the predominant phasic firing patterns of each region occurred in response to distinct events: excitations in the DMS were largely time-locked to alcohol delivery and alcohol-predictive cues, while DLS excitations primarily occurred prior to lever responses. Indeed, these regional specificities were observed in both behavioral models despite distinct alcohol-seeking patterns. Parallels with previous studies of dorsal striatal activation during instrumental behavior, discussed below, suggest common processing for alcohol and non-drug rewards. We additionally report novel evidence of differential encoding of conditioned cues in the two operant models. In the VI30-habit model, the DMS and DLS neurons exhibited more similar response patterns and the population response amplitudes were reduced as compared to the neuronal activity observed in FR5-trained rats. Moreover, putative MSNs in the DMS that responded to alcohol delivery and associated cues were more anterior in FR5-trained rats and more posterior in VI30-trained rats. These findings provide evidence that differential dorsal striatal encoding of alcohol-conditioned behavior accompanies differences in response contingencies that affect behavioral flexibility.
DMS activation to cues in two models of alcohol self-administration
Alcohol-associated cues are known to promote alcohol-seeking behavior (Epstein et al., 2006; Corbit and Janak, 2007). We observed higher amplitude excitations to alcohol-associated cues in the DMS versus DLS of FR5- and VI30-trained rats, consistent with our hypothesis based on the region’s associative connectivity, including reciprocal connections to the prefrontal cortex and midbrain dopamine neurons (McGeorge and Faull, 1989; Haber et al., 2000; Voorn et al., 2004). Our findings additionally agree with previous neurophysiological studies that found greater percentages of DMS/caudate than DLS/putamen neurons are activated by reinforcement-related stimuli (West et al., 1990; Carelli and West, 1991; White and Rebec, 1993; Rolls, 1994). The present study extends these observations to alcohol self-administration, suggesting that this is a common response to drug and non-drug rewards, although results of ongoing research will be necessary to directly compare alcoholic and non-alcoholic self-administration in the same model. The attenuation in both experiments of phasic DMS activation at reinforcement in later trials indicates that some aspect of neuronal encoding of reinforcement changes within session, such as reward value (satiety). In contrast, DMS activation time-locked to the lever-extension cue was undiminished across the session, arguing against alternative explanations such as decreased general arousal or a pharmacological effect of increasing alcohol concentrations.
Also in agreement with the expectation that FR5-trained rats would show predominant DMS activation, DMS excitation to start-of-session and reinforcement cues was of greater amplitude in rats on the FR5 versus the VI30 schedule. The diminished amplitude of neuronal firing patterns, accompanied by a greater proportion of neurons activated by cues in the VI30-trained rats, may be subsequent to habit formation in that group. However, the present study is limited due to its between-subjects design, and the differences in operant behavior necessitate caution in direct comparison of neural data from FR5 and VI30 schedules. An alternative explanation is that the reduced magnitude of the phasic firing patterns in the VI30-trained rats is directly associated with decreased expectancy of reinforcer and subsequently decreased arousal, which may be integral to the habit-promoting nature of the schedule. However, this explanation does not necessarily account for the diminished DMS firing to start-of-session cues, as they predicted alcohol availability equally between the two groups.
DLS activation to lever responses during alcohol self-administration
The activity of the DLS at lever responses is consistent with DLS connectivity to sensorimotor cortex and DLS encoding of specific motor actions, including forelimb movements required for lever responses (West et al., 1990). Notably, the DLS population excitation associated with lever responses was of higher amplitude in FR5- versus VI30-trained rats, in contradiction to our hypothesis. While this may be due to reduced goal-directed behavior under the VI30 schedule, a more parsimonious explanation is the differential response requirements: the FR5 schedule required 5 lever responses for each reinforcer delivery, while the VI30 model required a single response after a time delay. Moreover, Jin and Costa (2010) showed that MSN phasic activity encoding the start and stop of an FR8 response sequence emerged with learning, particularly in the DLS and to a lesser extent in the DMS; this is consistent with DLS population activity at the 1st and 5th responses observed in Experiment 1. Additionally, DLS activation was associated with inactive lever responses as well as 2nd, 3rd, and 4th active lever responses, though these correlates could not be isolated due to their temporal proximity to other responses. Thus, phasic firing patterns in the DLS associated with operant responses may be common to drug and non-drug rewards.
Studies from West and colleagues have shown diminished DLS phasic activation during a motor task (Carelli et al., 1997; Tang et al., 2007; Tang et al., 2009), which is apparently associated with more efficient task performance rather than habit formation (Tang et al., 2009). In contrast, Kimchi and colleagues demonstrated elevated proportions of phasically active DLS neurons during the development of habitual responding (Kimchi et al., 2009). An advantage of the present study was that rats in the goal-directed and habitual models had the same amount of instrumental training, and we did not observe a greater proportion of phasic DLS neurons in the VI30 model overall.
Functional gradient from anterior DMS to posterior DLS
Previous studies observed an anterior-medial to posterior-lateral gradient of behavioral plasticity (Miyachi et al., 1997; Corbit and Janak, 2010) and support the hypothesis that striatal control of behavior shifts from DMS to DLS with habit formation, but less is known regarding subregional shifts within the DMS. Goal-directed, action-outcome behavioral control clearly depends on the posterior DMS (e.g., Yin et al., 2005; Corbit and Janak, 2010), and consistent with this finding, we observed phasic activation of relatively posterior DMS neurons to alcohol-associated cues in FR5-trained rats. In contrast, the role of the anterior DMS is less clear. Yin et al. (2005) found that permanent, pre-training lesions of the anterior DMS did not disrupt goal-directed behavioral control when tested after 8 days of training, while Corbit and Janak (2010) disrupted goal-directed behavior with repeated, acute inactivation of the anterior DMS over 3 days of training. Furthermore, post-training, pre-test inactivation of the anterior DMS disrupted goal-directed behavior after 2 weeks of training (Corbit et al., 2012). While these apparently discrepant findings might simply be due to different lesioning techniques, another interpretation is that the anterior DMS is important for goal-directed behavioral control, but the posterior DMS can compensate for anterior DMS lesions if the rats are trained long enough in the absence of anterior DMS activity. In the present study, anterior DMS neurons displayed phasic firing patterns associated with start-of-session and reinforcement cues in FR5-trained, but not VI30-trained, rats. The persistent activation of the anterior DMS in the FR5-trained rats supports a role for this structure in goal-directed behavior that is sustained after extended training, while the lack of anterior DMS activation in VI30-trained rats suggests that anterior DMS contributions diminish with habit formation. Finally, posterior DMS activity was also apparent in the VI30-trained rats, and we suggest that this sustained activity may be related to the ability of habitually trained rats to exhibit goal-directed behavioral control after DLS lesions (Yin et al., 2004). Future studies monitoring neuronal activity across a broader anterior-posterior range of the DMS and throughout the duration of training are needed to fully interpret these results.
The DLS excitations we observed at lever responses were predominately posterior in both VI30- and FR5-trained rats. The more posterior DLS electrodes overlapped with areas that have previously been shown to inhibit habitual behavior when lesioned (Yin et al., 2004), indicating that our electrodes were placed in a region linked to habit control. Moreover, the fact that the phasic firing patterns were similarly positioned in both reinforcement models suggests that the neural activity is related to well-learned motor responses (Miyachi et al., 1997) that may not manifest as habits during goal-directed behavior (e.g., in FR5-trained rats), but may be expressed as habits when DMS activity is reduced due to lesion (Yin et al., 2005, Corbit and Janak, 2010). One intriguing possibility is that alcohol itself may facilitate DLS activity, or more broadly, a shift from anterior DMS to posterior DLS neuronal activation, as alcohol exposure can promote habit formation (Corbit et al., 2012). We are currently addressing this possibility by comparison of groups self-administering alcoholic and non-alcoholic rewards.
VI schedule of reinforcement reduces functional heterogeneity in the dorsal striatum
Although population activity differed between DMS and DLS at operant events, we found that firing patterns of individual neurons overlapped between the regions. Interestingly, the 1st-response activity in the FR5 model showed the most discrepancy in activation between striatal regions, with more phasic neurons in DLS (65%) than in DMS (44%; compared to 55% in each region in the VI30 model). We observed a greater degree of overlap in DMS and DLS firing patterns in the VI30 model, where twice as many DMS cells exhibited pre-excitatory activation to non-reinforced responses (characteristic response of DLS) and DLS cells displayed brief post-reinforcement excitation (characteristic response of DMS) in the VI30- as in the FR5-trained rats. This finding may reflect increasing involvement of the DLS with development of habits or well-learned behaviors observed in other studies (Yin et al., 2009; Kimchi et al., 2009), and it extends these studies to include persistent involvement of DMS neurons.
The present data extend our knowledge of neuronal encoding during alcohol seeking by revealing regionally specific activity in dorsal striatum during two alcohol self-administration models that differ in behavioral flexibility. Future studies can address whether alcohol accentuates habit-related response patterns that are common to drug and non-drug rewards, and extend the correlative measurements reported here to mechanistic by using local pharmacology or optogenetic manipulations to disrupt regional phasic firing patterns. Human-subject studies confirm that differential activation of striatal regions accompany different aspects of reward learning and habit expression (Jenkins et al., 1994; Tricomi et al., 2009; Vollstadt-Klein et al., 2010) as well as response to alcohol-associated cues (Filbey et al., 2008). Dorsal striatal signaling, thus, is important for understanding the processes involved in reward-related learning and, by extension, addiction.
Supplementary Material
Acknowledgements
The authors thank Hung-Yu Chen, Ian Everitt, Joel Shillinglaw, Vicki Burton, Lisa Bowen, and especially Dawnya Zitzman for their excellent technical assistance. Also thanks to Dr.s Joyce Besheer, Eric Fish, and Tatiana Shnitko for critical comments on the manuscript. This research was funded by NIH (R01AA018008 to D.L.R.; T32NS007431, to R.R.F., T32AA007573 to J.T.K. and R.R.F.), ABMRF/The Foundation for Alcohol Research, and the UNC Bowles Center for Alcohol Studies. R.R.F. is a Howard Hughes Medical Institute (HHMI) Med into Grad Scholar supported in part by a grant to the University of North Carolina from HHMI through the Med into Grad Initiative.
Abbreviations
- DLS
dorsolateral striatum
- DMS
dorsomedial striatum
- FR
fixed ratio
- IRI
inter-response interval
- MSNs
medium spiny neurons
- MWU
Mann-Whitney U test
- NP
non-phasic
- PreEx
pre-excitatory
- PreIn
Pre-inhibitory
- PostEx
post-excitatory
- PostIn
post-inhibitory
- VI
variable interval
Footnotes
Conflict of interest:
The authors declare no competing financial interests.
References
- Adams CD, Dickinson A. Instrumental responding following reinforcer devaluation. Q. J. Exp. Psychol. B. 1981;33b:109–121. [Google Scholar]
- Alexander GE, Crutcher MD. Functional architecture of basal ganglia circuits: Neural substrates of parallel processing. Trends Neurosci. 1990;13:266–271. doi: 10.1016/0166-2236(90)90107-l. [DOI] [PubMed] [Google Scholar]
- Balleine BW, Dickinson A. Goal-directed instrumental action: Contingency and incentive learning and their cortical substrates. Neuropharmacology. 1998;37:407–419. doi: 10.1016/s0028-3908(98)00033-1. [DOI] [PubMed] [Google Scholar]
- Belin D, Jonkman S, Dickinson A, Robbins TW, Everitt BJ. Parallel and interactive learning processes within the basal ganglia: Relevance for the understanding of addiction. Behav. Brain. Res. 2009;199:89–102. doi: 10.1016/j.bbr.2008.09.027. [DOI] [PubMed] [Google Scholar]
- Carelli RM. The nucleus accumbens and reward: Neurophysiological investigations in behaving animals. Behav. Cogn. Neurosci. Rev. 2002;1:281–296. doi: 10.1177/1534582302238338. [DOI] [PubMed] [Google Scholar]
- Carelli RM, Wolske M, West MO. Loss of lever press-related firing of rat striatal forelimb neurons after repeated sessions in a lever pressing task. J. Neurosci. 1997;17:1804–1814. doi: 10.1523/JNEUROSCI.17-05-01804.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carelli RM, West MO. Representation of the body by single neurons in the dorsolateral striatum of the awake, unrestrained rat. J. Comp. Neurol. 1991;309:231–249. doi: 10.1002/cne.903090205. [DOI] [PubMed] [Google Scholar]
- Colwill RM, Rescorla RA. The Psychology of Learning and Motivation. In: Bower G, editor. The psychology of learning and motivation. New York: Academic; 1986. pp. 55–105. [Google Scholar]
- Corbit LH, Nie H, Janak PH. Habitual alcohol seeking: Time course and the contribution of subregions of the dorsal striatum. Biol. Psychiatry. 2012;72(5):389–395. doi: 10.1016/j.biopsych.2012.02.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Corbit LH, Janak PH. Posterior dorsomedial striatum is critical for both selective instrumental and pavlovian reward learning. Eur. J. Neurosci. 2010;31:1312–1321. doi: 10.1111/j.1460-9568.2010.07153.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Corbit LH, Janak PH. Ethanol-associated cues produce general pavlovian-instrumental transfer. Alcohol. Clin. Exp. Res. 2007;31:766–774. doi: 10.1111/j.1530-0277.2007.00359.x. [DOI] [PubMed] [Google Scholar]
- Dickinson A. Action and habits: The development of behavioral autonomy. Phil. Trans. R. Soc. Lond. B. 1985;308:67–78. [Google Scholar]
- Dickinson A, Wood N, Smith JW. Alcohol seeking by rats: Action or habit? Q. J. Exp. Psychol. B. 2002;55:331–348. doi: 10.1080/0272499024400016. [DOI] [PubMed] [Google Scholar]
- Epstein DH, Preston KL, Stewart J, Shaham Y. Toward a model of drug relapse: An assessment of the validity of the reinstatement procedure. Psychopharmacology (Berl) 2006;189:1–16. doi: 10.1007/s00213-006-0529-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Everitt BJ, Robbins TW. Neural systems of reinforcement for drug addiction: From actions to habits to compulsion. Nat. Neurosci. 2005;8:1481–1489. doi: 10.1038/nn1579. [DOI] [PubMed] [Google Scholar]
- Everitt BJ, Dickinson A, Robbins TW. The neuropsychological basis of addictive behaviour. Brain Res. Brain Res. Rev. 2001;36:129–138. doi: 10.1016/s0165-0173(01)00088-1. [DOI] [PubMed] [Google Scholar]
- Filbey FM, Claus E, Audette AR, Niculescu M, Banich MT, Tanabe J, Du YP, Hutchison KE. Exposure to the taste of alcohol elicits activation of the mesocorticolimbic neurocircuitry. Neuropsychopharmacology. 2008;33:1391–1401. doi: 10.1038/sj.npp.1301513. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haber SN, Fudge JL, McFarland NR. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 2000;20:2369–2382. doi: 10.1523/JNEUROSCI.20-06-02369.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hammond LJ. The effect of contingency upon the appetitive conditioning of free-operant behavior. J. Exp. Anal. Behav. 1980;34:297–304. doi: 10.1901/jeab.1980.34-297. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hay RA, Jennings JH, Zitzman DL, Hodge CW, Robinson DL. Specific and nonspecific effects of naltrexone on goal-directed and habitual models of alcohol seeking and drinking. Alcohol. Clin. Exp. Res. 2013 doi: 10.1111/acer.12081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hilario MR, Clouse E, Yin HH, Costa RM. Endocannabinoid signaling is critical for habit formation. Front. Integr. Neurosci. 2007;1:6. doi: 10.3389/neuro.07.006.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hodge CW, Samson HH, Chappelle AM. Alcohol self-administration: Further examination of the role of dopamine receptors in the nucleus accumbens. Alcohol. Clin. Exp. Res. 1997;21:1083–1091. doi: 10.1111/j.1530-0277.1997.tb04257.x. [DOI] [PubMed] [Google Scholar]
- Hyman SE. Addiction: A disease of learning and memory. Am. J. Psychiatry. 2005;162:1414–1422. doi: 10.1176/appi.ajp.162.8.1414. [DOI] [PubMed] [Google Scholar]
- Jenkins IH, Brooks DJ, Nixon PD, Frackowiak RS, Passingham RE. Motor sequence learning: A study with positron emission tomography. J. Neurosci. 1994;14:3775–3790. doi: 10.1523/JNEUROSCI.14-06-03775.1994. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jin X, Costa RM. Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature. 2010;466:457–462. doi: 10.1038/nature09263. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kimchi EY, Laubach M. Dynamic encoding of action selection by the medial striatum. J. Neurosci. 2009;29:3148–3159. doi: 10.1523/JNEUROSCI.5206-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kimchi EY, Torregrossa MM, Taylor JR, Laubach M. Neuronal correlates of instrumental learning in the dorsal striatum. J. Neurophysiol. 2009;102:475–489. doi: 10.1152/jn.00262.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kish LJ, Palmer MR, Gerhardt GA. Multiple single-unit recordings in the striatum of freely moving animals: Effects of apomorphine and D-amphetamine in normal and unilateral 6-hydroxydopamine-lesioned rats. Brain Res. 1999;833:58–70. doi: 10.1016/s0006-8993(99)01496-1. [DOI] [PubMed] [Google Scholar]
- Krause M, German PW, Taha SA, Fields HL. A pause in nucleus accumbens neuron firing is required to initiate and maintain feeding. J. Neurosci. 2010a;30:4746–4756. doi: 10.1523/JNEUROSCI.0197-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mangieri RA, Cofresi RU, Gonzales RA. Ethanol seeking by long evans rats is not always a goal-directed behavior. PLoS One. 2012;7:e42886. doi: 10.1371/journal.pone.0042886. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McGeorge AJ, Faull RL. The organization of the projection from the cerebral cortex to the striatum in the rat. Neuroscience. 1989;29:503–537. doi: 10.1016/0306-4522(89)90128-0. [DOI] [PubMed] [Google Scholar]
- Miyachi S, Hikosaka O, Miyashita K, Karadi Z, Rand MK. Differential roles of monkey striatum in learning of sequential hand movement. Exp. Brain Res. 1997;115:1–5. doi: 10.1007/pl00005669. [DOI] [PubMed] [Google Scholar]
- Paxinos G, Watson C. The Rat Brain in Stereotaxic Coordinates. San Diego, CA: Academic Press, Inc; 1998. [Google Scholar]
- Rassnick S, Pulvirenti L, Koob GF. Oral ethanol self-administration in rats is reduced by the administration of dopamine and glutamate receptor antagonists into the nucleus accumbens. Psychopharmacology (Berl) 1992;109:92–98. doi: 10.1007/BF02245485. [DOI] [PubMed] [Google Scholar]
- Robinson DL, Carelli RM. Distinct subsets of nucleus accumbens neurons encode operant responding for ethanol versus. water. Eur. J. Neurosci. 2008;28:1887–1894. doi: 10.1111/j.1460-9568.2008.06464.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rolls ET. Neurophysiology and cognitive functions of the striatum. Rev. Neurol.(Paris) 1994;150:648–660. [PubMed] [Google Scholar]
- Taha SA, Fields HL. Inhibitions of nucleus accumbens neurons encode a gating signal for reward-directed behavior. J. Neurosci. 2006;26:217–222. doi: 10.1523/JNEUROSCI.3227-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Taha SA, Fields HL. Encoding of palatability and appetitive behaviors by distinct neuronal populations in the nucleus accumbens. J. Neurosci. 2005;25:1193–1202. doi: 10.1523/JNEUROSCI.3975-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tang C, Pawlak AP, Prokopenko V, West MO. Changes in activity of the striatum during formation of a motor habit. Eur. J. Neurosci. 2007;25:1212–1227. doi: 10.1111/j.1460-9568.2007.05353.x. [DOI] [PubMed] [Google Scholar]
- Tang CC, Root DH, Duke DC, Zhu Y, Teixeria K, Ma S, Barker DJ, West MO. Decreased firing of striatal neurons related to licking during acquisition and overtraining of a licking task. J. Neurosci. 2009;29:13952–13961. doi: 10.1523/JNEUROSCI.2824-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tricomi E, Balleine BW, O'Doherty JP. A specific role for posterior dorsolateral striatum in human habit learning. Eur. J. Neurosci. 2009;29:2225–2232. doi: 10.1111/j.1460-9568.2009.06796.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vollstadt-Klein S, Wichert S, Rabinstein J, Buhler M, Klein O, Ende G, Hermann D, Mann K. Initial, habitual and compulsive alcohol use is characterized by a shift of cue processing from ventral to dorsal striatum. Addiction. 2010;105(10):1741–1749. doi: 10.1111/j.1360-0443.2010.03022.x. [DOI] [PubMed] [Google Scholar]
- Voorn P, Vanderschuren LJ, Groenewegen HJ, Robbins TW, Pennartz CM. Putting a spin on the dorsal-ventral divide of the striatum. Trends Neurosci. 2004;27:468–474. doi: 10.1016/j.tins.2004.06.006. [DOI] [PubMed] [Google Scholar]
- West MO, Carelli RM, Pomerantz M, Cohen SM, Gardner JP, Chapin JK, Woodward DJ. A region in the dorsolateral striatum of the rat exhibiting single-unit correlations with specific locomotor limb movements. J. Neurophysiol. 1990;64:1233–1246. doi: 10.1152/jn.1990.64.4.1233. [DOI] [PubMed] [Google Scholar]
- White IM, Rebec GV. Responses of rat striatal neurons during performance of a lever-release version of the conditioned avoidance response task. Brain Res. 1993;616:71–82. doi: 10.1016/0006-8993(93)90194-r. [DOI] [PubMed] [Google Scholar]
- Yin HH, Mulcare SP, Hilario MR, Clouse E, Holloway T, Davis MI, Hansson AC, Lovinger DM, Costa RM. Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat. Neurosci. 2009;12:333–341. doi: 10.1038/nn.2261. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yin HH, Knowlton BJ. The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 2006;7:464–476. doi: 10.1038/nrn1919. [DOI] [PubMed] [Google Scholar]
- Yin HH, Knowlton BJ, Balleine BW. Inactivation of dorsolateral striatum enhances sensitivity to changes in the action-outcome contingency in instrumental conditioning. Behav. Brain Res. 2006;166:189–196. doi: 10.1016/j.bbr.2005.07.012. [DOI] [PubMed] [Google Scholar]
- Yin HH, Ostlund SB, Knowlton BJ, Balleine BW. The role of the dorsomedial striatum in instrumental conditioning. Eur. J. Neurosci. 2005;22:513–523. doi: 10.1111/j.1460-9568.2005.04218.x. [DOI] [PubMed] [Google Scholar]
- Yin HH, Knowlton BJ, Balleine BW. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 2004;19:181–189. doi: 10.1111/j.1460-9568.2004.03095.x. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.