Control over a stressor involves the posterior dorsal striatum and the act/outcome circuit

Abstract

Controllable/escapable tailshocks (ESs) do not produce the behavioral and neurochemical outcomes produced by equal yoked uncontrollable/inescapable tailshocks (ISs). The prelimbic cortex is known to play a key role in mediating the protective effects of control. The concepts of act/outcome learning and control seem similar, and act/outcome learning is mediated by a circuit involving the prelimbic cortex and posterior dorsomedial striatum (DMS). Thus, we tested the involvement of the DMS in the protective effect of ES, in rats. First, we examined Fos immunoreactivity in both the DMS and dorsolateral striatum (DLS) after ES and yoked IS. We then investigated the effect of blocking DMS or DLS N-methyl-D-aspartate receptors with the specific antagonist D-(−)-2-amino-5-phosphopentanoic acid (D-AP5) on the release of dorsal raphe nucleus serotonin (5-HT) during ES, as well as on the level of anxiety produced by the ES experience 24 h later. ES, but not yoked IS, produced a large increase of Fos activity in the DMS. Consistent with the Fos data, D-AP5 in the DMS, but not in the DLS, prevented the inhibition of dorsal raphe nucleus 5-HT release normally produced by ES. Furthermore, D-AP5 administered into the DMS before ES, but not into the DLS, increased anxiety 24 h later, leading to levels similar to those produced by IS. These results suggest that, as with appetitive act/ outcome contingency learning, the protective effects of behavioral control over a stressor require the DMS.

Introduction

Stress exposure plays a role in the etiology of numerous disorders, but not all individuals are equally susceptible (Maier et al., 2006; Southwick and Charney, 2012). This has led to great interest in factors that modulate resistance to the impact of negative events (Russo et al., 2012; Christianson & Greenwood, 2014). Behavioral control over aversive stimuli is a potent determinant of the behavioral and physiological effects of such stimuli (Maier et al., 2006). The controllability of an aversive event is typically manipulated experimentally by exposing one group of subjects to tailshocks for which an action, such as turning a wheel, will terminate the shock [escapable shock (ES)]. In contrast, a second group of subjects is yoked [inescapable shock (IS)] to the ES group and cannot affect the termination of shock. Here, each shock terminates whenever the paired ES subject turns the wheel. Thus, the duration of the tail-shocks is identical for the two groups, but the ES subjects can learn to control the tailshocks, whereas the IS subjects cannot. Importantly, the numerous consequences that typically follow IS treatment (learning deficits, anxiety-like behavior, etc.) do not develop after ES (Maier, 1990; Baratta et al., 2007; Christianson et al., 2010; see review by Maier & Watkins, 2010). Thus, the action that affects shock termination blunts the impact of the tailshock stressor (Maier et al., 2006).

Previous research has implicated the activation of 5-HT neurons in the dorsal raphe nucleus (DRN) and their projections as critical to producing the behavioral consequences of IS (Maier & Watkins, 2005; Christianson & Greenwood, 2014). However, the DRN is not itself sensitive to stressor controllability, and both ES and IS provide excitatory input to DRN 5-HT neurons (Amat et al., 2005). Detecting a complex behavior–environment relationship such as control would probably be a cortical function. The DRN receives most of its cortical input from the prelimbic region of the ventral medial pre-frontal cortex (PFC; Vertes, 2004). The prelimbic cortex (PL) sends glutamatergic projections to the DRN that synapse preferentially onto GABAergic interneurons (Varga et al., 2001) that are positioned to inhibit 5-HT cells. Indeed, electrical stimulation of the PFC inhibits DRN 5-HT neuronal activity (Varga et al., 2003). Thus, the PL to DRN projection could be a pathway by which behavioral control might inhibit the DRN activation produced by stressors, and evidence does indeed suggest that the protective effects of control require the PFC (Amat et al., 2005, 2006). Inactivation of the PFC with muscimol during ES eliminates the protective effects of control, and subjects behave later as if the stressor had been uncontrollable. In contrast, activation of the PFC with picrotoxin during IS induces protection, and the subjects behave as if the stressor had been controllable. Moreover, control does activate PL projections to the DRN (Baratta et al., 2009).

Interestingly, the PL is also critical in instrumental appetitive act/ outcome learning (see Discussion). This type of learning is mediated by a corticostriatal circuit between the PL and the posterior dorsomedial striatum (DMS; Balleine & O’Doherty, 2010). The concepts of behavioral control and act/outcome learning overlap and are possibly identical (see Discussion), suggesting that perhaps the DMS is also critical for engaging the protective effects of control over aversive events. To better understand the neural circuitry of control over stress we thus investigated whether the DMS plays a role in mediating the effects of behavioral control in the ES/IS paradigm, as it does in appetitive act/outcome learning.

Materials and methods

Rats

Male Sprague–Dawley rats (Harlan Laboratories, Indianapolis, IN, USA), weighing 275–350 g, were housed two per cage on a 12-h light/dark cycle (on at 07:00 h and off at 19:00 h). Experiments were conducted between 09:00 and 1600 h. This study was carried out in strict accordance with the recommendations in the Guide for the Care and Use of Laboratory Animals of the National Institutes of Health, and was approved by the Institutional Animal Care and Use Committee of the University of Colorado at Boulder.

Experimental design

The purpose of the first experiment was to determine whether exposure to ES or yoked IS tailshock activated neurons of the DMS. To this end, immunoreactivity to the early immediate gene product, Fos, was determined in coronal slices from the dorsal striatum, a region previously identified to be critical for act/outcome learning (Shiflett et al., 2010; Shiflett & Balleine, 2011). Importantly, ES led to a significant induction of Fos in the DMS. Thus, the following experiments sought to establish a causal role for neuronal activity within the DMS for two well-established consequences of ES, i.e. protection from stressor-induced anxiety-like behavior and suppression of stressor-induced 5-HT release (for reviews see Maier & Watkins, 2010; Christianson & Greenwood, 2014). For that purpose, initially muscimol was microinjected into the DMS at 5 min before tailshock. However, the rats did not learn to escape the tailshocks, and thus muscimol could not be used. As an alternative, the selective N-methyl-D-aspartate (NMDA) receptor antagonist D-AP5 (Pauli et al., 2012) was employed, and, as shown in Fig. 3A, it did not impair the escape response. In the second experiment, D-AP5 or vehicle was injected into the DMS, or into the dorsolateral striatum (DLS) as a site-specificity control (n = 12/group), and the subjects were then immediately subjected to ES. Social exploration, a measure of anxiety-like behavior (File & Seth, 2003; Christianson et al., 2008), was tested 24 h later. Two rats per group were excluded, due to either paw injury or injector tip misplacement. A number of additional treatment groups were included as experimental controls (see Results). To evaluate whether the DMS contributes to the inhibition of the DRN that typically occurs with ES, rats were implanted with microdialysis probes in the DRN and microinjection cannulas in the DMS or DLS (n = 10/group). Tailshock-induced changes in extracellular 5-HT within the caudal DRN were examined in rats injected with D-AP5 in the DMS or DLS (10 rats/group) after baseline sampling. The rats then received ES immediately after the microinjections.

Fig. 3.

Effects of NMDA receptor blockade in the dorsal striatum. (A) Wheel-turn escape latency during exposure to controllable tailshock (ES) in rats that had received DMS or DLS D-AP5 or vehicle prior to stress. Mean (± SEM) time to reach the escape criterion per five-trial block. (B) Mean (± SEM) time spent in exploration during a 3-min juvenile social encounter. At 24 h previously, rats received DMS or DLS microinjections of D-AP5 or vehicle followed by ES or IS or remained as HCC rats. *IS significantly reduced social exploration compared with ES and HCC rat vehicle groups (P-values < 0.05). ^#ES rats treated with D-AP5 in the DMS differed from HCC rat–vehicle and ES–vehicle groups (P-values < 0.05). (C) 5-HT as a percentage of baseline in the caudal DRN (mean ± SEM) before, during and after 100 escapable tail-shocks (depicted in black bar). The arrow indicates the time of injection of D-AP5 in either the DMS or DLS. For comparison to previous data examining 5-HT after controllable or uncontrollable stress without microinjections, a dotted line is added to represent 5-HT levels in response to a similar ES exposure and a dashed line in response to IS, both taken from Amat et al. (2005). D-AP5 in the DMS led to significant increase in 5-HT (P-values < 0.05).

Wheel-turn escape/yoked inescapable stress procedure

Each rat was placed in a Plexiglas box (14 × 11 × 17 cm) with a wheel mounted in the front and a Plexiglas rod extending from the rear. The rat’s tail was taped to the Plexiglas rod and affixed with copper electrodes. Rats received shocks in yoked pairs (ES and IS). The treatment consisted of 100 trials with an average inter-trial interval of 60 s. Shocks began simultaneously for both rats in a pair and terminated for both whenever the ES rat met a response criterion. Initially, the shock was terminated by a quarter turn of the wheel. The response requirement was increased by one quarter turn when each of three consecutive trials was completed in < 5 s. Subsequent latencies under 5 s increased the requirement by 50% up to a maximum of four full turns. If the requirement was not reached in < 30 s, the shock was terminated and the requirement reduced to a single quarter turn. This procedure was used to ensure that the ES rats learned an operant response. The front wheel was locked for the IS rat. Shock intensity was 1.0 mA for the first 30 trials, 1.3 mA for the second 30 trials and 1.6 mA for the last 40 trials, to maintain good escape responding. Non-shocked home cage control (HCC) rats remained undisturbed in the colony.

Fos immunohistochemistry

At 2 h after the last tailshock, rats were deeply anesthetized with sodium pentobarbital (60 mg/kg) and transcardially perfused with 100 mL of ice-cold 0.9% saline followed by ~250 mL of 4% paraformaldehyde in 0.1 M phosphate buffer (pH 7.4). Brains were removed and postfixed in the same fixative overnight. After postfixation, brains were transferred to 30% sucrose and stored at 4 °C until sectioning. Sections (35 μm) were obtained in a cryostat at −20 °C and stored at 4 °C in cryoprotectant until staining. Fos immunoreactivity was detected by incubating sections with rabbit anti-c-Fos primary antibody (1 : 15 000, sc-52; Santa Cruz Biotechnology) and then biotinylated goat anti-rabbit secondary antibody (catalog no. 111-065-003, 1 : 200; Jackson Immunoresearch). Fos was visualized with the avidin–biotin horseradish peroxidase method (Vectastain Elite ABC Kit; Vector Laboratories) and with nickel-enhanced 3,3′-diaminobenzidine (Sigma) as chromogen. Stained sections were dehydrated, cleared (Citrisolve; Fisher) and mounted onto glass slides with Permount (Fisher). Coronal sections corresponding to Bregma +0.20 mm to −0.40 mm including the DMS and DLS, as in Shiflett et al. (2010) (see Fig. 1 for illustration of the region of interest), were assessed for the number of Fos-immunoreactive cells under a bright-field microscope. Fos-stained nuclei were identified by dark black ovoid particles. Images of the medial and lateral dorsal striatum were obtained on an Olympus BX-61 Upright microscope at 10× (UPlanFl 10×, N.A. 0.3) and cells were counted using IMAGEJ and the method described by C. Labno (http://www.unige.ch/medecine/bioimaging/tricks/imagejtutorials/cellCounting.pdf). Automated counts were compared with manually-counted sections to validate IMAGEJ parameters. Adjustments for brightness and contrast were made uniformly to all sections. The mosaic image in Fig. 1 was created using the automatic photomerge function in Adobe Photoshop CS6.

Fig. 1.

Top – Illustration of regions of interest for Fos quantification in the posterior DMS or posterior DLS. Adapted from Paxinos & Watson (1998) with permission from Elsevier. Middle – Example digital photomicrograph depicting Fos immunoreactivity (black ovoid particles) in the DMS and DLS, which are outlined with dashed lines. Bottom – Mean (± SEM) number of Fos-immunoreactive cells per rat in the DMS and DLS. *Exposure to ES led to a selective increase in Fos immunoreactivity within the DMS compared with controls and both ES and IS increased Fos immunoreactivity within the DLS (P-values < 0.05).

Surgery and cannulation

Surgery was carried out under anesthesia with halothane (1.5–3% in O₂). Microinjection and microdialysis guide cannulas were implanted and fixed in place with stainless steel screws and acrylic cement as previously described (Amat et al., 2005). Cannula guides for microinjections were implanted bilaterally in either the DMS (−0.2 mm from bregma, ± 2.2 mm laterally and 3.5 mm from the skull surface) or posterior DLS (−0.2 mm from bregma, ± 4.3 mm laterally and 3.6 mm from the skull surface). In addition to microinjection cannulas, animals for microdialysis had a cannula guide (CMA 12, MW cutoff 20 kDa; Holliston, MA, USA), implanted with the tip terminating just above the caudal DRN (−8.3 mm from bregma, 5.5 mm from the skull at the midline). A screw cap of a 15-mL conical centrifuge tube, the central lid portion of which was removed, was also affixed to the skull so that its threads were exposed and it encircled the cannula guide. This was done so that the skull assembly could be protected during microdialysis. A stylet was placed in the cannulas and each rat was inoculated with 0.25 mL/kg (subcutaneous) penicillin (Combi-Pen; Agrilabs, St Joseph, MO, USA) and the non-steroidal analgesic Loxicam (0.5 mg/kg; Norbrook Laboratories, Lenexa, KS, USA). Rats were allowed to recover for 1–2 weeks after surgery before experimentation. At the end of the experiment the animals were given an overdose of pentobarbital (2.6 g/kg) and then decapitated, the brains were sliced at 40 μm, and stained with cresyl violet for cannula verification.

D-AP5 microinjections

Injections were made bilaterally into the DMS or DLS with 0.5 μL of a 30 mM solution of D-AP5 (Tocris; Bast et al., 2005; Matus-Amat et al., 2007) or artificial cerebrospinal fluid (pH 7.2) vehicle at 5 min before the ES/yoked IS session. HCC rats were also injected with D-AP5 or vehicle at 24 h before behavioral testing. Dual 33-gauge microinjectors (Plastics One) attached to PE 50 tubing were inserted through the guides, from which they protruded 1 mm. The other end of the tubing was connected to a 25-μL Hamilton syringe that was attached to a microinjection unit (Model 5000; Kopf). The volumes were injected over a period of 30 s, and the injector was left in place for 2 min to allow diffusion. At the end of the experiment, an overdose of pentobarbital (2.6 g/Kg) was administered and brains were removed and frozen. A cryostat was used to take 40-μm sections, which were then stained with cresyl violet for cannula placement verification. Subjects were included if the injector tip (Fig. 2A) or probe track (Fig. 2B) fell within the target structure.

Fig. 2.

Microinjection and dialysis cannula placements. (A) Gray areas represent the sector where the injector tips were located. (B) Gray bars represent the placement of microdialysis probes. Numerals indicate distance from Bregma (in mm). Illustrations adapted from Paxinos & Watson (1998) with permission from Elsevier.

Juvenile social exploration tests

Social exploration testing was conducted at 24 h after the ES/yoked IS procedure as described previously (Christianson et al., 2010). Briefly, each experimental subject was allocated a single plastic cage with shaved wood bedding. To begin, the test rats were placed into the test cage and, after 45 min, a 28 (± 2)-day-old juvenile was introduced to the cage for 3 min and an observer, blind to treatment, timed exploratory behaviors (snifffng, pinning, and allogrooming) initiated by the adult. Juveniles were used for multiple tests but were never used twice for the same adult rat. The testing order was counterbalanced for stress and drug treatments.

In vivo microdialysis

During the afternoon before the experiment, a CMA 12 microdialysis probe (0.5 mm in diameter, 1-mm membrane) was introduced through the cannula guide so that the membranous tip of the probe was within the caudal DRN. A portion of a 15-mL Eppendorf tube was screwed onto the skull-mounted screw cap, through which the dialysis tubing, protected within a metal spring, entered and attached to the probe. Each animal was placed individually in a Plexiglas bowl (Bioanalytical Systems) and infused with artificial cerebrospinal fluid (pH 7.2; 145 mM NaCl, 2.7 mM KCl, 1.2 mM CaCl, 1.0 mM KCl) at a rate of 0.2 μL/min overnight. At about 09:00 h the next day, the flow rate was increased to 1.5 μL/min and a 90-min stabilization period was allowed. The infusion flow remained constant throughout the experiment. Samples were collected every 20 min. After stabilization, three baseline samples were collected, and then D-AP5 was injected in the DMS or DLS. The rats were then placed in Plexiglas wheel-turn boxes that were designed to accommodate the dialysis tubing. There they received 100 ES or yoked IS tailshocks. Five samples were collected during this session. After the session the rats were transferred back to the Plexiglas bowls where three postshock samples were collected. During collection of the last sample, brisk movements of the skull-mounted screw cap were performed to test for possible 5-HT increases due to rat head movement during the dialysis. The data from the rat were discarded if that procedure caused 5-HT increases.

5-HT analysis

The 5-HT concentration was measured in dialysates by high-performance liquid chromatography with electrochemical detection. The system consisted of an ESA 5600A Coularray detector with an ESA 5014B analytical cell and an ESA 5020 guard cell. The column was an ESA HR-80X3.2 maintained at 38 °C, and the mobile phase was the ESA buffer MD-TM. The analytical cell potentials were kept at −100 mV and +200 mV and the guard cell at +220 mV. Dialysate (25 μL) was injected with an ESA 542 auto-sampler that kept the dialysates at 6 °C. External standards (Sigma) were run each day to quantify 5-HT.

Results

Dorsomedial striatum Fos immunoreactivity

Fos immunoreactivity was quantified in coronal sections with regions of interest corresponding to the DMS and DLS (Fig. 1). Both ES and IS produced a small increase in Fos in the DLS (Fig. 1). However, only ES led to an increase in the DMS, and this increase was quite large. A two-way ANOVA with stress as a between-subjects factor and region as a within-subjects factor revealed a significant main effect for stress (F_2,18 = 7.675, P = 0.004) and region (F_1,18 = 2.248, P = 0.047) but no interaction (F_2,18 = 1.680, P = 0.214). Pair-wise post hoc comparisons between stress treatments in the DMS revealed that ES differed significantly (P-values < 0.05, Fisher, protected least significant difference) from both the IS and HCC groups, which did not differ from each other. No group differences reached significance in the DLS. In sum, exposure to ES led to a selective increase in Fos immunoreactivity within the DMS, but not the DLS.

Wheel-turn escape learning

Either D-AP5 or vehicle was microinjected through indwelling cannulas aimed at the DMS or DLS at 5 min before the start of the session. As in all of our previous research (e.g. Amat et al., 2005), the number of wheel turns required to terminate each shock was increased as the rats became more proficient at escape. To quantify escape performance, the latency to terminate each tailshock was recorded (Fig. 3A). As is evident, rats injected with D-AP5 in the DMS or DLS learned to escape throughout the stress session as efficiently as did the controls that received vehicle. No effects of drug or injection site were found in a repeated-measures ANOVA (F_2,23 = 0.120, P > 0.05). The location of cannula placements within the DMS and DLS are shown in Fig. 2A.

Juvenile social exploration at 24 h after stress

The mean time spent exploring the juvenile is depicted in Fig. 3B. As we have previously reported, IS reduced social exploration and ES did not. Importantly, intra-DMS D-AP5 eliminated the protection afforded by control, whereas intra-DLS D-AP5 did not. The time spent exploring in unstressed, HCC rats injected with either vehicle or D-AP5 into the DMS did not differ (t-test, P > 0.05) and they were pooled for subsequent analysis. A one-way ANOVA yielded a significant main effect of treatment (F_4,51 = 7.461, P < 0.001). Pairwise post hoc comparisons were made between treatment groups using the Fisher protected least significant difference procedure. Exposure to IS, but not ES, led to a significant reduction in social exploration (DMS HCC vs. DMS vehicle IS and DMS HCC vs. DMS vehicle ES, P-values < 0.05). Intra-DMS injection of D-AP5 before ES caused a reduction in juvenile exploration similar to that observed in IS-treated rats (DMS D-AP5 ES vs. DMS vehicle ES and DMS D-AP5 ES vs. DMS HCC, P-values < 0.05) and it was reduced to the same level as in IS-treated rats. By contrast, D-AP5 injected in the DLS before ES had no effect on social exploration (DMS HCC vs. DLS D-AP5 ES, P > 0.05).

Extracellular dorsal raphe nucleus 5-HT during escapable shock

Figure 3C shows the extracellular levels of 5-HT as percentages of baseline. ES led to an initial increase in 5-HT that quickly returned to baseline levels in rats injected with D-AP5 in the DLS. This is the pattern typically observed during ES [see dashed line, taken from Amat et al. (2005)]. However, 5-HT remained elevated throughout the stressor and beyond when D-AP5 was injected in the DMS. This is the pattern that we normally observe in rats that receive IS [see dotted line, taken from Amat et al. (2005)]. The absolute 5-HT levels did not differ between groups at baseline. A two-way ANOVA with injection region as a between-subjects factor and sample as a repeated measure identified a main effect of region (F_1,15 = 8.633, P = 0.010), a main effect of sample (F_10,150 = 7.580, P < 0.0001) and their interaction (F_10,150 = 2.307, P = 0.015). Post hoc comparisons identified significant differences between DMS and DLS injections at all samples at 40 min after stressor onset until the end of sampling (Fisher protected least significant difference, P-values < 0.05).

Discussion

The presence of behavioral control typically mitigates the impact of stressors on behavior and DRN 5-HT activity (Maier et al., 2006; Maier & Watkins, 2010). The results of this series of experiments are, first, that ES but not IS increased Fos in the DMS, i.e. the controllable stressor, not the stressor per se, induced neuronal activity leading to greater Fos expression in the DMS. Second, subjects exposed to ES after intra-DMS D-AP5 administration behaved as if they had received IS in the social exploration test. Stressor-induced changes in social exploration depend upon stressor-induced sensitization of the DRN (Christianson et al., 2010). We therefore predicted that intra-DMS NMDA receptor blockade would also affect DRN 5-HT released during stress. Indeed, intra-DMS D-AP5 prevented the inhibition of 5-HT that typically occurs with ES. Thus, the neuronal activation induced in the DMS by ES exposure is critically involved in the protection from stressor effects on the 5-HT system and anxiety-like behavior. In our studies, infusions into the DLS appeared to have no effects.

These data clearly suggest that the DMS is involved in mediating the effects of behavioral control over stressors. The role of the DMS was investigated because of the seeming similarity between the concepts of behavioral control and act/outcome or contingency learning. In their original work, Maier & Seligman (1976) defined the concept of behavioral control over a stressor with reference to a two-dimensional space. One dimension was formed by the conditional probability of stressor termination occurring in the presence of a defined response, and the other dimension was formed by the conditional probability of stressor termination in the absence of that designated response. Maier & Seligman (1976) argued that when these two probabilities are equal the organism has no control over the stressor, but any situation in which these two conditional probabilities are unequal is one in which control is possible, either by performing or withholding a response. In the usual situation in which there is behavioral control, as in the present experiments in which a wheel-turn response is followed by shock termination, the probability of shock termination given the defined response is much higher than the probability of shock terminating in the absence of this response. However, by the definition of control provided by Maier & Seligman (1976), a very different kind of control should also be possible. Maier (1970) determined the outcome of a procedure in which shock terminated only when the subjects withheld a response that occurred naturally at a high rate, for a fixed period of time, i.e. the experiment arranged a circumstance in which the two probabilities described above were unequal, but here the probability of the stressor occurring in the absence of a response was much higher than in the presence of the response. Importantly, even this type of control was protective.

The development of the behavioral control concept occurred within the context of aversive learning and stress research. Interestingly, very similar concepts have emerged in the instrumental reward learning literature. These concepts grew out of an old debate concerning whether instrumental learning is to be understood as the formation of a stimulus–response association, a habit, or instead a response–outcome association, an expectancy. However, the study of brain mechanisms involved in instrumental reward learning came to indicate that both types of learning occur under different circumstances, and involve separable brain mechanisms (Dickinson & Balleine, 1994; Yin et al., 2005; Shiflett & Balleine, 2011). Learning that occurs using the act/outcome system is said to be sensitive to the contingency between the response and the reward (Balleine & Dickinson, 1998). Contingency is defined as the ‘difference between the probability of gaining a target reward (r) given that a specific action (a) is performed and the probability of gaining the reward in the absence of that action’ (Liljeholm et al., 2011, p. 2474). Instrumental performance learned using this system is also sensitive to changes in the value of the outcome. Thus, for example, if the outcome is made less valuable (e.g. the subject is satiated), responding immediately declines (Adams & Dickinson, 1981). The habit system is quite different. Responses learned using this system are not sensitive to contingency and so do not decline if rewards are given in the absence of the response, and also are not sensitive to changes in reward value (Shiflett & Balleine, 2011).

A substantial body of research indicates that act/outcome reward learning is mediated by a system that includes the DMS and PL. For example, lesion, inactivation, or NMDA blockade of either the PL or DMS prevents act/outcome learning (Yin et al., 2004, 2005). Subjects that have received these treatments can learn instrumental responses to obtain appetitive reinforcers, but these responses are then insensitive to contingency or to changes in reward value, i.e. they are then acquired by the habit system. The habit system involves the DLS, with no involvement of the PFC (Everitt & Robbins, 2013).

Virtually all of the act/outcome–habit systems literature involves experiments utilizing positive rewards. However, the concepts of control and contingency seem identical, and so here we sought to determine whether the findings from this appetitive reward literature would extend to the aversive/stress domain. This extension was encouraged by prior findings that the PL is critical in the mediation of the protective effects of control (Maier & Watkins, 2010). The present findings indicate that providing an escape response in a shock situation activates the same PL–DMS system that has been implicated in instrumental reward learning. Intriguingly, the escape response had to be acquired by this system for escape learning to be protective against the behavioral and DRN 5-HT activating effects of the stressor. Just as is the case with reward learning, learning of the escape response was possible during NMDA blockade of the DMS, presumably by the habit system (Yin et al., 2006). The implication is that the PL–DMS act/outcome system has to be engaged during the stressor for the provision of an escape response to be protective.

Thus, it is not the exercise of control at a procedural level that blunts the impact of the stressor, i.e. the subjects administered D-AP5 both intra-DMS and intra-DLS learned to turn the wheels to escape tailshock and turned the wheels to terminate the tailshocks with equal efficiency. However, the performance of this escape behavior was of no benefit if NMDA receptors were blocked in the DMS. Similarly, PL inactivation with muscimol did not reduce learning/performance of the wheel-turn escape response (Amat et al., 2005), but it too eliminated the behavioral protection afforded by control, as well as the inhibition of DRN 5-HT activation by control (Amat et al., 2005). Clearly, simply turning the wheel to terminate tailshocks is not the critical event. Instead, it would appear to be the activation of the PL–DMS act/outcome system during the aversive experience that is critical.

The PL has now been implicated in mediating the impact of stressor control in two very different ways. Previous studies have indicated that, when control is present, descending PL pyramidal neurons inhibit DRN 5-HT neurons that are activated by the stressor, thereby reducing behavioral effects proximately mediated by the DRN. The present data suggest that the PL may also be involved in the detection of control, together with the DMS, in a corticostriatal circuit. It should be noted that PL neurons that project to the DRN and DMS are located in distinctly different regions of the PL (Gabbott et al., 2005). Thus, these two functions, i.e. detection of control and using this information to inhibit reactions to the stressor, may be distinct yet both involve the PL.

The experience of behavioral control over a stressor not only blunts the immediate impact of the stressor being experienced, but it also reduces the impact of subsequent stressors, even if they are uncontrollable. Thus, for example, exposure to ES as in the present studies blocks the behavioral and neurochemical effects of either IS (Amat et al., 2006), or even of social defeat (Amat et al., 2010), occurring 7 days later. It has been suggested that this ‘immunization’ occurs because ES induces plasticity in the PL (Varela et al., 2012). Perhaps plasticity also occurs in the DMS, or more properly in the corticostriatal circuit. Evidence regarding this possibility awaits future research. However, this possibility does provide a means of conceptualizing how procedures such as cognitive–behavioral therapy are protective. It has been argued (DeRubeis et al., 2008) that the act/outcome neural system is a mechanism for detecting one’s causal efficacy in altering the environment, although this argument was made in the context of appetitive rewards. The teaching of efficacy/control is part of behavioral therapies, and perhaps this engages the PL–DMS circuitry, inducing lasting plasticity in these structures.

Abbreviations

5-HT

serotonin

D-AP5

D-(−)-2-amino-5-phosphopentanoic acid

DLS

dorsolateral striatum

DMS

dorsomedial striatum

DRN

dorsal raphe nucleus

ES

escapable shock

HCC

home cage control

IS

inescapable shock

NMDA

N-methyl-D-aspartate

PFC

prefrontal cortex

PL

prelimbic cortex