Behavioral control blunts reactions to contemporaneous and future adverse events: Medial prefrontal cortex plasticity and a corticostriatal network

Abstract

It has been known for many years that the ability to exert behavioral control over an adverse event blunts the behavioral and neurochemical impact of the event. More recently, it has become clear that the experience of behavioral control over adverse events also produces enduring changes that reduce the effects of subsequent negative events, even if they are uncontrollable and quite different from the original event controlled. This review focuses on the mechanism by which control both limits the impact of the stressor being experienced and produces enduring, trans-situational “immunization”. The evidence will suggest that control is detected by a corticostriatal circuit involving the ventral medial prefrontal cortex (mPFC) and the posterior dorsomedial striatum (DMS). Once control is detected, other mPFC neurons that project to stress-responsive brainstem (dorsal raphe nucleus, DRN) and limbic (amygdala) structures exert top–down inhibitory control over the activation of these structures that is produced by the adverse event. These structures, such as the DRN and amygdala, in turn regulate the proximate mediators of the behavioral and physiological responses produced by adverse events, and so control blunts these responses. Importantly, the joint occurrence of control and adverse events seems to produce enduring plastic changes in the top–down inhibitory mPFC system such that this system is now activated by later adverse events even if they are uncontrollable, thereby reducing the impact of these events. Other issues are discussed that include a) whether other processes such as safety signals and exercise, that lead to resistance/resilience, also use the mPFC circuitry or do so in other ways; b) whether control has similar effects and neural mediation in humans, and c) the relationship of this work to clinical phenomena.

1. Introduction

There are large differences in how individuals react to seemingly the same adverse life events, with some being strongly impacted (vulnerable) while others either show little impact (resistant) or recover quickly (resilient). This has led to intensive investigation of factors that modulate how organisms react to adverse events (here called “stressors” for convenience), factors that are either contemporaneous with the stressor being experienced (e.g., the presence of safety signals), or historical and predispose how organisms react to adverse events in the future (e.g., early handling). It is not at all clear how to categorize or classify these processes. Some of these are non-experiential, such as genetic polymorphisms and changes in the microbiome. Others are experiential, with some being physical/physiological (e.g., elevated carbon dioxide) and some involving how the organism processes the adverse event (e.g., cognitive/behavior therapy). Clearly, these are not distinct categories and there are factors that induce resistance or resilience that are a mixture. For example, exercise could have beneficial effects because it confers a sense of efficacy, or because exercising muscles release a substance that enters the nervous system and directly alters neural function. It would be highly unlikely that all of these would modulate vulnerability and resistance/resilience by the same mechanisms, and this will indeed be one conclusion of this review.

Our laboratory has been interested in psychological variables, that is, variables that involve how the organism processes a stressor. In order to implicate a psychological factor it is necessary to vary the factor while at the same time holding the physical aspects of the stressor constant, and we have developed paradigms to do so (see below). In humans, how adverse events are appraised and viewed is key (Southwick et al., 2005), as is the individuals assessment of her ability to cope (Dicorcia and Tronick, 2011). These are the types of processes that we have set out to understand at a neural circuit and neurochemical level. Perceived behavioral control over an adverse event is at the core of coping, and this is what we have studied in animals where neural processes can be explored in detail. The paradigm that we employ involves triads of subjects, typically rats. Each of the subjects is placed in a small box with a wheel located on the front wall, and its tail extends from the rear of the chamber and is affixed with shock electrodes. Two of the rats receive periodic tailshocks, with each tailshock beginning at the same time for both rats. For one of the shocked rats, turning the wheel at the front of the chamber terminates each shock. If the subject does not turn the wheel each shock persists to an experimenter defined limit. Thus, this rat has an instrumental escape response (escapable shock, ES) and has behavioral control over the duration of each of the tailshocks. This rat cannot avoid a tailshock, but it can reduce its duration. For the second shocked rat each tailshock is yoked to its ES partner and terminates whenever the ES subject turns the wheel. For this rat turning the wheel has no consequence, and this subject does not have control over the shock durations. That is, the shocks are inescapable (IS). Thus, the physical aspects of the tailshocks (intensity, durations, temporal distributions, etc.) are identical for the ES and IS subjects, but ability to exert behavioral control over an aspect of the adverse event differs. The third rat is not shocked, and with this paradigm it is possible to determine whether any behavioral, neurochemical, endocrine or other consequence of the tailshock stressor is modulated by control.

Since exposure to potent stressors is known to produce a variety of changes in subsequent behavior often summarized as either anxiety-like or depression-like, it is not surprising that IS has been found to alter a broad range of behaviors for a number of days. Exposure to IS has been shown to lead to failure to escape shock in a new situation such as a shuttlebox (the “learned helplessness effect”), reduced aggression, reduced social dominance, immobility, neophobia, exaggerated fear conditioning, impaired fear extinction, anxiety on standard measures such as juvenile social investigation, hyper-vigilance as indicated by eaxaggertated attention to external cues, reduced food and water intake, etc. (Maier and Watkins, 1998 for review). Importantly, none of these occur following exactly equal ES. That is, the presence of control blocks all of these behavioral changes. Importantly, the presence of control does more than blunt the behavioral impact of the stressor being controlled. In addition, it alters the organism in such a way that the behavioral and neurochemical effects of later experiences with uncontrollable stressors are blocked, a phenomenon coined “immunization” (Maier and Seligman, 1976, Williams and Maier, 1977). Physically identical IS does not reduce the impact of subsequent uncontrollable stressors, and indeed, often exacerbates them. Thus, it is not the prior occurrence of the stressor that is immunizing, but rather the experience of control over the stressor. Several features of ES-induced immunization are noteworthy here. First, Such immunization effects can be quite long lasting. For example, the experience of ES in adolescence was shown to block the behavioral effects of IS in adulthood (Kubala et al., 2012). Second, immunization is trans-situational. Thus, ES in one environment/apparatus can block the effects of IS in a very different apparatus/environment. For example, Amat et al. (2010) demonstrated that exposure to ES blocked the behavioral and neurochemical effects of social defeat occurring 7 days later. Social defeat and ES are very different physically, were administered in very different apparati, and even on different floors of the building by different experimenters to minimize common cues.

The purpose of this review is to summarize the research that we have conducted directed at understanding the neural mechanisms by which the experience of control blunts the behavioral impact of the stressor being controlled, here tailshock, as well as subsequent uncontrollable stressors occurring in the future. However, this research will be difficult to understand without at least a brief summary of some of the mechanisms by which IS produces the behavioral changes that it does.

2. The dorsal raphe nucleus (DRN)

How could IS produce all of the diverse behavioral outcomes that follow? As a starting point we used the work on conditioned fear as a model. The central nucleus of the amygdala had been shown to serve as a final common efferent structure, sending projections to regions of the brain that are the proximate mediators of the wide ranging responses that occur during fear. Thus, for example, the central nucleus projects to the periaqueductal gray (PAG) thereby producing the freezing response that is part of fear, the hypothalamus thereby leading to the cardiovascular changes that are part of fear, etc. The central nucleus is activated during fear, and the diverse array of behavioral and physiological changes that constitute fear occur because the central nucleus projects to the relevant controlling structures.

It seemed to us that the changes produced by exposure to IS could be summarized as inhibited fight/flight and exaggerated fear/anxiety. The dorsal PAG (dPAG) was known to be critical for mediating fight/flight (Brandao et al., 1994), while the amygdala was known to be critical for fear/anxiety (LeDoux, 2003). It was also known that the dorsal raphe nucleus sends serotonergic (5-HT) projections to both structures, and that 5-HT facilitates amygdala function and inhibits dPAG function (Graeff et al., 1997). Thus, if IS, relative to ES, were to selectively activate the DRN, this would recapitulate many of the behavioral changes that are produced by IS. Moreover, the DRN projects to the striatum, a structure important for instrumental learning such as escape learning. Indeed, IS proved to produce a much more intense activation of 5-HT neurons in the mid to caudal regions of the DRN than does ES, the region of the DRN that projects to regions such as the amygdala (Hale et al., 2012). Thus, IS was found to induce Fos in 5-HT labeled neurons (Grahn et al., 1999) and to produce large increases in extracellular 5-HT in both projection regions such as the amygdala (Amat et al., 1998a), and within the DRN itself (Maswood et al., 1998), likely from axon collaterals (Tao et al., 2000).

The fact that DRN 5-HT neurons are only activated if the stressor is uncontrollable does not imply that activation of these cells is either necessary or sufficient to produce the behavioral sequelae of IS. To examine whether DRN 5-HT activity is necessary, DRN 5-HT activation has been blocked by microinjection of a variety of pharmacological agents during exposure to IS. In all cases, blockade of 5-HT activation within the DRN blocked the occurrence of the behavioral changes normally produced by IS (Maier et al., 1993, 1995b, 1994). Moreover, pharmacological blockade of 5-HT receptors in target regions of the DRN blocked the behaviors altered by IS that are mediated by those structures. For example, blockade of 5-HT2C receptors in the basolateral amygdala prevented the anxiety-like changes such as reduced juvenile social investigation (Christianson et al., 2010), while blockade of 5-HT2C receptors in the striatum prevented the shuttlebox escape learning deficits (Strong et al., 2011). In addition, simply activating DRN 5-HT neurons pharmacologically, in the absence of any stressor at all, produced the behavioral consequences that are produced by IS (Maier et al., 1995a).

However, IS-induced increases in DRN 5-HT activity continue for only a few hours beyond the termination of IS, yet the behavioral effects of IS persist for a number of days, and blockade of 5-HT receptors at the time of later testing blocks the behavioral effects. These findings suggest that perhaps the intense activation of DRN 5-HT neurons produced by uncontrollable stressors produces a persistent sensitization of these neurons so that the later testing procedures (footshock in the shuttlebox, the presence of a juvenile, etc.) now activate these neurons. Indeed, a single footshock (Amat et al., 1998b) and even the mere presence of a juvenile (Christianson et al., 2010) lead to activation of DRN 5-HT neurons if the subjects had experienced IS a day earlier. Without prior IS no activation at all was observed in response to these mild stressors. A number of mechanisms are likely responsible for this uncontrollable-stress induced sensitization of DRN 5-HT neurons. One mechanism for which there is strong evidence concerns 5-HT1A inhibitory autoreceptors present on the soma and dendrites of DRN 5-HT cells. As noted above, IS leads to the accumulation of very high extracellular levels of 5-HT within the DRN itself, with this elevation persisting for a number of hours (Maswood et al., 1998). Rozeske et al. (2011) have shown that this 5-HT accumulation desensitizes these inhibitory autoreceptors for a number of days, thereby reducing the normal inhibitory control over these neurons.

Why does an uncontrollable stressor produce a greater activation of DRN 5-HT neurons than does a physically identical controllable stressor? One possibility is that this is intrinsic to the DRN, with the DRN itself detecting presence versus absence of behavioral control. However, this is most unlikely. In order to detect whether a tailshock is or is not controllable, that is, whether there is a contingency between behavioral responses and shock termination, a structure must receive sensory input indicating whether the stressor is present or not, and detailed motor input indicating whether a behavioral responses has or has not occurred. The DRN does not receive detailed sensory or motor input from cortical areas (Peyron et al., 1998). If s structure does not receive information as to whether a stressor is present or not, nor whether a behavior has occurred, it cannot detect control. This suggests that the DRN cannot operate in isolation and must receive inputs from other regions, thereby leading to its activation by IS. An obvious explanation for the dierential activation of DRN 5-HT neurons by IS relative to ES would be that ES does not lead to these inputs, or does so to a lessor degree. Here, the protective effects of ES would be produced passively, that is, by an absence of some “drive” to the DRN that is produced by IS. Therefore, we have examined a number of inputs to the DRN that stimulate DRN 5-HT activity during exposure to the IS stressor. We have found 3 that are clear: a CRH input, likely from the BNST; a noradrenergic (NE) input, likely from the locus coeruleus (LC), and a glutamate (GLU) input, likely from the habenula. Thus, blockade of CRH receptors (Hammack et al., 2002, Hammack et al., 2003), NE receptors (Grahn et al., 2002) or GLU receptors (Grahn et al., 2000) within the DRN during IS prevents the behavioral effects of IS, as well as the DRN 5-HT activation. Furthermore, lesion of these structures blocks the effects of IS (Amat et al., 2001, Hammack et al., 2004). However, contrary to the expectation that ES would not then activate these structures and inputs to the DRN, or do so to a lessor degree than does IS, ES produced the same level of activation and input (Amat et al., 2001). For example, in an extensive series of studies examining LC activation, McDevitt et al. (2009) found that both IS and ES intensely activate the LC as assessed by c-fos mRNA, Fos protein, and tyrosine hydroxylase mRNA, but to exactly the same degree.

Before leaving the DRN and 5-HT, it should be noted that intense DRN activation is not restricted to IS as a stressor. For example, social defeat (which is arguably uncontrollable) does so as well (Amat et al., 2010). However, all stressors do not do so, and it has been suggested that stressors have to be prolonged and intense (Takase et al., 2005). In addition, IS and other uncontrollable stressors certainly do more than activate the DRN, and produce outcomes that are not mediated by the DRN. For example, IS conditions fear to cues that are present, and this is mediated by the standard amygdala circuitry (Maier et al., 1993). Finally, there has recently been a large amount of research devoted to a more general understanding of the role of the DRN in stress-related phenomena than the focus on controllability phenomena that is the subject of this review (Valentino et al., 2010).

3. Neural mediation of stressor control

3.1. The medial prefrontal cortex (mPFC)

The research reviewed above indicates that uncontrollable stressor exposure differentially activates DRN 5-HT neurons relative to controllable stressors, but that both types of stressors appear to provide equivalent excitatory input to the DRN. This juxtaposition of findings leaves only one obvious possibility, namely, that controllable stressors lead to an input to the DRN that differentially inhibits 5-HT activity. That is, both ES and IS induce inputs to the DRN that activate the DRN, but only ES produces an input that inhibits DRN 5-HT. Under this view control does not produce its protective effects passively by lacking something that uncontrollability produces as in the original view, but instead does so actively.

If the detection/processing of control were to lead to the inhibition of DRN 5-HT neuronal activity, the cortex would be an obvious source. Interestingly, the DRN receives virtually all of its cortical input from the prelimbic (PL) region of the ventral medial prefrontal cortex (vmPFC) (Peyron et al., 1998, Vertes, 2004). Importantly, electrical stimulation in this region leads to the inhibition of DRN 5-HT neuronal firing (Hajos et al., 1998). This inhibition occurs because glutamatergic pyramidal output neurons from the PL to the DRN synapse preferentially within the DRN on GABAergic interneurons that in turn inhibit 5-HT cells (Jankowski and Sesack, 2004). This arrangement is schematized in Fig. 1.Thus, if ES were to selectively (relative to IS) activate PL output to the DRN, then the presence of control would inhibit DRN 5-HT activity, leading to the differential activation by stressors of differing controllability. This model is schematized in Fig. 2. Here, a number of stress-responsive structures drive the DRN without regard to stressor controllability. The DRN is a point of convergence, summing the inputs and projecting to regions that are the proximate mediators of the behavioral changes. Importantly, the DRN itself is under top–down inhibitory control from the mPFC, with the descending activation being triggered by the presence of behavioral control.

Fig. 1.

Schematic representation of prefrontal projections to the DRN that lead to prefrontal inhibitory control over DRN 5-HT neurons.

Fig. 2.

Proposed model by which stressor controllability regulates the DRN and behavior. Inputs from a number of structures that respond to aversive stimulation converge on the DRN. These inputs are not sensitive to behavioral control. The DRN integrates thee inputs and innervates structures that are proximate regulators of behavior. The detection of control leads to top–down inhibition of DRN 5-HT neurons, thereby blunting the behavioral impact of the aversive stimulation. From Maier and Watkins (2010).

Over the past several years we have collected a large amount of evidence in support of this model. To summarize: 1) Clearly, this model requires that the presence of control activate mPFC PL pyramidal neurons that project to the DRN. To evaluate this possibility Baratta et al. (2009) injected the retrograde tracer FluoroGold into the mid/caudal DRN in order to label PL cells that project to the DRN. Then, subjects received ES, yoked IS, or no shock, and then Fos was examined in the PL. ES, relative to IS, did indeed induce Fos in FluoroGold labeled cells, thus directly demonstrating that control activates PL neurons that project to the DRN. 2) The buffering effect of control should require activation of the mPFC-to-DRN pathway (see Fig. 1). The projecting pyramidal neurons are under GABAergic inhibition (see Fig. 3), and so GABA agonists would inhibit the glutamatergic pyramidal output neurons. Thus, to examine this prediction, the GABA agonist muscimol or vehicle was microinjected in vmPFC before exposure to ES, yoked IS, or no shock, with separate experiments examining either the DRN 5-HT activation produced by the stressors or the later behavioral sequelae such as shuttlebox escape learning deficits and reduced juvenile social investigation. Inactivation of PL output during stressor exposure completed prevented the protective effects of control, both neurochemically and behaviorally (Amat et al., 2005). That is, ES now led to the same behavioral changes and DRN 5-HT activation as did IS. It is important to note that the ES subjects performed the wheel turn escape response in an unimpaired manner. Thus, they turned the wheel, terminated the tailshocks, but this was of no benefit if the mPFC was inhibited. Of course, simply inhibiting the mPFC in the absence of shock had no effect at all. 3) The buffering effects of control should be mimicked by simply exogenously activating mPFC ouput during exposure to uncontrollable stressors. To examine this possibility Amat et al. (Amat et al., 2008) microinjected the GABA antagonist picrotoxin to activate the pyramidal output cells during ES, IS, or no shock. Activating the mPFC during the stressor duplicated the effects of control. Now, IS produced neither DRN 5-HT activation nor shuttlebox deficits and reduced social investigation. A shortcoming of this experiment is that intra-mPFC picrotoxin activates mPFC projections to regions other than the DRN. However, Warden et al. (Warden et al., 2012) have reported that selective optogenetic activation of the vmPFC-to-DRN pathway reduces inactivity in a swim test.

Fig. 3.

Schematic representation showing GABAergic inhibitory control over mPFC glutamatergic pyramidal neurons. GABA agonists would prevent control from activating these neurons that project to the DRN.

3.2. The posterior dorsal medial striatum (DMS) and the act/outcome system

Detecting/processing the presence of control and regulating the DRN as a consequence are conceptually separable functions. The research summarized above clearly indicates that the mPFC is involved in regulating the DRN under conditions in which a stressor is controllable via its descending projections, but does the mPFC by itself also detect that the stressor is controllable? A consideration of the concept of control suggests an intriguing possibility. Maier and Seligman (Maier and Seligman, 1976) defined control over a stressor with regard to the difference between 2 conditional probabilities—the conditional probability of the stressor being altered (e.g., shock termination) given that a behavioral response (e.g., turning the wheel) has occurred and the conditional probability of the stressor being altered given that the response has not occurred. Control is present whenever the 2 probabilities are unequal. Under this circumstance, the probability of stressor alteration can be increased either by making, or withholding a response. When the 2 probabilities are equal there is nothing that the organisms can do to alter the adverse event, that is, it is uncontrollable.

Interestingly, research concerning the neural mechanisms that mediate appetitive instrumental learning has involved a similar concept. There has been a long debate as to whether such learning involves the formation of a Stimulus-Response habit or instead a Response-Reinforcer expectancy. Work at the neural level has made it clear that both can take place and involve different neural systems (Balleine and O'Doherty, 2010). One system, called the act/outcome system, is said to be sensitive to the contingency between response and reinforcer. Contingency has been defines as “the difference between the probability of obtaining a target reward (r) given that a specific action (a) is performed and the probability of gaining the reward in the absence of the action” ((Liljeholm et al., 2011) p. 2474). The act/outcome system leads to “flexible” learning, and is sensitive to changes in the outcome or reward. A second system, called the habit system, is not sensitive to contingency but instead to only the temporal pairing between response and reward, and produces inflexible learning not sensitive to changes in the characteristics of the reward (Balleine and Dickinson, 1998). A large body of work indicates that the act/outcome system involves a corticostriatal circuit consisting of the PL and the posterior dorsal medial striatum (DMS), while the habit system has no prefrontal cortical involvement, but instead sensorimotor cortex and the dorsal lateral striatum (DLS). Thus, lesion, NMDA blockade, and inactivation of either the PL or DMS prevents act/outcome learning. Responses can still be learned, but only the habit system can be used, and so the learning is insensitive to contingency and to changes in the outcome (Shiflett and Balleine, 2011).

Behavioral control and contingency would appear to be identical concepts, albeit developed in different literature, and the impact of control clearly involves the PL in some fashion. A natural question, then, is whether sensitivity to control over a stressor is accomplished by the same corticostriatal circuitry as mediates act/outcome appetitive learning. First, Amat et al. (2014) examined Fos in the DMS and DLS after ES, IS, or control treatment. ES selectively induced Fos in the DMS, but not the DLS. Next, the NMDA antagonist AP5 was microinjected in either DMS or DLS before ES, yokes IS, or control treatment. Strikingly, AP5 in the DMS eliminated the buffering effects of control on both DRN 5-HT activation and behavior, just as does inactivation of the PL. That is, now ES activated the DRN and produced the typical behavioral consequences of IS. In contrast, intra-DLS AP5 was without effect and control was fully protective. As with PL inactivation, intra-DMS AP5 did not interfere with acquisition and performance of the wheel turn escape response during ES. The implication is that the wheel turn escape response was acquired via the habit system, but that controlling the shock with this system is not protective. Rather, the implication is that the controlling response must be learned by the act/outcome system. Thus, the PL seems to serve two functions. First, to detect the presence of control, in cooperation with the DMS. Second, to inhibit the DRN when control is detected. It should be noted that PL neurons that project to the DMS and the PL are located in distinctly different subregions of the PL (Gabbott et al., 2005), and thus different populations of PL neurons are likely involved in these 2 processes. The communication between these two is unknown. See Fig. 4 for a schematic representation of this concept.

Fig. 4.

Dual role of the PL in stressor controllability phenomena. First, in cooperation with the DMS behavioral control is detected. After detection, separate PL neurons project to the DRN, inhibiting DRN 5-HT activation.

4. The proactive stressor-blunting effects of control and plasticity in the PL

As already noted, the experience of control blunts the DRN activation and prevents the behavioral impact of subsequent IS or even other uncontrollable stressors such as social defeat, an effect of control that is quite enduring (Amat et al., 2010). It is important to understand the magnitude of the stressor resistance that is induced by control, and so a small amount of data from Amat et al. (2006) will be shown. Fig. 5 depicts the levels of extracellular 5-HT in the DRN assesses every 20 min with in vivo microdialysis before (B), during (S), and after (P) a session of IS. As already noted, when DRN 5-HT neurons are activated they release 5-HT within the DRN, and so this is a measure of DRN activation across time. There are 3 groups. One simply received no treatment before the IS, and as is evident, IS produced a large and prolonged increase in DRN 5-HT levels. A second group had received a session of yoked IS 7 days earlier, and this did not alter the 5-HT response to the IS. However, The third group received ES 7 days earlier and this almost completety eliminated the 5-HT increase produced by the IS. Clearly, the experience of control produced a profound change in how the brain responded to the IS.

Fig. 5.

Extracellular levels of 5HT in the DRN during a session of inescapable tailshocks for subjects that had experienced either IS, ES, or control treatment 7 days earlier. From Amat et al. (2006).

Not surprisingly, engagement of the mPFC and DMS is required at the time of the original ES for the blunting of the impact of the subsequent stressor to occur (Amat et al., 2005, Amat et al., 2014). A perhaps more interesting question is whether activation of the mPFC or DMS is also required at the time of the later uncontrollable stressor for production of resistance. To answer this question, muscimol was microinjected in vmPFC not during the original ES, but during the second IS stressor 7 days later. Thus, the subjects were allowed full use of the mpFC during the learning of control, but not during the subsequent second uncontrollable stressor. The clear result was that inhibiting the mPFC during the second stressor prevented immunization, both at the neurochemical and behavioral level. Now, the uncontrollable stressor exerted its full impact (Amat et al., 2008).

These data suggest that experiencing control induces plasticity in the mPFC so that a later experience with uncontrollable stressor exposure, which would normally not activate mPFC inhibition of the DRN, now does so. To examine this possibility Baratta et al. (2009) retrogradely labeled PL cells that project to the mid/caudal DRN. Subjects then received ES or IS in wheel turn boxes or control treatment, and then, 7 days later, IS while in restraining tubes. The target IS 7 days after the first treatment did not, of course, activate (induce Fos) DRN projecting PL neurons if the subjects had experienced IS or control treatment 7 days earlier. However, if ES had been experienced, now the IS did activate these projecting cells.

The Baratta et al. data suggest that the experience of control alters the functional properties of PL cells that project to the DRN. To directly determine whether this is the case, mPFC slices were prepared after the experience of ES or yoked IS and whole-cell current clamp recordings were made from PL pyramidal neurons in layers 5 and 6 (Varela et al., 2012). The experience of ES, but not exactly equal IS, increased the excitability of PL pyramidal neurons in layers 5 and 6, the location of cells that project to the DRN. ES shortened the membrane time constant, increased the action potential rise time rate and amplitude as well as the postspike afterdepolarization area. These changes would render the PL neurons more responsive to subthreshold inputs and more likely to produce multiple action potentials to input.

Neural plasticity is thought to require the production of new proteins, and often requires NMDA activation and the ERK pathway. Amat et al. (2006) microinjected the protein synthesis inhibitor anisomycin into mPFC before or immediately after ES. Blockade of protein synthesis at the time of ES prevented ES from blunting the behavioral and neurochemical effects of uncontrollable stressor exposure occurring 7 days later. Interestingly, microinjection of anisomycin at the time of later IS did not reduce the immunizing effects of earlier ES, even though muscimol does so (see above). These data support the idea that the original experience of control induces plastic changes in mPFC neurons that then respond to even uncontrollable stressors and inhibit the DRN. In further support, Christianson et al. (2014) found that ES, but not IS increases phosphorylated ERK in the PL, and that the immunizing effects of ES are prevented by PL microinjection of AP5 or the MEK inhibitor U0126. It might be noted that the role of the DMS in control-induced plasticity is still under investigation.

The PL and the PL-DMS act/outcome system are engaged under numerous conditions, and instrumental learning occurs frequently during development. Clearly, these experiences do not produce immunization against the impact of severe stressors. Thus, it must be the engagement of this system during an aversive experience that is critical. It is often stated that “neurons that fire together wire together”. This all suggests a scheme as depicted in Fig. 6. Imagine a set of neurons that are activated by intense stressors and PL neurons that are activated by control or contingency. Only when both occur is the plasticity/connection process initiated, so that later, stressors themselves will activate the PL and its projecting neurons. If this model is correct, then simply activating PL projection neurons during exposure to even IS, should lead to immunization. Thus, intra-PL picrotoxin or vehicle was administered during ES, yoked IS or control treatment. IS in a different environment occurred 7 days later. The critical finding (Amat et al., 2008) was that even IS blocked the later DRN activating and behavioral effects of subsequent IS if the PL was activated during the experience. Consistent with the model, intra-PL picrotoxin was without effect if it was given in the absence of a stressor. That is, PL activation plus uncontrollable stressor was immunizing, whereas neither were by themselves.

Fig. 6.

Proposed scheme for the experience of control over an adverse event alters subsequent mPFC responding in the presence of adverse events.

5. Is only the DRN modulated by control?

The mPFC projects to many structures other than the DRN, and the glutamatergic pyramidal projections often synapse on GABAergic interneurons that inhibit the principal cells in the region. For example, pyramidal neurons from the infralimbic cortex (IL) region of the vmPFC project to an intercalated cell cluster (ITC) in the amygdala (Vertes, 2006). The ITC consists of GABAergic cells that inhibit output from the central nucleus (Berretta et al., 2005). Thus, stimulation of ITC cells inhibits conditioned fear responses. Although we have conducted far less work here, stressor control also appears to activate this mPFC-to-amygdala pathway. It is perhaps not surprising that exposure to IS facilitates later fear conditioning, a phenomenon reported by a number of laboratories (Rau et al., 2005). It would not have been surprising if having control, ES, simply failed to alter later fear conditioning. However, ES actually retarded fear conditioning occurring 7 days later and also facilitated fear extinction (Baratta et al., 2007, Baratta et al., 2008). As would be expected from the research already summarized, inhibition of the mPFC during ES prevented the subsequent inhibition of fear. Interestingly, ES did not interfere with fear learning, but rather fear expression. This is suggested by an experiment in which subjects were first exposed to ES (or IS) and then 7 days later given fear conditioning. Fear conditioning was assessed 24 h after conditioning by exposing the subjects to the fear cues. As previously demonstrated, prior ES resulted in reduced fear on the test day. However, inhibition of the mPFC with muscimol before the test restored fear to normal levels in ES subjects (Baratta et al., 2008). This means that the fear conditioning must have proceeded normally after ES, otherwise how could normal levels of fear be unmasked at the time of testing? ES-inhibition of fear expression is consistent with the argument that the fear inhibiting effects of ES are mediated by an IL-to-ITC pathway, given that the ITC inhibits central nucleus output.

Clearly, the implication is that the ES experience inhibits later fear expression, an effect mediated by the mPFC. This conclusion would suggest that prior ES should facilitate fear extinction, in addition to retarding acquisition, and this proved to be the case (Baratta et al., 2007). It should be noted that these experiments did not attempt to distinguish whether the effects of ES on later fear conditioning and extinction are mediated by the PL versus IL regions of the vmPFC. A large body of work indicates that it is IL projections to the amygdala that mediate fear response inhibition (Sierra-Mercado et al., 2011). We have not done retrograde labeling from the amygdala as we described above from the DRN, but the expectation would be that ES activates IL neurons that project to the amygdala. More work needs to be done, but it would appear that the experience of control over an intense stressor blunts later amygdala-related processes in a manner similar to its modulation of the DRN.

6. Control does not modulate all reactions to adverse events

It is common to conceptualize factors that lead to vulnerability or resistance/resilience as operating with a “broad brush”, modulating all or most reactions to the stressor. The thinking is often that the adverse event itself is sensitized or blunted. However, it is important to understand that the presence of control does not block or even reduce all of the behavioral sequelae of IS, let alone other types of changes. For example, IS produces a profound and persistent reduction in running wheel activity in animals that live with a wheel attached to their home cage, but ES produces a reduction that is as large and as persistent (Woodmansee et al., 1993). Furthermore, stressors also produce a set of physiological and endocrine changes, and some of these are also insensitive to control. For example, hypothalamo-pituitary-adrenal (HPA) activity is not modulated by control, at least in the paradigm described above. Thus, neither the peak nor the decay timecourse of plasma ACTH or corticosterone are reduced by control (Maier et al., 1986). Consistent with these findings, ES and IS produce identical increases in corticotrophin releasing hormone (CRH), arginine vasopressin (AVP), enkephalin, and neurotensin mRNA in the paraventricular nucleus of the hypothalamus (PVN) (Helmreich et al., 1999). Similarly, IS increases circulating thyroid hormones, but ES does so to the same extent (Helmreich et al., 2012). Autonomic measures show a similar pattern, with ES and IS producing the same size increases in core body temperature, heart rate, mean arterial pressure, systolic blood pressure, and diastolic blood pressure (Thompson et al., 2013). We have also examined a number of peripheral immune measures, and they are also not modulated by stressor control (Maier and Laudenslager, 1988). This does not mean that a paradigm cannot be found in which control reduces these stressor-induced changes, but it does not do so in the very same paradigm in which control blunts other behavioral and neurochemical outcomes.

The implication is that control, and perhaps other processes that lead to vulnerability or resistance/resilience, do not operate as a generalized sensitizing or damping switch, but rather operate on a specific neural circuit, and only responses to stressors that are modulated by that circuit will be affected. If it is true that control is detected by the mPFC and then operates by activating output pathways that modulate the DRN, amygdala, and perhaps other structures, only stressor driven changes controlled by those mPFC modulated structures can be blunted (or enhanced). The stressor-induced responses that are unaffected by control seem to be hypothalamically mediated, and mPFC projections to the hypothalamus emanate from a quite different part of the mPFC than do the projections to the DRN and amygdala (Gabbott et al., 2005). Moreover, projections to the PVN are indirect, via the bed nucleus of the stria terminalis (Spencer et al., 2005). Although the argument is admittedly circular, perhaps control does not activate projections to the hypothalamus, or does so only weakly. Or, perhaps, the tailshock stressor is so intense that hypothalamic activation is so powerful that it cannot be readily modulated.

7. All factors that produce resistance/resilience do not do so via actions at the mPFC

It is tempting to consider that all factors that lead to resistance/resilience do so via a common mechanism. However, the data suggest that this is not so (Christianson and Greenwood, 2014). For example, we (Christianson et al., 2008a) and others (Rogan et al., 2005) have studied the mechanism(s) by which safety signals blunt the consequences of stressor exposure. Safety signals are external cues that are followed by the absence of the stressor, that is, cues that predict a time interval free from an adverse event. In these experiments shocks appear periodically, but a tone or a light signals that there will be no shock for a period of time. If there is no signal present shock can occur at any moment, but when the signal is present the organism is safe. Other experimental groups receive identical shocks and tones or lights, but the stimuli are randomly related to the shocks and have no predictive value. The presence of such safety cues blunt the behavioral impact of the shocks as does control, but the mPFC does not mediate the protective effects of the safety signals. Inactivation of the mPFC does not diminish the effects of safety signals, but instead the insular cortex is required (Christianson et al., 2008b). However, insular cortex inactivation does not reduce the beneficial effects of control, providing a double dissociation. Recall that we have argued that immunization against future stressors is mediated by mPFC plasticity, and the safety signals, which do not utilize the mPFC, also do not produce immunization. That is, even though the provision of safety cues reduce the impact of the stressor being experienced, it does not reduce the impact of future stressors (Christianson and Greenwood, 2014).

Voluntary exercise provides another example. Access to a running wheel for 4–6 weeks blocks the typical DRN activation and behavioral effects (shuttlebox escape deficits, potentiated fear conditioning, reduced juvenile investigation, etc) of IS (Greenwood et al., 2003). However, mPFC lesions do not reduce the stressor-blunting effectiveness of exercise (Greenwood et al., 2013), and exercise appears to act directly on the DRN, upregulating somatodendritic 5-HT1A receptors so that autoinhibiton of these cells is enhanced. The prediction would be that the effects of exercise on DRN-mediated behavioral effects would only persist as long as these receptors remain downregulated. Of course, exercise alters many other processes as well.

If different resistance/resilience inducing factors are mediated by different mechanisms, then it might be expected that each factor will blunt a unique set of reactions to adverse events. For example, it was noted above that behavioral control does not modulate the HPA reaction to the stressor, but exercise, which does not exert its effects via the mPFC, does blunt HPA responses to subsequent stressors (Hare et al., 2014). Each consequence of stressor exposure is proximately controlled by its own neural structure or circuit, and different resistance/resilience inducing manipulations will impact on these with different patterns, depending on the circuit that these manipulations utilize. It is not a matter of too much or too little of a transmitter, a hormone, etc., but rather a specific neural circuit.

8. Caveat

It should be noted that not all of the reported data studying the effects of IS, or ES-IS comparisons point to the same characteristics and mechanism(s). This review has concentrated on data from our own laboratory, in large part because the procedures are constant across behavioral and mechanistic studies. A review of all the data is beyond the scope of this review, but there are reasons to argue that the differing procedures across laboratories produce different phenomena that are mediated by differing mechanisms. For example, escape testing has often been conducted in the same apparatus as the one used to deliver IS. Typically, inescapable footshocks are delivered while the subject is confined to one side of a shuttlebox, and then later learning to cross the shuttlebox to escape or avoid is assessed. In contrast, our laboratory always tests for behavioral changes in an environment very different from that in which IS is delivered. One procedure is not superior to the other, but they do seem to produce different phenomena mediated by different mechanisms. In addition to any activation of DRN 5-HT neurons produced by IS, IS also has other effects such as conditioning fear to environmental contextual cues. Greenwood et al. (2010) have argued that when testing for escape is in the same environment as that in which IS has occurred, poor shuttlebox escape could be caused by fear-induced freezing. However, when testing is in a different environment, context fear-induced freezing is not a factor. Indeed, subjects do not freeze before the first shuttlebox shock when the IS has been delivered in wheel-turn boxes, as in our studies (e.g., (Maier et al., 1995b)). This dichotomy could explain why the shuttlebox escape deficit assessed after IS in wheel turn boxes persists for only a few days, while it is quite persistent when IS has been administered in the shuttleboxes (Maier, 2001). DRN 5-HT sensitization persists for only a few days, while fear conditioning is long-lasting. In support of this argument, Greenwood et al. (Greenwood et al., 2010) found that amygdala lesions given after IS eliminate the long-lasting shuttlebox escape deficit that follows IS delivered in the shuttlebox, but has no effect on the shorter-term trans-situational deficit. It might also be mentioned that laboratories differ in their use of fixed electrode versus gridshock as the means to deliver the putatively uncontrollable shocks, and we have found these to sometimes produce different outcomes, likely because the possibility of some behavioral control over the experienced intensity of gridshock is inevitable.

9. Human research on behavioral control and the mPFC, and relationship to clinical phenomena

There is a long history of research that has studied the impact of behavioral control in humans, with control being shown to blunt a variety of outcomes of aversive stimulus exposure (Abramson et al., 1978). However, only recently has control been manipulated in the context of neuroimaging. A number of studies employing painful stimulation have found that providing control, or inducing perceived control, reduces the experienced intensity of the painful stimulus. More importantly for this review, perceived control in these pain studies increased mPFC activity (Salomons et al., 2004). A more direct human analog has been provided by Kerr et al. (2012). These investigators reasoned that the anxious anticipation of negative events is a key factor in psychiatric disorders, and that perhaps the perceived controllability of the anticipated event is a major factor that modulates the degree of anxiety experienced. Furthermore, based on the animal work reviewed above, they suspected that the vmPFC might be engaged by control and inhibit amygdala activity in top–down fashion. Their subjects were snake phobics and were exposed to both snake and neutral fish videos. Stimulus checks confirmed that the snake videos were indeed highly aversive for these subjects, and the fish videos were not. Each trial began with an anticipation period of variable duration in which a cue signaled that a snake video or a fish video might follow in that trial. A second cue indicated that the participant would have control over whether the video (either snake or fish) would occur on that trial, or would not have control on that trial. Then, after a variable period of time, a response target occurred and the subject was instructed to press it as rapidly as possible. The video or a fixation point then appeared. On a controllable trial subjects were told that if they responded fast enough the fixation point rather than the video would appear, but if they were too slow they would see the video. On uncontrollable trials the subjects were told that regardless of how fast they pressed, the video and the fixation point would each occur half the time, but were asked to press as fast as possible anyway. In actuality, the speed required on controllable trials was adjusted so that the subjects succeeded about half the time in avoiding the video, and the actual frequencies on the uncontrollable trials was equated to this frequency. Thus, the controllable and uncontrollable trails were accurately yoked, as in animal studies. Importantly, questionnaire data indicated that the subjects perceived the controllable trials as controllable and the uncontrollable trials as uncontrollable. A variety of results were obtained, but most important here, there was one condition that selectively engaged fMRI vmPFC activity—snake controllable trials. Control did not increase vmPFC activity on neutral fish trials, even though the subjects pressed. vmPFC activity was higher on snake controllable trials than in any of the other conditions. Finally, there was a negative relationship between vmPFC and amygdala activity on snake trials. These findings provide strong support for generalizing the animal data reviewed above to humans.

One of the more surprising results in our animal work was that the experience of control over a stressor is not just neutral with regard to later fear conditioning, but rather retards conditioning and facilitates extinction. Hartley et al. (2014) have very recently reported a direct human verification. Subjects were first exposed to escapable, yoked inescapable, or no shock. Again, questionnaire data indicated that the subjects given control perceived that they did have control, relative to the other groups. Fear conditioning followed by fear extinction occurred 7 days later, followed by an extinction recall test on the next day. The conditions during fear training were quite different than during IS and ES, and even occurred in a different room. It is difficult to assess whether IS or ES altered fear acquisition, as data was presented only for late acquisition, late extinction, and extinction recall. All groups showed strong fear to the fear CS during late acquisition, as assessed by skin conductance. As expected, fear was augmented by IS during late extinction and extinction recall. The key finding was that as in the animal work, ES subjects showed facilitated extinction and extinction recall, relative to the no previous shock condition. Interestingly, there was a strong correlation between the extent to which an ES subject believed that she had control and the reduction in later fear expression, and this included fear acquisition trials, again providing a compelling analogy to the rat data.

At a broader level, there is a large literature directed at understanding the ability to regulate negative emotions that further supports the role here proposed for the vmPFC. A number of studies have shown that a persons deliberate reduction of negative affective responses to negative stimuli increases activity in lateral and dorsal regions of PFC, while decreasing activity in the amygdala (Beauregard et al., 2001, Ochsner et al., 2004, Schaefer et al., 2002). Urry et al. (2006) noted that these regions of PFC do not project to the amygdala, but that they do project to the vmPFC, which does project to the amygdala. Subjects were shown negative or neutral pictures, and asked on separate trials either to increase the negativity (e.g., in response to a picture of a snake imagine it crawling up your leg), decrease the negativity (e.g., view the situation as fake), or simply attend to the picture. A variety of manipulation checks assessed the subject's ability to increase and decrease the negative emotion. The striking result was that in the decrease condition, there was a strong negative correlation between vmPFC and amygdala activity. Those subjects that were successful in decreasing negative emotion and decreasing amygdala activity showed strongly increased vmPFC activity, and a number of analytic procedures suggested that the vmPFC mediated the negative correlation between dorsal/lateral PFC activity and amygdala. Indeed, Urry et al. (2006) concluded that top–down inhibition of the amygdala by the vmPFC is a major mechanism by which cognitive factors can decrease negative emotional reactions. From our perspective, emotion regulation is a form of control, albeit internal.

The animal research that we have reviewed also articulates well with recent research concerning both affective and anxiety disorders. In both cases there has been a convergence of work implicating mPFC dysregulation. Clearly, both types of conditions involve a failure to regulate affect in effective ways, and the mPFC is a driver of such regulation. An extensive neuronal network has been implicated in depressive and anxiety disorders, and a consideration of this work goes well beyond this review. However, it has been suggested that for both PTSD (Hartley and Phelps, 2010, Koenigs and Grafman, 2009, Shin and Liberzon, 2010, Stevens et al., 2013) and depression (DeRubeis et al., 2008, Rive et al., 2013) that limbic hyperactivity is a key alteration, with mPFC hypoactivity being a cause as top–down inhibition is thereby diminished. The fact that this sort of model has been proposed for two different DSM categories is not problematic since there is considerable co-morbidity between categories. Indeed, it may be that reduced mPFC inhibition of stress-responsive limbic and brainstem structures is the type of dysregulated biopsychological dimension that is envisioned by the RDoc effort (Cuthbert and Insel, 2013).

The work reviewed in this paper may provide some insight with regard to therapies. The two major treatments for depression, for example, are anti-depressant medications (ADM) such as selective serotonin reuptake inhibitors (SSRIs) and cognitive therapy (CT). A number of reviews and meta-analyses have indicated that both are effective in reducing depressive symptoms, but that relapse after discontinuation is much higher following ADM than CT (Hollon et al., 2005). That is, CT has a more enduring protective impact. In CT patients are taught to identify the thoughts and images that lead to aversive emotional reactions, and to examine and re-evaluate the validity of these beliefs. Thus, the patient is taught how to reduce the negative emotions that they often experience. From the present perspective, this training has a strong element of perceived control—the patient is taught that they can reduce the negativity of their emotions and experiences by using the techniques of thought re-evaluation that they are being trained to perform. It has been argued (DeRubeis et al., 2008) that this process would engage the mPFC, leading to top–down inhibition of limbic structures. Our work would suggest that this might induce long-lasting plasticity in the mPFC, thereby producing enduring positive effects. Although speculative, perhaps ADM acts directly on limbic structures, or even at the PFC, but does not lead to plasticity, resulting in effects that are not enduring.

10. Summary and conclusions

For over 40 years (Seligman and Maier, 1967, Weiss, 1968) it has been known that the presence of a stressor-controlling response, in the form of an escape response, blunts the impact of the stressor being experienced. However, the mechanism(s) by which this occurs has remained a matter of debate. Maier and Seligman (1976) originally argued that it was learning about uncontrollability that led to the behavioral consequences of uncontrollable stressors, and that controllable stressors simply failed to lead to the learning of uncontrollability, and so did not produce the behavioral outcomes. By this view control is a passive factor, but the research reviewed above clearly goes counter to this idea. At roughly the same time Weiss (1971) argued that the proprioceptive feedback from the escape/coping response is paired with shock termination, and in essence, becomes a safety signal, thereby reducing the fear in the situation. Indeed, Minor et al. (1990) demonstrated that providing a safety signal mitigated the effects of IS, just as does control. However, the work reviewed above suggests that although safety signals are indeed effective, the mechanism by which they blunt the impact of adverse events is different than the mechanism that mediates the impact of behavioral control.

Instead, the current evidence suggests that the controlling escape response engages the corticostriatal act/outcome learning circuit, which then engages mPFC top–down inhibition of brainstem and limbic stress-responsive structures. It should be highlighted that control was not stress-blunting if either the PL or the DMS was inactivated during the ES exposure thereby preventing the engagement of corticostriatal act/outcome circuit, even though the subjects turned the wheel and escaped with the same latencies as did subjects from whom neither structure was inactivated. The escape response was learned and performed without deficit, presumably by engagement of the habit system, but the impact of the stressor was as if it was inescapable. Clearly, it is not just turning the wheel and terminating shock, or even learning of the response per se that is critical—it is engagement of the PL-DMS act/outcome circuit, which then leads to mPFC inhibition of the DRN, amygdala, etc.

Activation of the PL-DMS machinery also leads to plasticity. ES increases the excitability of PL neurons, and after exposure to ES, later IS activates this system, which it would not do without the prior ES experience. These changes lead to behavioral and neurochemical immunization, and require the production of new proteins, NMDA activity, and ERK phosphorylation in the PL. Importantly, it is not just activation of the act/outcome system, but rather activation of the system in the presence of an adverse event that is required. It is as if the two become tied together in some fashion. It is as if the system, once having experienced control over a very potent event, is biased towards controllability being present in the future. If an adverse event can be mitigated in some fashion by active behavior, then it is likely best to do so. However, if an aversive event is uncontrollable, then passivity/withdrawal and the emotions (e.g., helplessness, fear) that mediate passivity may well be adaptive. This would allow the organism to conserve resources until active coping becomes possible. Indeed, subjects that are exposed to uncontrollable stress become hyper-vigilant in the sense that they show exaggerated attention to external cues rather than the internal cues which guide instrumental responding (Minor et al., 1984). This sort of process might increase the odds of the organism detecting any change in circumstances. Perhaps if there has been a history that adverse events are controllable, it is reasonable in a new situation for the organism to continue attempts at active coping for a longer period of time than had the control experiences not occurred previously. The neural mechanisms proposed here would lead to this scenario.

If, as argued here, the mPFC can exert inhibitory control over limbic and brainstem stress-responsive structures, and if there is plasticity in this circuitry initiated by control, then a number of clinical implications can be drawn. Strengthening of these pathways would lead to reduced passivity/withdrawal and the emotions that drive these behaviors, and weakening these pathways would have the opposite effect. If part of resistance/resilience is the maintenance of active coping in the face of adverse circumstances, then teaching individuals that they can influence what happens to them, how they feel, and how others see them, might alter how they respond to future adverse events in the direction of resistance/resilience.