Animal Models of (or for) Aggression Reward, Addiction, and Relapse: Behavior and Circuits

Abstract

Inappropriate and pathological aggression plays a leading role in the suffering and death of millions of people, and further places an untenable strain on the caregivers and families of those afflicted. In some cases, such as addictive drugs, aggression can be highly rewarding (appetitive) and continually pursued despite short- and long-term negative consequences. Similarly, recidivism (relapse) rates for repeat violent offenders are as high as relapse rates for drug addicts. Appetitive aggression and relapse to aggression seeking can be modeled in mice studies using conditioned place preference and self-administration procedures followed by a period of abstinence and subsequent tests for relapse to aggression preference and aggression seeking. These procedures allow for the study of the mechanisms that control the appetitive versus the consummatory (attack) phases of aggressive behavior. In this review, we first discuss the behavioral procedures developed to probe appetitive aggression in mouse models, spanning from Pavlovian to operant tasks, and we also describe the recently proposed phenomenon of “aggression addiction.” Next, we discuss the pharmacological and circuit mechanisms of aggression conditioned place preference and aggression self-administration, seeking, and relapse, highlighting mechanistic congruence and divergence between appetitive and consummatory phases of aggression. We conclude by discussing clinical implications of the studies reviewed.

Introduction

The past decade has seen a resurgence of preclinical aggression research using mice as the experimental subjects (Fig. 1A,B). In large part, this is due to the development of optogenetic (Boyden et al., 2005) and chemogenetic (Armbruster et al., 2007) methods that can be used in transgenic mice (Gordon et al., 1980) to study the role of genetically defined neural populations and circuits in innate and learned aggression (Anderson, 2012). This resurgence is also due to renewed interest in developing ethologically relevant behavioral procedures (Miczek and O'Donnell, 1978; Blanchard et al., 2003) that allow researchers to better capture complex social behaviors (Keifer and Summers, 2016; Garner et al., 2017; Krakauer et al., 2017). In this regard, different animal models of (or for per the new National Institute of Mental Health terminology) human aggression are needed to capture the complex spectrum of this behavior that includes both adaptive reactive and instrumental aggression, as well as appetitive and often pathological aggression (Moran et al., 2014; de Almeida et al., 2015; Chester and DeWall, 2016).

Figure 1.

Trajectory of PubMed citations on aggressive behavior across three model species. Methods were adapted from Blanchard et al. (2003). We found the number of citations in response to the search terms “aggressive behavior AND mouse” (or “… AND rat” or “… AND hamster”). We binned the number of citation counts every 5 years starting from 1970. A, The number of cumulative citations since 1970. B, The number of citations within each 5 year bin.

At the extreme, pathological aggression mimics cardinal features of drug addiction, such as being highly rewarding and strongly pursued despite immediate or long-term adverse consequences (Porges and Decety, 2013; Chester and DeWall, 2017). Additionally, relapse (recidivism) rates of repeat violent offenders are similar to those observed in drug addicts (Hunt et al., 1971; Sinha, 2011; Durose et al., 2014). We and others proposed that preclinical rodent models can recapitulate components of pathological aggression (Kudryavtseva, 2004; Golden and Shaham, 2018) previously postulated to be uniquely human (Elbert et al., 2018).

In this review, we first describe behavioral procedures used to study appetitive aggression. These include aggression conditioned place preference (CPP) (Golden et al., 2016), aggression self-administration (Fish et al., 2002), relapse to aggression seeking after forced or voluntary (food choice- or punishment-induced) abstinence (Golden et al., 2017a), and a variation of the Diagnostic and dtatistical manual of mental disorders, 4th edition (DSM-IV) rodent addiction model (Deroche-Gamonet et al., 2004) that we have developed to study compulsive aggression “addiction” in mice (Golden et al., 2017a). Next, we describe pharmacological and circuit-related studies on the mechanisms of aggression CPP and operant aggression self-administration and seeking. We conclude by describing the implications of the reviewed studies to pathological aggression in humans. In Table 1, we summarize the main findings of studies using CPP and operant aggression self-administration methods. Due to space limitations, we do not describe results from recent studies in which cell-type and circuit-specific optogenetic and chemogenetic methods were used to elicit unconditioned aggressive behavior in different brain areas. We refer the readers to excellent recent reviews on this topic, which provide overviews summarizing these brain regions and circuits (Miczek et al., 2015; Aleyasin et al., 2018a; Yamaguchi and Lin, 2018; Flanigan and Russo, 2019).

Table 1.

A selected list of published papers on aggression motivation in mice using Pavlovian conditioning and operant-based self-administration/relapse models and their major findings. The data are based on PubMed research.

Year	Citation	Behavior	Major findings
Historical Papers
1964	(Lagerspetz, 1964)	Conflict (electric barrier)	Male mice bred for aggressiveness crossed an electric grid to a submissive mouse more readily if permitted to fight immediately before the trial.
1969	(Tellegen et al., 1969)	T-maze	Male BALB/cJ mice acquired, extinguished, and reversed a position preference in a T-maze when the opportunity to attack a submissive mouse was a reinforcer.
1970	(Legrand, 1970)	Runway	Male BALB/cJ mice ran aross a runway to defeat a submissive mouse as a reinforcer. High aggressiveness ratings and a brief fight immediately prior facilitated running behavior.
1972	(Tellegen and Horn, 1972)	T-maze	Male mice from three inbred strains (BALB/cJ, RF/J, and SJL/J) acquired a position preference in a T-maze when the opportunity to attack a nonaggressive mouse was a reinforcer.
Aggression Conditioned Place Preference (CPP)
1995	(Martinez et al., 1995)	Biased CPP	Male OF-1 outbred mice form CPP to an intruder-paired context. However, the analysis for this preference was carried out in a context biased manner.
2016	(Golden et al., 2016)	Unbiased CPP	Male CD-1 outbred mice form CPP to an intruder-paired context. GABAergic forebrain projections to the lateral habenula (LHb) bidirectionally control aggression CPP but do not control unconditioned aggression. Direct LHb manipulation does the same.
2017	(Golden et al., 2017b)	Unbiased CPP	Male CD-1 mice show persistent aggression CPP across time, and non-aggressive mice switch phenotypes. Hybrid F1 generation transgenic mice show strong unconditioned aggression.
2018	(Aleyasin et al. (2018b)	Unbiased CPP	Male CD-1 mice exhibit increased ΔFosB in NAc Drd1-MSNs after repeated aggressive encounters. In hybrid F1 transgenic mice induction of ΔFosB in Drd1-MSNs of the NAc increases aggression severity without effecting CPP. In contrast, ΔFosB induction in Drd2-MSNs reduces aggression CPP without affecting the severity of aggression.
2018	(Stagkourakis et al. 2018)	Unbiased CPP	Activation of dopamine transporter-expressing neurons in the hypothalamic ventral premammillary nucleus (PMvDAT neurons) triggers attack behavior; silencing these neurons interrupts attacks. PMvDAT projections to the ventrolateral part of the ventromedial hypothalamic and the supramammillary nuclei control attack execution and aggression reward, respectively.
2018	(Flanigan et al. 2018)	Unbiased CPP	In CD-1 mice, orexin neurons from the lateral hypothalamus activate a small population of GABAergic interneurons in the LHb via orexin receptor 2. Stimulation of this projection enhances aggression severity and aggression CPP.
Aggression Self-administration (SA) and Relapse
2002	(Fish et al., 2002)	SA (FR and FI)	First published study of operant aggression SA in mice. Male CFW outbred mice were maintained on a FR10 or FI10 reinforcement schedule; GABA(A) positive modulator heightened aggression severity and operant responding in a dose-dependent manner.
2005	(Fish et al. 2005)	SA (FI)	In male CFW mice, corticosterone elevations are required for operant responding motivated by aggressive behavior and for escalated aggression that follows this responding. Corticosterone elevations appear to inhibit the aggression heightening effect of GABA(A) receptor positive modulators.
2007	(Bannai et al., 2007)	SA (FI)	In male CFW mice, 5-HT(1B) agonists CP-94,253 and CP-93,129 reduced escalated aggression towards the intruder at doses lower than those required to affect operant responding.
2008	(Fish et al., 2008)	SA (FI)	In male CFW mice, 5-HT(1B) agonist CP-94,253 reduced operant responding for aggression, alcohol-induced aggression, drinking, and wheel running. Of these behaviors, alcohol-heightened aggression is the most sensitive to the 5-HT(1B) receptor agonist.
2008	(Couppis and Kennedy, 2008)	SA (VR)	Male CFW mice were trained on a VR5 reinforcement schedule of aggression SA. NAc injections of Drd1 and Drd2 antagonists microinjected inhibited operant responding, but also caused sedative effects at higher doses.
2009	(May and Kennedy, 2009)	SA (FR, FI, PR, DRL)	Male CFW mice were trained on FR, FI, PR and DRL reinforcement schedules for aggression SA.
2016	(Falkner et al. 2016)	SA (FR, PR)	In male CFW mice, VMHvl neurons are active during aggression-seeking and their activity tracks changes in task learning and extinction. Inactivation of the VMHvl reduced aggression seeking, whereas optogenetic stimulation of the VMHvl accelerated immediate aggression seeking and intensified future attacks.
2017	(Golden et al., 2017a)	SA (FR, PR); forced, punished, voluntary abstinence	Male CD-1 mice trained on aggression SA show relapse vulnerability following forced abstinence, punishment-induced suppression, and choice-based suppression of aggression seeking. Cluster analysis of the operant aggression measures identified a subset of compulsive aggressors (∼19%) that scored higher on aggression taking and seeking across all operant measures. Using procedures established to model drug addiction, we showed that a subpopulation of CD-1 mice demonstrate “addiction-like” aggressive behavior, suggesting an evolutionary origin for compulsive aggression.
2018	(Covington et al. 2018)	SA (FI)	Male C57BL/6J inbred mice trained on an FI reinforcement schedule for aggression SA while receiving daily non-contingent alcohol injections. Alcohol augmented FI response rates for aggression SA but suppressed fighting performance. Systemic injections of NMDA or AMPA receptor antagonists (ketamine, dizocilpine, or NBQX) during later challenges with alcohol had minimal effect on alcohol-escalated rates of FI responding.
2019	(Golden et al., 2019)	SA (FR), forced relapse	In male CD-1 mice, aggression SA and relapse testing induced higher Fos expression in NAc shell than in core, while Fos colocalized with Drd1 and Drd2 in both subregions, Chemogenetic inhibition of Drd1-, but not Drd2-, expressing neurons decreased aggression self-administration and relapse. Results indicate a cell-type specific role of Drd1-expressing neurons that is critical for both aggression self-administration and relapse to aggression seeking.

Abbreviations: CPP, conditioned place preference; CFW, Swiss Webster; SA, self-administration; FR, fixed ratio; FI, fixed interval; VR, variable ratio; PR, progressive ratio; DRL, differential reinforcement of low rate behavior reinforcement; LHb, lateral habenula; Drd1, dopamine D1 receptor; Drd2, dopamine D2 receptor; MSN, medium spiny neuron; NAc, nucleus accumbens; PMvDAT, Dopamine transporter-expressing neurons in the hypothalamic ventral premammillary nucleus; VMHvl, ventrolateral part of the ventromedial hypothalamus.

Behavioral methods to study appetitive aggression in mice

Brief history

The notion of aggression reward is not new in behavioral neuroscience. In the early 1960s, Thompson et al. (Thompson, 1963; Thompson and Sturm, 1965a) reported that male Siamese fighting fish (Betta splendens) will perform instrumental responses to attack visual representations of conspecifics. They also reported that conditioned stimuli paired with the visual presentations elicit subsequent operant aggressive behavior (Thompson and Sturm, 1965b; Thompson, 1966). Operant responding in this procedure was potentiated by morphine administration (Braud and Weibel, 1969). Additionally, when the Siamese fighting fish were given a choice between a live conspecific versus inanimate model representation, they selected the live target (Craft et al., 2003). Similar observations were made in both male fighting cocks (Thompson, 1964) and homing pigeons (Cole and Parker, 1971). However, a limitation of these early pioneering studies is that the investigators did not include behavioral measures of conspecific aggressive behavior; that is, there was not any actual physical contact.

The use of mouse models, where physical attack bouts against subordinate intruders, rather than visual or model representations, were used as the reinforcer led to the first clear examples of what we now term “aggression reward.” An early study established that aggressive mice will more readily cross an electric grid to gain access to a submissive opponent if permitted to fight before the trial (Lagerspetz, 1964). Because noncontingent electric shock had already been established as a key aggression-inducing stimulus (Azrin et al., 1964a, b; Ulrich and Craine, 1964), follow-up experiments used T-maze (Tellegen et al., 1969; Tellegen and Horn, 1972; Kelsey and Cassidy, 1976; Legrand, 1978) (Fig. 2A) or runway (Legrand, 1970) procedures to study appetitive aggression seeking. These studies and subsequent studies showed that physical aggression, in the absence of external aversive stimuli, was sufficient to condition mice to prefer aggression-paired contexts (Potegal, 1979; Taylor, 1979). These pioneering studies established the utility of mouse models for the study of appetitive aggression.

Figure 2.

Schematics of different behavioral procedures to study motivated aggression seeking behavior. A, In the T-maze test, dominant mice undergo preliminary aggressive experiences with a subordinate in their homecage. On test day, they are placed at the end of the long arm of the T-maze in a start box. At the ends of the short arms are “correct” or “incorrect” goal boxes. A subordinate mouse is placed at each end of the goal boxes and separated by a partition. Upon choosing the “correct” goal box, the partition separating the subordinate mouse is raised and the dominant mouse can engage in attack. Upon choosing the “incorrect” goal box, the subordinate mouse is removed before the partition is raised, eliminating the possibility of attack. B, In the partition test, the behavior of a dominant mouse is assessed when a subordinate mouse is placed at the opposite end of a box separated by a partition. The partition allows for all forms of sensory contact with the subordinate mouse, except for tactile contact. Approach behaviors and time spent in the “interaction zone” are recorded. C, In the CPP test, dominant mice are conditioned to two different contextual chambers, with one chamber paired to the presence of a subordinate mouse and the other chamber serving as an unpaired control. On test day, dominant mice are placed in a middle chamber connecting both contextual chambers. Time spent in the paired chamber compared with the unpaired chamber is measured. D, Operant approach used by Covington et al. (2018). A nose-poke apparatus is inserted into the dominant mouse's homecage. Nose-pokes in the active port are reinforced on a fixed-interval schedule (FI) with presentation of an intruder mouse into the homecage. An aggression-paired houselight illuminates upon insertion of an intruder. The other port serves as an inactive control. Bottom, A schematic of the FI trial design. E, Operant approach used by Falkner et al. (2016). As in D, a noseport panel containing two ports with infrared detectors is inserted into the dominant mouse's homecage. However, nose-pokes in the active port are reinforced on a fixed ratio-1 (FR-1) reinforcement schedule. Bottom, A schematic of the FR-1 trial design. F, Operant approach used by Golden et al. (2017a). Behavior is assessed in an operant chamber with an active and inactive lever. Active lever presses are reinforced on an FR-1 reinforcement schedule. A successful lever press results in sounding of a discriminative tone and opening of an automated guillotine door housing an intruder on the opposite side. An intruder is then guided into the operant chamber. Bottom, A schematic of the trial design. F, Adapted from Golden et al. (2017a).

After these early studies in the 1960s-1970s, most neuropharmacological research on aggression used variations of the resident-intruder (Miczek and O'Donnell, 1978) and sensory contact (Kudryavtseva et al., 1991) procedures where an intruder mouse is placed within the home-cage of a resident mouse and agonistic encounters are recorded (Miczek et al., 2002; de Almeida et al., 2015). However, from the perspective of appetitive aggression or aggression reward, the resident-intruder and sensory contact procedures are limited by the difficulty in determining whether the observed behaviors reflect defensive or reactive aggression, instrumental aggression, or appetitive (rewarding) aggression (Golden et al., 2017a). [It is also important to note that the preclinical aggression procedures described below are designed to minimize harm to both aggressive and nonaggressive mice; all procedures are immediately ceased at the first sign of physical harm.] In the section below (Aggression CPP), we describe Pavlovian (CPP) and operant (self-administration) procedures that have been used to study learned appetitive aggression in mice.

Aggression CPP

The CPP procedure has been used to study the rewarding effects of abused drugs for many years (Beach, 1957; Mucha et al., 1982). In this procedure, one distinct context is paired with drug injections, whereas another context is paired with vehicle injections. During a subsequent drug-free test, the laboratory animal chooses between the drug- and the vehicle-paired contexts. An increase in preference for the drug-paired context is indicative of the drug's rewarding effects (Bardo and Bevins, 2000).

Based on earlier studies in female Syrian hamsters (Meisel and Joppa, 1994) and male OF-1 mice (Martínez et al., 1995), we adapted a CPP procedure in combination with the resident-intruder social defeat procedure (Miczek and O'Donnell, 1978; Kudryavtseva et al., 1991; Golden et al., 2011) to study aggression reward in CD-1 mice (Golden et al., 2016) (Fig. 2C). We and others have selected outbred male mice as experimental subjects due to their long and consistent categorization as innately aggressive (Connor, 1975; Jones and Brain, 1987), far more so than commonly used inbred lines, such as C57BL/6J (Golden et al., 2017b). Such strain differences are also observed between inbred mice (Miczek et al., 2001); therefore, strain selections are a critical aspect of preclinical aggression experimental design. The genetic basis for aggression has been a focus of many studies and falls outside the scope of this review (Sluyter et al., 1996; Nelson and Chiavegatto, 2000; Anholt and Mackay, 2012; Takahashi and Miczek, 2014; Thomas et al., 2015).

In our initial study, we first used the resident-intruder procedure to characterize unconditioned aggression of dominant CD-1 adult male mice toward adolescent submissive C57BL/6J male mice during daily sessions. Next, we performed aggression CPP training by repeatedly placing the CD-1 mice in both contexts of the CPP apparatus and introducing a C57BL/6J mouse to one of those contexts, creating two distinct contexts: intruder-paired and intruder-unpaired. One day after CPP training, we tested the CD-1 mice for aggression CPP by giving them access to both contexts. We found that CD-1 mice that initially exhibited unconditioned attacks on the C57BL/6J mouse during the screening phase (∼70% of the mice) developed aggression CPP, whereas those that did not attack during screening (∼30%) did not (Golden et al., 2016).

In the study described above (Golden et al., 2016), we focused on mice categorized as “aggressors” or “nonaggressors” in the resident-intruder procedure. In actuality, aggression falls along a continuum in mice, and based on this notion, we further characterized individual differences in aggression CPP by testing a third phenotype, termed “variable aggressors,” composed of mice that performed inconsistently when repeatedly tested in the resident-intruder procedure (Golden et al., 2017b). Like the aggressive mice, the variable aggressive mice exhibited significant, although weaker, aggression CPP. This observation suggests that repeated unconditioned aggression experiences can transform nonrewarding aggressive encounters into a subsequently rewarding experience. To test this idea, we exposed a large cohort of nonaggressive CD-1 mice to 10 d of repeated resident-intruder sessions and then tested them for aggression CPP. We found that a subset (∼50%) of the nonaggressive mice began to exhibit unconditioned aggression in the resident-intruder test, which increased in severity over time. This subset of mice also showed aggression CPP that was similar in magnitude to those of the innately aggressive mice who immediately attack during all resident-intruder sessions (Golden et al., 2017b).

These findings are reminiscent of the “winner effect” phenomenon, the observation that once aggression results in a victorious social encounter, the winner mice are more likely to continue successful aggressive bouts (Ginsburg and Allee, 1942; Kudryavtseva et al., 2004; Oyegbile and Marler, 2005). This phenomenon is conserved across many species, invertebrate to vertebrate (Hsu et al., 2006), and is closely linked with the establishment of social dominance and subordinate hierarchies (Zhou et al., 2018). As such, there is a strong adaptive component to the winner effect (Ghosal et al., 2019), as social hierarchies determine access to resources, territory, and reproductive partners.

In our study, we also examined whether aggression CPP progressively increases or “incubates” after CPP training (Golden et al., 2017b). This research question was inspired by two incubation-related phenomena termed “incubation of drug craving” and the “fighting deprivation effect.” The first phenomenon refers to the time-dependent increase in drug seeking after withdrawal from drug self-administration in rat models (Grimm et al., 2001; Venniro et al., 2016). The second refers to the behavior of aggressive mice who increase their aggressive behavior toward previously defeated intruder mice after periods of separation (Kudryavtseva, 2004; Kudryavtseva et al., 2011). However, while we found persistent aggression CPP for up to 18 d after CPP training in both the aggressive and variable aggressive mice, there was no evidence for incubation of learned aggression preference in the CPP model (Golden et al., 2017b).

Aggression self-administration

The Miczek group was the first to develop the mouse aggression self-administration model. They designed a modular operant conditioning panel that includes active and inactive nose-poke devices and introduced the panel into the homecage of aggressive resident outbred CFW mice (Fig. 2D). Using this operant system, they showed that mice will nose-poke to gain access to attack subordinate intruder mice under both fixed-ratio (up to fixed-ratio 10) and fixed-interval (up to fixed-interval 10 min) reinforcement schedules (Fish et al., 2002, 2005; Bannai et al., 2007). These findings were replicated and extended by Kennedy and colleagues (Couppis and Kennedy, 2008; May and Kennedy, 2009) who reported reliable operant aggression self-administration in mice under progressive ratio, differential reinforcement of low rate behavior, and variable ratio reinforcement schedules. Together, these early studies established reliable operant aggression self-administration procedures in mice.

More recently, Falkner et al. (2016) adopted and optimized a self-initiated aggression task based on the method developed by the Miczek group (Fig. 2E). In this variation, the resident mouse selects a “social” or “null” nose-poke port to initiate each self-administration trial at its own pace during each 35 min training trial over 6–10 d. Notably, when nonsubmissive intruder mice were presented contingently after nose-pokes, the resident mice rapidly decreased nose-poking and only resumed the operant response after subsequent presentations of subordinate intruders. These data support the notion that winning an aggressive encounter is a critical component of aggression reward.

Most recently, we customized a standard Med Associates chamber to introduce a high throughput and less labor-intensive method for operant aggression self-administration (Fig. 2F). Using this apparatus, we showed robust operant self-administration in a subpopulation (∼60%-70%) of adult male CD-1 mice that lever pressed to attack an adolescent subordinate C57 mouse, and further used the apparatus to study “aggression addiction” and relapse (Golden et al., 2017a), as described below.

Aggression addiction and relapse

As mentioned above, pathological aggression in humans is often highly rewarding and pursued despite adverse consequences, and relapse (recidivism) rates among violent offenders are very high; these observations mimic cardinal features of drug addiction (Sinha, 2011; Porges and Decety, 2013; Durose et al., 2014). Additionally, like drug addiction, which develops in only ∼20% of people who have experience with addictive drugs (Anthony et al., 1994), pathological aggression develops only in a minority of people who engage in aggressive encounters during their lifetime (Lacourse et al., 2002; Provençal et al., 2015). Based on these observations and inspired by the DSM-IV rat addiction model (Deroche-Gamonet et al., 2004; Piazza and Deroche-Gamonet, 2013) and our studies on relapse to drug seeking in rats (Venniro et al., 2016), we have used our custom-made operant self-administration apparatus to determine whether a subpopulation of CD-1 male mice demonstrate “aggression addiction” (Golden et al., 2017a).

We first combined aggression self-administration with models of relapse after homecage forced abstinence (Pickens et al., 2011), punishment-induced abstinence (Krasnova et al., 2014; Marchant et al., 2019), and food choice-induced voluntary abstinence (Caprioli et al., 2015) procedures. In each case, we observed that ∼70% of adult sexually experienced outbred CD-1 mice acquired robust aggression self-administration when given the opportunity to lever press for a younger subordinate male C57BL/6J intruder. These aggressive mice also showed persistent time-independent relapse to aggression seeking after forced abstinence (Fig. 3A,D) and food choice-induced voluntary abstinence (Fig. 3B,E), as well as time-dependent resumption of relapse to aggression seeking after punishment-induced abstinence (Fig. 3C,F).

Figure 3.

Operant assessment of aggression self-administration, aggression suppression, and relapse to aggression seeking. A, Upward trajectory of reward and attack trials over 9 d of self-administration training on an FR-1 reinforcement schedule. B, Voluntary food-choice suppression during 10 d of mutually exclusive choice sessions for food or aggression. C, Decrease of reward and attack trials over 10 d of punishment-imposed suppression of aggression. Mice received response-contingent shocks on 50% of reinforced lever presses. D, Aggression-seeking (relapse) test after forced abstinence in the homecage. During the relapse test, lever presses were recorded after mice were returned to the self-administration chamber and reexposed to the contextual cues associated with aggression self-administration. E, Aggression-seeking test after food choice-imposed suppression of aggression self-administration. F, Aggression seeking test after punishment-imposed suppression of aggression self-administration. Data are from Golden et al. (2017a). *p < 0.05.

Next, we used an experimental procedure inspired by a DSM-IV-based rat model of addiction (Deroche-Gamonet et al., 2004; Piazza and Deroche-Gamonet, 2013). We trained a large cohort of male CD-1 mice for aggression self-administration, and then tested them for choice-based voluntary suppression, relapse to aggression seeking, responding under a progressive ratio reinforcement schedule, and aggression self-administration despite adverse consequences (punishment). This provided five dimensions for cluster analysis (Fig. 4A): (1) attacks, (2) relapse, (3) aggression choice, (4) aggression progressive ratio, and (5) punishment resilience ratio. We used unsupervised cluster analyses to obtain an unbiased estimate of the number of aggression-seeking phenotypes within the population, and to classify individual aggressive mice into the identified subpopulations. We used two algorithms: a proprietary SPSS classification procedure (TwoStep clustering) to determine the number of clusters in the dataset and to assign every mouse to a cluster, followed by an agglomerative hierarchical algorithm (Ward's method) to validate the initial results (Fig. 4B). This resulted in the identification of two clusters: one that we termed typical aggression seeking (52%) and the second that we termed compulsive aggression seeking (19%) (Fig. 4C). Using the TwoStep assignments, subsequent comparison of the cluster means confirmed that mice classified as “compulsive aggression seeking” were significantly higher on all five aggression-seeking measures (Fig. 4D).

Figure 4.

Cluster analysis of operant aggression-related behaviors reveals that a subset of the population are compulsive aggressors. A, Distributions of individual responses for the five operant aggression measures used for cluster analysis: attacks, relapse, choice, progressive ratio (PR), and punishment. B, Unbiased clustering of operant aggression behavior resulted in two phenotypic clusters: compulsive aggressors and typical aggressors. Shown is a 3D representation of clusters using projection data of the first three principal components of the five measures. C, Pie chart showing phenotypic assignments following cluster analysis. Included are nonaggressors that did not acquire operant aggression. D, Comparison of compulsive and typical aggressor clusters of the measures in A. *Different from the typical aggression group, p < 0.05. PC, Principal component. Data are from Golden et al. (2017a).

Overall, our study identified a subset of mice that exhibited “addiction-like” aggressive behavior characterized by intense operant-reinforced attack behavior, decreased likelihood to select an alternative food reward over aggression, heightened relapse vulnerability and progressive ratio responding, and resilience to punishment-induced suppression of aggression self-administration. Based on these results, we proposed that preclinical addiction models can be used to identify neural mechanisms controlling appetitive aggression and relapse, as well as pathological or compulsive manifestations of aggression.

Conclusions

Aggression can be a learned rewarding experience in subpopulations of male mice of certain strains and can be studied using experimental methods, such as Pavlovian CPP and operant self-administration, which have been used for many years to study learning factors and circuits controlling the rewarding effects of addictive drugs and nondrug rewards. Additionally, behavioral procedures traditionally used to study compulsive drug use and relapse in rodents in the addiction field can be used in mice as models of (or for) pathological aggression seeking in humans, and to study underlying mechanisms.

Neuropharmacological and neuroanatomical mechanisms of appetitive aggression

Systemic neuropharmacological manipulations of neurotransmitters and hormones

GABA(A) receptors.

Positive modulators of the GABA(A) receptor complex increase aggressive behavior in both animal models (Miczek, 1974; Fish et al., 2001) and humans (Bond and Lader, 1988). These compounds also increase operant aggression self-administration. Fish et al. (2002) reported that, in CFW mice trained under a fixed interval reinforcement schedule, a low dose of the GABA(A)-positive modulator allopregnanolone increases nose-poke responding for access to aggressive interactions without changing the severity of aggression bouts. In contrast, a higher dose has an opposite effect, increasing aggressive severity but not operant responding. These data suggest that appetitive aggression seeking is mechanistically dissociable from the consummatory component of aggression.

Corticosterone.

Fish et al. (2005) showed that aggression self-administration increases plasma corticosterone levels and studied whether this stress hormone interacts with the effects of allopregnanolone and midazolam (another GABA(A)-positive modulator) on aggression self-administration using the corticosterone synthesis inhibitor metyrapone (Jenkins et al., 1958). Metyrapone injections decreased both operant responding and aggression bouts. However, although midazolam had no effect on aggression self-administration, combined injections of metyrapone and midazolam increased aggression. Additionally, metyrapone did not prevent the aggression-escalating effect of allopregnanolone. Together, these data suggest that corticosterone contributes to operant aggression, but that aggression-induced elevation of corticosterone inhibits the proaggressive effect of GABA(A) positive modulators. However, these data should be interpreted with caution because of metyrapone's off-target (corticosterone-independent) effects (Jain et al., 1993; Rotllant et al., 2002).

5-HT1B receptors.

Systemic injections of 5-HT1B agonists (Miczek and de Almeida, 2001; De Almeida et al., 2006) decrease alcohol-induced potentiation of unconditioned aggression in rodents and primates (Miczek et al., 1984, 1993). Similarly, Fish et al. (2008) showed that the 5-HT1B agonist CP-94,253 decreases both aggression self-administration and number of attack bouts, with a stronger inhibitory effect on aggression bouts. In a follow-up study, Bannai et al. (2007) confirmed these results with CP-94,253, and also reported that injections of a different 5-HT1B agonist (CP93,129) into the dorsal raphe nucleus (DRN) decrease both operant self-administration and aggression bouts. These results implicate dorsal raphe serotonin as a critical modulator of both learned aggression self-administration and unconditioned aggression.

Glutamate receptors.

Covington et al. (2018) tested the effect of the NMDA receptor antagonists, ketamine and MK801, and the AMPA receptor antagonist NBQX on escalated aggression self-administration induced by repeated noncontingent alcohol exposure (1.8 and 2.2 mg/kg) and subsequent challenge injection of a lower alcohol dose (1 mg/kg) that causes long-lasting (up to 1 month) sensitization of aggressive behavior. While all three drugs decreased aggression bouts, only ketamine decreased alcohol-induced escalated operant aggression. These results provide additional evidence that the mechanisms of appetitive operant aggression seeking and the consummatory phase of aggression (attack bouts) are partially dissociable.

Neuroanatomical mechanisms

NAc.

The NAc is a key hub in the limbic system, and there is evidence that this brain region plays a role in aggressive behaviors. The NAc major projection neurons are GABAergic and express dopamine receptor 1 and/or dopamine receptor 2 (Drd1 and Drd2), which predominantly (Gerfen, 1992), but not exclusively (Kupchik et al., 2015), project along the direct (Drd1) or indirect (Drd2) pathways and generally exert opposite effects on target brain regions. Inbred mice selected for high aggression (NC900 line) show elevated c-Fos immunoreactivity in the NAc (Nehrenberg et al., 2013). In dominant rats, both aggressive behavior and exposure to aggression-associated contexts increase extracellular dopamine levels in the NAc (van Erp and Miczek, 2000; Ferrari et al., 2003). This is presumably driven by dopaminergic VTA projections, as optogenetic activation of the VTA increases the severity of aggressive attacks (Yu et al., 2014). Methamphetamine (a dopamine reuptake blocker and releaser) increases aggression in rodents (Miczek, 1974; Miczek and Haney, 1994), whereas systemic injections of Drd1- or Drd2-family antagonists decrease aggression in resident-intruder procedures (Kudryavtseva et al., 1999; Fragoso et al., 2016). In California mice, systemic injections of Drd1- and Drd2-family antagonists delay the emergence of a resident-intruder winner during repeated agonistic encounters (Becker and Marler, 2015).

NAc dopamine also plays a critical role in aggression self-administration. Couppis et al. (2008) reported that NAc injections of Drd1- or Drd2-family antagonists (SCH23390 or sulpiride) decrease aggression self-administration. SCH23390 decreased operant responding but minimally affected attack behavior, whereas sulpiride decreased both measures, albeit with off-target effects on locomotor activity. More recently, using aggression CPP, Aleyasin et al. (2018b) reported that repeated aggression exposure in the resident-intruder procedure selectively elevates ΔFosB in Drd1-expressing neurons. ΔFosB is a truncated splice product of the FosB gene that is highly stable (Carle et al., 2007) and accumulates in NAc after repeated exposure to drugs, stress, and social environments (Robison and Nestler, 2011; Nestler, 2015). To test the causal role of ΔFosB in aggression reward, Aleyasin et al. (2018b) used a hybrid breeding strategy (Golden et al., 2017b) where transgenic inbred lines of interest (Drd1- and Drd2-Cre) are crossed with outbred CD-1 mice (Fig. 5A) and then the aggressive hybrid D1-Cre and D2-Cre × CD-1 F1 generation is used as the subjects (Fig. 5B). Viral overexpression of ΔFosB in Drd1-expressing neurons increased unconditioned aggression in the resident-intruder task, but had no effect on aggression CPP. In contrast, overexpression of ΔFosB in Drd2-expressing neurons (which do not exhibit elevated ΔFosB levels after aggression exposure) had no effect on unconditioned aggression but decreased aggression CPP.

Figure 5.

NAc Drd1-expressing neurons control aggression self-administration and aggression seeking. A, A schematic of the breeding approach used to generate D1-Cre or D2-Cre F1 hybrid mice. B, Mean latency to first attack bout in resident-intruder screenings of C57, CD-1, C57 × D1-Cre, CD-1 × D1-Cre, C57 × D2-Cre, and CD-1 × D2-Cre mice. F1 hybrids of CD-1 mice show similar latency to attack latencies as CD-1 mice. C, D, D1 hybrid mice were injected with hM4Di or mCherry virus in the NAc. Injection of clozapine, but not vehicle, decreased aggression self-administration and aggression seeking in hM4Di but not mCherry-injected mice. E, F, D2 hybrid mice were injected with hM4Di or mCherry virus in the NAc. There was no effect of clozapine or vehicle on aggression self-administration and aggression seeking in hM4Di or mCherry-injected mice. *p < 0.05. Data are from (Golden et al. 2017b, 2019).

To determine whether this cell-type specificity is relevant to operant aggression and aggression seeking (relapse), we recently used the same breeding approach in conjunction with aggression self-administration and relapse procedures (Golden et al., 2019). We found that aggression self-administration and relapse (nonreinforced aggression seeking on abstinence day 1) induced c-Fos immunoreactivity NAc core and shell. Using in situ hybridization, we observed similar distributions of Drd1- and Drd2-expressing neurons colocalized with c-Fos. Using cell-type-specific chemogenetic manipulations, we inhibited Drd1- and Drd2-expressing neurons using clozapine, the back-converted metabolite of CNO that after systemic injections activate neurons infected with hM4Di or hM3Dq in the CNS (Gomez et al., 2017). We found that Drd1-, but not Drd2-expressing, neurons are required for both operant aggression self-administration and aggression seeking (Fig. 5C–F). These data demonstrate a selective role of NAc Drd1-expressing neurons in aggression self-administration and relapse.

Lateral habenula (LHb).

The LHb is comprised predominantly of glutamatergic neurons projecting to the VTA/substantia nigra, the rostromedial tegmental nucleus, and the dorsal and median raphe nuclei (Hikosaka, 2010). While the outputs of the LHb are relatively few and well defined, the inputs are more diverse. These include GABAergic neurons from basal forebrain (BF), diagonal band, ventral pallidum, entopeduncular nucleus, and midbrain ventral tegmental neurons (Shabel et al., 2012; Stamatakis et al., 2013; Golden et al., 2016; Meye et al., 2016), and glutamatergic neurons from lateral hypothalamus, anterior cingulate, medial PFC, and entopeduncular nucleus (Li et al., 2011; Poller et al., 2013; Stamatakis et al., 2016). The diverse organization of LHb efferent and afferent projections allows control over dopaminergic and serotonergic tone, and therefore robust regulation of aggressive behavior (Flanigan et al., 2017).

Several studies implicate the LHb in aggression reward. We recently used the aggression CPP procedure to identify a critical GABAergic projection from the BF to LHb that bidirectionally controls aggression motivation in male CD-1 mice (Golden et al., 2016). Specifically, circuit-specific optogenetic silencing of BF GABAergic terminals in the LHB of aggressive mice increased LHb neuronal firing and decreased aggression CPP. Conversely, optogenetic activation of GABAergic terminals in nonaggressive mice promoted aggression CPP. Direct optogenetic manipulation of cell bodies within the LHb recapitulated the previous BF-LHb circuit-specific results. Notably, these manipulations did not initiate aggressive bouts but did bidirectionally modulate the severity of attack bouts that were self-initiated by the aggressive mice.

Extending these findings, Flanigan et al. (2018) reported that orexinergic neurons within the lateral hypothalamus project to a small population of GABAergic interneurons within the LHb and activate them via orexin receptor 2 to promote both unconditioned aggression and aggression CPP. Using fiber photometry (Gunaydin et al., 2014) to monitor fluorescence changes of the calcium indicator GCaMP6 in the LHb of aggressive mice, they found time-locked decreases in neuronal population activity when a submissive intruder was attacked during resident-intruder testing. Similarly, the expression of aggression CPP was associated with decreased neuronal activity in the LHb of aggressive mice. However, both during aggressive bouts in the resident-intruder test and during aggression CPP, a small population of GAD2-expressing GABAergic neurons showed robust activation. Although the LHb is predominately composed of glutamatergic neurons, the medial aspect of the LHb contains a small population of local GABAergic inhibitory interneurons (Smith et al., 1987; Zhang et al., 2016, 2018). Flanigan et al. (2018) showed that this LHb GABAergic interneuron population expresses orexin receptor-2 and inhibits LHb glutamatergic projection neurons via the action of lateral hypothalamic orexinergic projections.

The importance of the LHb in aggression motivation has also been identified in other species, from zebrafish to humans. Using an aquatic variation of the winner effect, Chou et al. (2016) found that the zebrafish ventral habenula, a heavily evolutionarily conserved homolog of the mammalian LHb (Amo et al., 2010), is critical to the resolution of aggressive conflict behavior. Using an elegant transgenic strategy to express Tetanus neurotoxin within selective habenular subregions, they found that Tetanus neurotoxin-mediated inhibition of ventral habenula terminals projecting to the median raphe resulted in continued aggressive behavior in zebrafish that had previously lost and should no longer exhibit aggression. Although not a direct assay of aggression motivation, in combination with the previously results, these findings support the role of the habenular complex in controlling aggression-related social behaviors. Similarly, in a preliminary pilot study of men diagnosed with severe intermittent-explosive disorder, resting-state fMRI results suggest that high levels of trait aggression are linked to lower global efficiency of the left habenula (Gan et al., 2018). Additionally, lower resting-state functional connectivity was observed between the left habenula and the left ventrolateral PFC, a region involved in inhibitory control. This association suggests that the habenula exerts inhibitory control during the expression of extreme reactive aggression, and that heightened habenula activity or integration may be needed to lower aggression severity.

Hypothalamic nuclei.

The hypothalamus is strongly linked with aggression, due to the depth of cross-species experiments and pan-methodological approaches repeatedly attributing the initiation of aggression to its subnuclei. Termed the hypothalamic attack area, this region classically spans the lateral through ventromedial hypothalamus, and has been the focus of aggression research for decades (Siegel et al., 1999; Hashikawa et al., 2017b; Yamaguchi and Lin, 2018). Recent work by Lin and colleagues (Lin et al., 2011; Falkner et al., 2014; Hashikawa et al., 2017a) has more selectively identified Esr1⁺ neurons within the ventrolateral portion of the ventromedial hypothalamus (VMHvl) as the critical population controlling the initiation of aggression in both male and female mice. They also reported that the VMHvl contributes to aggression self-administration (Falkner et al., 2016). Specifically, using operant aggression procedures in combination with single-unit electrophysiological recordings in awake, freely behaving mice, they found that activity in nearly one-third of recorded VMHvl neurons was significantly correlated with aggression self-administration. These same neurons were also more active during an actual fighting epoch. Using fiber photometry, the authors found that neuronal population activity increased as mice learned to successfully self-administer an intruder, whereas mice that failed to acquire self-administration showed no change in activity. During extinction sessions, VMHvl activity decreased and chemogenetic inactivation and optogenetic activation of VMHvl neurons decreased or increased operant aggression self-administration, respectively. Together, these data suggest that a subpopulation of VMHvl neurons is critical to both the initiation of unconditioned aggression and learned aggression reward.

More recently, other hypothalamic nuclei have been implicated in aggression reward. Using the aggression CPP procedure (Golden et al., 2016), Stagkourakis et al. (2018) demonstrated dissociable roles of glutamatergic projections from the hypothalamic ventral premammillary nucleus (PMv) to the VMHvl or supramammillary nucleus (SuM) in unconditioned aggression and aggression CPP. Activation of PMv terminals in VMHvl selectively initiated attack behavior, whereas activation of PMv terminals in the SuM selectively modulated aggression CPP. Like observations in LHb (Golden et al., 2016), the ability of the PMv-SUM projections to modulate reward was not limited to aggression, as cocaine CPP was also modulated by manipulation of the projections. Finally, using optogenetic methods, Stagkourakis et al. (2018) showed that the PMv-SUM projections are also critical to the development and maintenance of social dominance hierarchies in male mice, suggesting a more general role of this projection in social behavior.

Conclusions

We reviewed studies on neuropharmacological and circuit mechanisms of aggression reward, as assessed in the aggression CPP and self-administration procedures. The results of the studies reviewed implicate several neurotransmitter and hormonal systems, including GABA, dopamine, serotonin, and corticosterone. These studies have also identified several critical brain systems, including NAc, LHb, DRN, and varying hypothalamic nuclei. An emerging conclusion from these studies is that the mechanisms of reactive unconditioned aggression and learned appetitive aggression are partially dissociable. To illustrate the circuitry that has been associated with appetitive aggression in mice using aggression CPP or self-administration procedures, Figure 6 integrates findings across the NAc, BF, LHb, hypothalamic nuclei, and DRN. A question for future research is the convergence and divergence of the circuit controlling unconditioned “consummatory” aggression versus the circuit controlling learned appetitive aggression.

Figure 6.

Neural circuitry of aggression seeking and aggression severity. Top, A circuit schematic of the brain regions implicated in aggressive behavior. Bottom, The main experimental manipulations in these regions pertaining to aggressive behavior and their outcomes, summarized in a table. For the circuit diagram, the legend indicates the effect on aggressive behavior following stimulation of a region or circuit mechanism. In the table, the legend indicates experimental outcome on aggressive behavior. For this review, increased aggression seeking was operationally defined as an increased CPP score or operant responding. Increased aggression severity was operationally defined by higher number of attacks, shorter latency to attack, longer attacks, or increased intensity of an attack bout quantified by measures, such as bite number. HY, Hypothalamus; LH, lateral hypothalamus; VMH, ventromedial hypothalamus.

Clinical implications

Aggression is the direct cause of suffering and death for millions of people around the world (Sumner et al., 2015). Like addictive drugs, aggression can be highly rewarding, pursued despite adverse consequences (Chester and DeWall, 2016; Gan et al., 2016), and sought anew after lengthy enforced abstinence (Durose et al., 2014). Yet, excessive aggression is not viewed by either the medical profession or the public as “addictive” in nature (Golden et al., 2017a; Golden and Shaham, 2018). Indeed, in the recent formulation of Research Domain Criteria of National Institute of Mental Health, the term “aggression” does not appear under any of the research domains. Not surprisingly, over the last decades, little progress has been made in the treatment of pathological aggression. During this time period, the standard treatment for inappropriate or maladaptive aggression has been based on neuroleptic dopamine antagonists, such as haloperidol (Ostinelli et al., 2017), although the efficacy of such treatments is often due to the neuroleptic's sedative effects (Calver et al., 2015). The failure to identify and bring to the clinic novel therapeutic strategies for the treatment of aggression negatively affects the well-being of both the people who suffer from comorbid psychiatric disorders with aggression and their treatment providers. In this regard, we believe that using behavioral approaches for the study of aggression reward in animal models, and their better incorporation into the already rich clinical literature on this topic, is an important step in the right direction. It is incumbent for preclinical aggression researchers to step out from the overbearing shadow of reactive/defensive aggression and use behavioral procedures that cover the full spectrum of ethologically relevant aggressive behaviors, which include both reactive and instrumental aggression, as well as appetitive and pathological aggression. We propose that doing so will greatly enhance the utility and translation of ongoing preclinical aggression research.