Rage Against the Machine: Advancing the study of aggression ethology via machine learning.

Abstract

Rationale:

Aggression, comorbid with neuropsychiatric disorders, exhibits with diverse clinical presentations and places a significant burden on patients, caregivers and society. This diversity is observed because aggression is a complex behavior that can be ethologically demarcated as either appetitive (rewarding) or reactive (defensive), each with its own behavioral characteristics, functionality, and neural basis that may transition from adaptive to maladaptive depending on genetic and environmental factors. There has been a recent surge in the development of preclinical animal models for studying appetitive aggression-related behaviors and identifying the neural mechanisms guiding their progression and expression. However, adoption of these procedures is often impeded by the arduous task of manually scoring complex social interactions. Manual observations are generally susceptible to observer drift, long analysis times, and poor inter-rater-reliability, and are further incompatible with the sampling frequencies required of modern neuroscience methods.

Objectives:

In this review we discuss recent advances in the preclinical study of appetitive aggression in mice, paired with our perspective on the potential for machine learning techniques in producing automated, robust, scoring of aggressive social behavior. We discuss critical considerations for implementing valid computer classifications within behavioral pharmacological studies.

Key results:

Open-source automated classification platforms can match or exceed the performance of human observers, while removing the confounds of observer drift, bias, and inter-rater reliability. Furthermore, unsupervised approaches can identify previously uncharacterized aggression-related behavioral repertoires in model species.

Discussion and Conclusions:

Advances in open source computational approaches hold promise for overcoming current manual annotation caveats while also introducing and generalizing computational neuroethology to the greater behavioral neuroscience community. We propose that currently available open-source approaches are sufficient for overcoming the main limitations preventing wide adoption of machine learning within the context of pre-clinical aggression behavioral research.

1. Introduction

There is an increased risk for abnormal or pathological aggression in individuals suffering from neuropsychiatric disorders. A growing literature indicates that certain types of aggression, namely compulsive aggression seeking, may be a distinct externalizing pathology that potentially functions through dysregulation of reward processing in a manner akin to drug addiction (Blair 2016; Chester and DeWall 2016). This is supported by the common finding that, in humans, some neuropsychiatric disorders present comorbid with inappropriate aggression, including depression and substance abuse (Tyrer et al. 2015; Fazel et al. 2015), and recidivism rates of violent offenders closely mimic relapse rates for drug addicted individuals (Sinha 2011; Ducrose et al. 2014). We and others have hypothesized that studying the motivational component of aggression, termed appetitive aggression, may provide novel therapeutic approaches to the treatment of maladaptive aggression presenting both within and outside of neuropsychiatric comorbidities (Miczek et al. 2015, 2017; Golden et al. 2019b; Covington et al. 2019). Unfortunately, from a preclinical perspective, there are relatively few established models for mechanistically studying the neurobiological basis of maladaptive appetitive aggression (Hashikawa et al. 2018; Flanigan and Russo 2019; Golden et al. 2019b).

Aggression motivation has long been a focus of preclinical research (Thompson 1963), invariably examined through the lens of ethological analysis, but often stymied due to the tremendous effort and durations required to manually score complex social behavior with sampling frequencies and accuracies that match modern neuroscience techniques. Several recent reviews have highlighted the power of machine learning approaches for creating automated behavioral classifiers to study social behavior, including aggression (Anderson and Perona 2014; Egnor and Branson 2016; Robie et al. 2017; Gris et al. 2017; Brown and de Bivort 2018; Akay and Hess 2019; Mathis and Mathis 2019; Datta et al. 2019). The ability to recognize and categorize common behaviors in model species is an integral component of model reproducibility and extendibility, and these observations can be combined with computational neuroethology to circumvent several inherent issues with manual annotation - most notably including observer drift and bias, long analysis times, and inter-rater-reliability (Anderson and Perona 2014; Egnor and Branson 2016; Datta et al. 2019). Additionally, both supervised and unsupervised machine learning algorithms have uncovered previously unknown behavioral repertoires in model organisms and have confirmed foundational assumptions of ethology (Vogelstein et al. 2014; Wiltschko et al. 2015; Rudolf et al. 2019). Several machine learning-based open source packages have been developed that can track one or multiple animals during freely moving behavior, while others, including one recently released by our lab (Nilsson et al. 2020), can be used to classify behaviors based on pose-estimation tracking (Table 2). These approaches have led to a renaissance in the study of ethology, which is poised to catapult classical behavioral neuroscience into the realm of “big data”.

Table 2.

Currently available open source tracking or behavioral classification software.

Name	Species Multiple	Multiple animals?	Main output Behavioral classifiers validated in original	Behavioral classifiers validated in original publication	Mechanism	Software website
Ctrax (Branson et al. 2009)	Drosophila	Yes	Behavior	Walk, stop, sharp turn, crabwalk, backup, touch, chase	Computer vision	http://ctrax.sourceforge.net/
CADABRA (Dankert et al. 2009)	Drosophila	Yes	Behavior	Lunging, tussling, wing threat/extension, circling, chasing, copulation	Supervised	http://www.vision.caltech.edu/cadabra/
Autotyping (Jhuang et al. 2010)	Mice	No	Behavior	Home-cage behaviors: drink, eat, groom, hang, micromovement, rear, rest, walk	Supervised	http://cbcl.mit.edu/software-datasets/mouse/
JAABA (Kabra et al. 2013)	Drosophila, mice	Yes	Behavior	Flies: walk, stop, crabwalk, backup, touch, chase, jump, copulation, wing flick/grooming/extension, righting, center pivot, tail pivot. Mice: follow, walk.	Supervised	https://sourceforge.net/projects/jaaba/files/
MiceProfiler (de Chaumont et al. 2012)	Mice	Yes, if no obstruction	Behavior	Head-head contact, anogenital contact, side by side, approach, leave, follow, chase	Physics engine	http://icy.bioimageanalysis.org/plugin/mice-profiler-tracker/
Unnamed* (Burgos-Artizzu et al. 2012)	Mice	Yes	Behavior	Approach, walk away, circle, chase, attack, copulation, drink, eat, clean, sniff, rear	Supervised
Unnamed* (Matsumoto et al. 2013)	Rats	Yes	Behavior	Rearing, head-head and head – hip contact, approach, leave, follow, mount, intromission, ejaculation	Physics engine	See supplementary material
MotionMapper (Berman et al. 2014)	Flies	Yes	Behavior	Grooming: wing, leg, abdomen, head. Wing waggle, running	Unsupervised	https://github.com/gordonberman/MotionMapper
Unnamed* (Hong et al. 2015)	Mice	If distinct	Behavior	Attack, close investigation, mounting	Supervised
MoSeq* (Wiltschko et al. 2015)	Mice	No	Behavior	Dart, micromovement, pause, rear, walk	Unsupervised	MTA and private repository
Unnamed (Unger et al. 2017)	Mice	Yes	Behavior	Direct contact, nose to nose, anogenital sniffing, following, mating, self-grooming	Unsupervised
DuoMouse (Arakawa et al. 2017)	Mice	Yes	Behavior	Sniffing, following, indifferent	Unsupervised	http://www.mgrl-lab.jp/DuoMouse.html
B-SOiD (Hsu & Yttri, 2019)	Mice	No	Behavior	Pause, rear, groom, sniff, orient left/right, locomotion	Unsupervised	https://github.com/YttriLab/B-SOID
SimBA (Nilsson et al., 2020)	Mice, rats	Yes	Behavior	Attack, pursuit, lateral threat, anogenital sniff, mounting, upright submissive, allogrooming, flee, scramble, boxing, approach, avoidance, drink, clean, eat, rear, walk away, circle	Supervised	https://github.com/sgoldenlab/simba
Stytra (Stih et al. 2019)	Zebrafish larvae	No	Closed loop tracking & stimulus admin. (real time)		Computer vision	https://github.com/portugueslab/stytra
ToxID (Rodriguez et al. 2017; also see ToxTrac Rodriguez et al. 2018)	Fish, ants, mice	Yes	Individual tracking in groups (real time)		Computer vision	https://sourceforge.net/projects/toxtrac/
MoST (Thanos et al. 2017)	Mice	Yes	Individual tracking in groups		Computer vision
idtracker.ai (Romero-Ferrero et al. 2019, also see idTracker Pérez-Escudero et al. 2014)	Drosophila, fish, ants, mice	Yes	Individual tracking in groups		Unsupervised	http://idtracker.ai/
LocoMouse* (Machado et al. 2015)	Mice	No	Kinematics		Supervised	https://github.com/careylab/LocoMouse
DeepLabCut (Mathis et al. 2018)	Drosophila, mice, horses, humans, fish	Yes	Pose		Supervised	http://www.mousemotorlab.org/deeplabcut
LEAP (Pereira et al. 2019)	Drosophila, mice	Yes,	Pose		Supervised	https://github.com/talmo/leap
Successor: Social LEAP Estimates Animal Pose (SLEAP)		through SLEAP				https://github.com/murthylab/sleap
DeepFly3d* (Günel et al. 2019)	Drosophila, humans	No	Pose		Unsupervised	https://githubcom/NeLy-EPFL/DeepFly3D
DeepPoseKit (Graving et al. 2019)	Drosophila, locusts, zebras	If distinct	Pose (real time)		Supervised	https://github.com/jgraving/DeepPoseKit
Sensory Orientation Software (Gomez-Marin et al. 2012)	Drosophila, mice, fish	No	Posture & trajectory real time)		Computer vision	https://sourceforge.net/projects/sos-track/

^*Next to software name indicates multiple cameras or specialized equipment necessary.

However, even as these techniques have removed several hurdles impeding high throughput behavioral analysis, the use of automated behavioral classifiers has proven oddly difficult to adopt and generalize across labs. Implementation of these techniques is often slowed by a lack of computational knowledge, the need for specialized and expensive equipment, and the high computational expense to adequately train new classifiers. Due to the complexity of these programs, even the initial installation can be intimidating or difficult regardless of previous programming experience. In this perspective, we will briefly review the state of the appetitive aggression literature (also see Golden and Shaham 2018; Golden et al. 2019b), and then within this context, provide a primer on how machine learning approaches (regarding both acquisition/tracking and predictive classifier analysis) may be incorporated into future studies. Definitions of commonly used terms in computational neuroethology are included in Table 1. We propose, predominantly thanks to efforts of numerous labs in developing and advancing machine learning methods for behavioral tracking (He et al. 2017; Mathis et al. 2018; Graving et al. 2019; Pereira et al. 2019), that currently available approaches are sufficient to overcome the main limitations preventing wide adoption of machine learning for scoring complex social behavior within the context of pre-clinical aggression research.

Table 1.

Definitions of common terms in computational neuroethology.

Term	Description
Pose	The relationships and location of body-part ‘key-points’ (e.g., individual joints/digits in motor control tasks; or the nose, ears, flanks, centroid and tail base in an open-field task) that are relevant for determining an individual’s activity in a video frame.
Pose-estimation	Computer-generated predictions of the individual(s) pose in a video frame.
Behavior classification	A computer-generated categorical prediction describing the presence or absence of a behavior (e.g., attack vs. no attack) in a video frame.
Features	Independent variables derived from pose-estimation (e.g., animal velocities and the distances between animals) that are used to predict the presence and absence of a behavior in a video frame.
Feature weight / importance	The relevance of a specific feature for accurately classifying the presence and absence of a behavior in a video frame. For example, the distance between the first animal’s nose and the second animal’s tail base has importance or weight for accurately classifying anogenital sniffing.
Ensemble algorithm	The grouping of multiple learning algorithms based on different subsets of features. The individually weak learning algorithms are combined to create a strong or accurate behavior classification (e.g., random forest algorithm).
Hidden states	Unobservable factors determining the probability for ensuing expression of a behavior given the current behavior.
Model interpretability	The extent to which a machine model’s decision processes can be understood by humans (e.g., how does the algorithm discriminate between attack and non-attack events?).
Model explainability	The extent to which a machine learning system allows a qualitative comprehension of the association between the features of a specific observation and its prediction (e.g., why is the video frame classified as containing an attack event?). Note that an explainable model also is an interpretable model.
‘Black-box’ model	A metaphor applied to machine learning algorithms with decision processes that are not readily explainable or interpretable.

2. Appetitive aggression

2A. Winners like to win: revisiting aggression reward

Sixty years of behavioral research have shown that the opportunity for, or experience of, an aggressive encounter with a conspecific can be reinforcing in many species, including select mammals. Early work established the propensity of Siamese fighting fish (Beta splendens) to perform operant tasks for the opportunity to attack a static (Thompson 1963; Thompson and Sturm 1965) or animate representation of a conspecific (Craft et al. 2003, 2007; Elcoro et al. 2008). Similar behaviors were also demonstrated in homing pigeons (Cole and Parker 1971) and male fighting cocks (Thompson 1964). Although this pioneering work showed that animals will seek the opportunity for aggression (once thought to be a uniquely human trait), these studies did not explore the relationship between seeking aggression and the actual experience of an aggressive encounter.

Subsequent work allowing physical attacks against conspecifics has highlighted the importance of dominance in solidifying aggression reward seeking. Among many species, those animals that win their first aggressive encounter are more likely to win subsequent bouts, while those that lose are more likely to lose subsequent bouts. Such “winner” and “loser” effects (Ginsburg and Allee 1942; Oyegbile and Marler 2005, 2006; Kudryavtseva et al. 2011) and have been demonstrated in meadow voles (Vlautin and Ferkin 2013),crayfish (Momohara et al. 2016), flour beetles (Okada et al. 2019), Drosophila (Trannoy et al. 2015; Kim et al. 2018), lobster cockroaches (Kou et al. 2019), Syrian hamsters (Schwartzer et al. 2013), and many species and strains of mice (Oyegbile and Marler 2006). Some mammals do not display the winner effect, however, and species such as the white-footed mouse only do so following experimental manipulations of testosterone levels (Fuxjager et al. 2011). Aggressive encounters may be sought prior to exposure, but winning often matters in maintaining the drive to seek aggression and in determining future performance.

To minimize the effects of bout outcome on behavioral measures, many mouse assays for aggression use a variation on the resident-intruder task where a smaller subordinate intruder is introduced into the home-cage of a larger, older resident. Early studies using these procedures found that mice will cross electric grids (Lagerspetz 1964) and navigate T-mazes (Tellegen and Horn 1972; Legrand 1978) and runways (Legrand 1970) to attack a subordinate. Furthermore, physical aggression alone is sufficient to condition mice to prefer aggression-paired contexts (Potegal 1979; Taylor 1979). While these early influential studies demonstrate the utility of mouse models in the study of appetitive aggression, subsequent work shifted toward using resident intruder (Miczek and O‟Donnell 1978; Brain et al. 1981) or sensory contact (Kudryavtseva et al. 1991) assays where pairings are repeated and the subsequent interactions are recorded and scored for a variety of behavioral measures. However, such procedures may be biased toward reactive aggression due to their inescapable and involuntary nature (Kudryavtseva et al. 2011, 2014). To overcome these limitations, recent work has focused on behavioral procedures that examine both reactive and appetitive aggression using operant conditioning tasks, and this approach is instrumental for understanding the neurobiological differences underlying different types of aggression.

2B. Individual variability in inbred and outbred lines

A technical confound associated with preclinical aggression research is the relative lack of innately expressed aggression exhibited by commonly used inbred mouse strains relative to outbred strains (Jones and Brain 1987). Unlike outbred strains, inbred mice often require significant experimenter manipulation, either in the form of extended social isolation (Banerjee 1971) or repeated social instigation (Kudryavtseva et al. 2014; Covington et al. 2018), to exhibit significant levels of aggression towards conspecifics. To overcome this confound, preclinical aggression research, and especially studies of appetitive aggression, has focused on the use of outbred mice (Chia et al. 2005) that exhibit a spectrum of innate aggression behavior (Golden et al. 2016). While inbred mice are typically preferred for many research applications, recent meta-analysis shows that inbred strains do not have greater trait stability than outbred mice, and data from outbred mice may be more generalizable across conditions and populations (Tuttle et al. 2018).

We and others have used CFW or CD-1 outbred mice as these strains display several naturally occurring aggression phenotypes ranging from compulsive addictive-like aggression seeking to aggression avoidance (Golden et al. 2016, 2017b). Unfortunately, the use of outbred strains precludes the use of genetically defined Cre-recombinase based methods. To overcome this, we have introduced a hybrid breeding strategy using the F1 hybrid offspring derived from an inbred strain of interest and an outbred CD-1 (Golden et al. 2017a; Aleyasin et al. 2018). Following behavioral phenotyping for baseline reactive and appetitive aggression levels between inbred and hybrid populations, as well as molecular phenotyping for appropriate transgene expression, this approach introduces genetic selectivity to innately aggressive mice populations. We propose that this approach, in combination with more traditional but time-consuming backcrossing of Cre lines onto outbred strains or the impending development of outbred CRISPR transgenic lines, will provide a strong foundation for aggression research moving forward.

2C. Unconditioned vs. conditioned aggression

The conditioned place preference (CPP) procedure has historically been used to evaluate the rewarding effects of drugs and alcohol (Beach 1957; Mucha et al. 1982). In this procedure, one distinct context is paired with the conditioned stimulus while another context is paired with the unconditioned stimulus. During a subsequent stimulus-free test, the laboratory animal chooses to spend time within the conditioned or unconditioned context. An increase in preference for the conditioned stimulus paired context is indicative of rewarding effects (Bardo and Bevins 2000). More recently, CPP procedures have been developed to assess the relative reward of affiliative social interactions (Panksepp and Lahvis 2007; Dölen et al. 2013; Goodwin et al. 2018).

Similarly, based on studies in female Syrian hamsters (Meisel and Joppa 1994) and male outbred OF-1 mice (Martínez et al. 1995), we have adapted a CPP procedure in combination with the resident-intruder social defeat procedure (Miczek et al. 1982; Kudryavtseva et al. 1991; Golden et al. 2011) to study aggression reward in CD-1 mice (Golden et al. 2016). Using this method, we first categorized unconditioned reactive aggression in adult CD-1 male mice through repeated daily resident-intruder assays with adolescent submissive C57BL/6J intruder mice. Mice that attacked the intruders during these screening assays (70%) were termed aggressors, while mice that did not attack (30%) were termed non-aggressors. We then evaluated conditioned aggression motivation using the aggression CPP assay. Mice that displayed aggression during the initial screening tests developed aggression CPP, while those that did not attack the intruders demonstrated conditioned place aversion (Golden et al. 2016). In a series of follow-up experiments, we parametrically explored the aggression CPP phenomenon and observed several key findings. First, based on the observation that unconditioned reactive aggression falls along a continuum in CD-1 mice, we characterized individual differences in aggression CPP by testing a third phenotype, termed “variable aggressors,” composed of mice that performed inconsistently when repeatedly exposed to the resident-intruder procedure (Golden et al. 2017a). The variable aggressive mice exhibited significant, although weaker, aggression CPP, suggesting that repeated unconditioned aggression experiences can transform non-rewarding aggressive encounters into a rewarding experience. Second, aggression CPP is a learned phenomenon that can be acquired even by initially non-aggressive mice. Specifically, we exposed a large cohort of non-aggressors to 10 days of repeated resident-intruder testing and found that 50% transitioned to variable aggressors and exhibited aggression CPP. Lastly, aggression CPP is persistent, lasting several weeks following the final condition session (Golden et al. 2017a).

Notably, the portion of CD-1 mice that fail to show aggression is small and has not been the focus of study due to the difficulty in screening animals and filling group sizes. Advances in automated behavioral tracking, however, may be able to distinguish these phenotypes at an earlier age, alleviating these restrictions, as will be discussed in the second part of this review.

2D. Addiction-like aggression behavior and relapse

Beyond measures of reactive aggression and conditioned aggression reward, several groups have developed operant tasks that measure appetitive aggression. The Miczek group has designed an operant conditioning panel including active and inactive nose-poke ports that can be introduced into the home cage of outbred CFW mice. The resident mice are trained to nose poke on fixed ratio and fixed interval schedules of reinforcement to attack intruders (Fish et al. 2002, 2005; Bannai et al. 2007). The development of operant aggression tasks has allowed the pharmacological decoupling of aggression seeking versus aggression consumption behaviors. The Miczek group reported that the GABA_a positive allosteric modulator allopregnanolone increases operant response rates at lower doses than are required for increased attack behaviors (Fish et al. 2002), but that the effects are inhibited by the rise in corticosterone that are necessary for both operant responding and escalated aggression behaviors (Fish et al. 2005). While alcohol administration increases the motivation to fight, these effects are distinct from fighting performance and were not impacted by the antagonism of NMDA or AMPA receptors (Covington et al. 2018). 5-HT1b receptor agonism was found to decrease attack intensity without changing operant responding (Bannai et al. 2007).

The Kennedy group subsequently replicated and extended these results, finding reliable operant aggression self-administration in mice under progressive ratio, differential reinforcement of low rate behavior, and variable ratio reinforcement schedules (Couppis and Kennedy 2008; May and Kennedy 2009). Local nucleus accumbens dopamine receptor 1 or 2 antagonism in Swiss Webster mice inhibited both operant responding and select attack behaviors (Couppis and Kennedy 2008), and we have found similar results following chemogenetic inhibition of dopamine receptor (Drd) type 1, but not type 2, in hybrid F1 CD-1 x Drd1-Cre or Drd2-Cre mice (Golden et al. 2019). Extensions of these operant procedures have also shown that animals rapidly cease aggression self-administration when confronted with a non-submissive intruder (Falkner et al. 2016), further highlighting the necessity of winning in promoting aggression reward.

Pathological aggression in humans mimics cardinal features of drug addiction. Aggressive encounters are often sought despite severe negative consequences, pathological aggression develops only in a minority of individuals (Lacourse et al. 2002; Provencal et al. 2015), and recidivism rates for violent offenders who are incarcerated for repeat violent offenses are similar to the relapse rates of individuals who take addictive drugs (Anthony et al. 1994; Ducrose et al. 2014). Such cardinal features have been reverse-translated from the clinic to create animal models of addiction-like behavior. Deroche-Gamonet (2004) used a combination of operant procedures including fixed and progressive ratio tasks, as well as fixed ratio with cue-contingent shock punishment, within a rodent model of cocaine addiction that has high face validity to the DSM IV criteria (Deroche-Gamonet 2004). In a cohort of rats that initially showed equal levels of cocaine self-administration and sensitization, 17% of animals went on to show addiction-like indicators including difficulty stopping or limiting intake, high motivation for access, and continued use despite negative consequence. These measures also correlated with relapse propensity (Deroche-Gamonet 2004; Piazza and Deroche-Gamonet 2013). Based on the above considerations, we have developed a modified operant chamber to test relapse to aggression seeking following (i) home-cage forced abstinence (Pickens et al. 2011), (ii) voluntary choice-based abstinence (Caprioli et al. 2015), and (iii) punishment-induced abstinence (Krasnova et al. 2014; Marchant et al. 2019).

Using this approach, we have shown that preclinical addiction models can be used to identify the neural mechanisms controlling appetitive aggression and relapse, as well as pathological or compulsive manifestations of aggression (Golden et al. 2017b). About 70% of aggressive mice learn to lever-press for aggressive interactions, and using several gold-standard models derived from the preclinical addiction literature, we observed aggression relapse after forced abstinence, punishment-induced abstinence, or choice-based voluntary abstinence that persists long after the last aggressive act. Through cluster analysis of the aggression-related measures we also identified a subset of mice that met criteria previously developed to denote compulsive addiction in rodent models (Deroche-Gamonet 2004). Specifically, the cluster analysis identified a subset of compulsive addiction-like aggressive mice (~19%) that exhibited intense operant-reinforced attack behavior, decreased likelihood to select an alternative palatable food reward over aggression, heightened relapse vulnerability and progressive ratio responding, and resilience to punishment-induced suppression of aggression-reinforced operant responding.

Importantly, this study found that contingent punishment is effective for suppressing aggression-seeking behavior in the majority of aggressive mice, but not in those exhibiting compulsive aggression-seeking behaviors, and ultimately fails to prevent spontaneous recovery of aggression-seeking following extended abstinence in nearly all aggressive mice. These data are especially interesting in light of reports that, in rodents, footshock elicits both unconditioned aggression (O‟kelly and Steckle 1939; Azrin et al. 1967) and Pavlovian conditioned aggression to a footshock-paired tone (Vernon and Ulrich 1966). However, work in non-human primates (Ulrich et al. 1969; Azrin 1970) and rats (Baenninger and Grossman 1969; Roberts and Blase 1971) have shown that mechanical pain or shock-induced aggression is suppressed by contingent, but not non-contingent, shock punishment. Further, non-contingent footshock inhibits aggression in dominant but not subordinate mice (Frischknecht et al. 1985). Together, and within the common context of „punishment as a tool to prevent aggression‟, these data suggest greater nuance and present areas for future study within the context of adaptive and maladaptive compulsive aggressive behavior.

2E. Conclusions

The recent renaissance in aggression research has been assisted by the integration of carefully designed behavioral assays, derived from animal models traditionally used to study compulsive drug use and relapse in the addiction field, and modern neuroscience techniques such as chemogenetics (Coward et al. 1998; Armbruster et al. 2007) and optogenetics (Boyden et al. 2005). These manipulations have begun to help us understand the neural regions driving attack behavior and aggression salience (Miczek et al. 2001; Falkner et al. 2016; Han et al. 2017; Stagkourakis et al. 2018; Golden et al. 2019a), but the timescale of neural manipulations and recordings necessitate higher resolution behavioral scoring than hand-scoring is able to consistently and reliably provide. Both aggression seeking and consumption phases of behavioral assays will benefit from more rapid, reliable and non-subjective quantification of behavior. Within the field of pharmacology, real-time quantification and behavioral prediction are necessary for closed-loop manipulations that can causally differentiate the behavioral sequence constituents of reactive and appetitive aggressive behavior. Specific to current operant procedures, such approaches will allow a detailed understanding of the sequala of behavioral events that occur after an operant action is contingently reinforced with an intruder. All of these time locked events, at previously unfeasible time scales and objectivity, may provide additional information on the motivational state of the aggressor and the neural mechanisms guiding this behavior.

Currently, aggression consumption outcome measures are typically restricted to latency to first attack, or proportion of attack versus non-attack behaviors. Such simplifications leave valuable ethological data unexplored, but these analytical omissions have been necessary due to the complexity of aggression behavior and the length of time and training required to accurately manually score assays. Species typical attack behavior in mice generally includes bites directed toward the back and flanks of intruder mice, and decreased attack behavior upon displays of submission by an intruder. Additionally, shifts toward more damaging bite locations or continued attack despite submission can both be indicative of escalated aggression (Takahashi and Miczek 2014; Newman et al. 2018). Automated behavioral classifiers can simultaneously measure these and other behaviors, of both the resident and intruder animal, allowing for a richer ethological continuum. The classifiers - once validated – can be curated and disseminated through online repositories, which can eliminate observer variability and present significant opportunities for cross-site standards within behavioral analyses. Accessible classifiers, that are fast, operational across labs - and interpretable and explainable (Table 1) - also has obvious benefits for transparency and scientific rigor.

We and others propose that open source machine learning techniques will allow for rapid, high-throughput explorations of the incredible nuance of these behaviors without sacrificing accuracy. The integration of these automated behavioral classifiers will overcome the hurdle of hand scoring which has bottlenecked the field since its emergence.

3. Machine learning

3A. Embracing machine learning

Automated video assessments can exceed human performance (Gris et al. 2017), and behavioral classifiers increase throughput and consistency (Schaefer and Claridge-Chang 2012) in addition to reducing human bias and anthropomorphism in scoring (Robie et al. 2017). The development of specialized machine learning behavioral classifiers have uncovered previously unknown behavioral repertoires in animals including mice, drosophila, bats, and C. elegans (Vogelstein et al. 2014; Wiltschko et al. 2015; Rudolf et al. 2019; Zhang and Yartsev 2019). Markerless pose estimation and behavioral classification algorithms are rapidly improving, and many of the requisites are met for expanding machine learning approaches to classify complex social behavior. A key component to the adoption and success of these efforts is promoting open source packages that can be easily and widely adopted throughout the behavioral neuroscience community.

There are several requirements for the ready adoption of machine learning approaches:

1. Ease of use: The field of computational neuroethology is advancing at rapid pace, such that the ability of many behavioral neuroscientists to take advantage of these pipelines are curtailed by their skill behind a command line. Many of the recent additions to the field such as DeepLabCut (Mathis et al. 2018) and DeepPoseKit (Graving et al. 2019), however, provide graphic user interphases (GUIs) that allow the user to avoid setting hyperparameters or organizing projects via command line entries. Programs with easy installation and approachable graphical user interfaces will open doors for labs without easy access to computationally inclined individuals.

2. Generalizability: Behavioral assays within and between labs can often be filmed under different lighting conditions, with animals and backgrounds shifting in hues, using variable hardware and acquisition parameters such as frame rate, resolution, color scales, and video format. Many current implementations of machine learning for animal behavior (Table 2) require specialized hardware setups that can be prohibitively expensive or challenging to implement and scale-up for high-throughput behavioral assays, and often do not generalize well from one preparation to another (Anderson and Perona 2014). Machine learning approaches that can use standard, readily accessible recording hardware without the need for specialized builds are vital for generalizability and comprehensive implementation.

3. Cost management. Current pose estimation and behavioral classifier programs benefit from the use of depth cameras, multiple filming angles, specialized acquisition hardware, and proprietary software. Access to high-end analytical platforms, either in the form of cloud-based or local solutions, are a requirement for generating both tracking and classifier models. These costs propel computational neuroethology out of the reach of many behavioral neuroscience labs. However, the behavioral assays commonly utilized for the study of aggressive behavior are well defined and often recorded with a single camera, making it possible to generate and share „base‟ tracking and classifier models that preclude the need for individual labs to heavily invest in specialized hardware or commercial software.

4. Accuracy. At a minimum, automated classifiers need to meet the accuracy of manual annotation conducted by highly trained observers. Regardless of the adoption of machine learning techniques, the ability to consistently recognize behaviors of interest in model species is integral to behavioral research, and the experimenter aptitude to recognize significant behavioral events determines the success in machine model generation, tuning and validation. We have found, however, that even observers trained to high rates of interrater reliability on strict operational definitions of behaviors (Table 3) may occasionally miss shorter behavioral bouts when scoring long video recordings (e.g., 20k+ video frames). Machine learning approaches provide the advantage of never tiring, and in our experience, classifiers consistently detect shorter behavioral bouts missed by human observers. While ideal algorithms for specific tasks are typically dependent on data structure and the computational knowledge within a lab, we and other labs have found that supervised approaches (Nilsson et al. 2020), such as random forest machine learning algorithms, provide an appropriate balance of ease of use, interpretability, explainability, and accuracy that often exceeds human annotation accuracy.

5. Easy expansion of training sets. Ethological datasets provide an interesting challenge for machine learning. Training data provided to algorithms must be representative of the testing data, but behavioral assays are often conducted in discrete cohorts with unique properties. Training sets should contain a balanced mix of control and treatment videos, and diverse and unstable environmental influences (cage change schedules, seasonally noisy HVAC systems, etc.) must be considered as potential confounds if not represented in the original classifier training data. Specific to pharmacological research, when individual animal movement characteristics and social behaviors may be are altered by pharmacological interventions, it is not feasible to create representative training sets from the outset. We and others have approached this challenge by creating the ability to easily add additional hand annotated videos (several short videos are typically sufficient) to training sets, which can then rapidly be used to create updated, iterative, behavior classifiers and animal tracking models.

6. Interpretability. Interpretable methods are of outmost importance for reproducibility, transparency and rigor (Table 1). Methods for creating accurate classification algorithms in behavioral neuroscience should provide transparent processes allowing for clear understanding of how classifications are made and, ideally through GUI control, the capacity to titrate model parameters to operationalized standards. This may include the ability to visualize decision paths and the importances of individual independent variables or features for the classification result and how the data and features are balanced within the model. These concepts have however – traditionally – not always been favored within computer and data sciences (Rudin, 2019) and may introduce additional challenges when creating accessible methods within preclinical social behavior and aggression research.

Table 3:

Operational definitions for aggression behaviors exhibited between Resident CD-1 mice and Intruder C57B6/J mice.

Classifier	Description	Start frame	Duration of behavior	End frame
Attack	Clear physical antagonistic interaction initiated by the Resident mouse. Brief pauses can occur (equal or less than 133ms, or 4 frames at 30fps) as long as the Resident mouse remains oriented toward the Intruder.	First frame when the Resident mouse makes physical antagonistic contact with the Intruder. Typically, this is characterized by outstretched Resident forepaw(s) contacting the Intruder, while the Resident has an open mouth to initiate a bite. Can also be characterized by the first frame of a slap or quick bite without the forepaws being outstretched.	Short breaks (equal or less than 133ms, or 4 frames at 30fps) may be present in attack behavior, as long as Resident is still oriented toward Intruder. Attacks can include tussling, biting, boxing, and corralling as part of the attack bout.	First frame when Resident mouse orients away from Intruder. Typically, this is a slight turning of the head to look in a different direction, followed by a relaxation of the body and moving away from the Intruder.
Anogenital sniffing	Resident mouse is sniffing the anogenital region of the Intruder. Resident must be sniffing at base of tail, not further up on back or on legs.	First frame when the Resident mouse is clearly sniffing anogenital region of Intruder, rather than side, back, or leg.	Uninterrupted sniffing of anogenital region.	First frame when Resident mouse moves head away from anogenital region, either to move away from Intruder or to sniff non-anogenital region.
Lateral threat	Resident mouse is in close proximity (typically less than one body length away from the Intruder) to face of Intruder with back arched and side displayed toward Intruder. Ears are often pinned with shoulder and side of face nearest to Intruder mouse tilted slightly toward ceiling.	First frame when Resident mouse orients side to Intruder and tilts front of body toward Intruder.	Resident will often circle Intruder or move front half side to side in front of Intruder, feigning attacks prior to actual attack.	First frame when lateral threat posture is dropped. Animal will shift head away to look at other target or will begin an attack. If animal does not attack, the back posture will relax.
Tail rattle	Resident’s tail is curved at two or more points (typically in an “S” shape) and showing rapid back and forth movement.	Defined as rapid back and forth movement of tail. In Resident mice this manifests as curling of tail into “S” shape, where next frames show inversion of shape and rapid whipping back and forth of the tail.	The tail continues to move rapidly back and forth and can occasionally straighten if tail is moving fast enough. Tail side to side movement is continuous.	First frame when the curled side to side movement of the tail stops. The tail typically relaxes and straightens out.
Pursuit	Resident mouse is following in the Intruders path as the Intruder moves away from the Resident.	First frame when the Resident is moving toward Intruder as Intruder moves away. Typically, this is characterized by the Intruder running away after an attack ends and the Resident follows directly after, or when the Intruder walks past the Resident and the Resident markedly changes directions to pursue the Intruder in the Intruders path.	The Intruder is specifically moving away from the Resident. The Intruder is not sniffing the arena, foraging, etc. Intruder typically moves in a straight line until it reaches the edge of the area, at which point it will turn to face and watch the Resident. Resident is moving along the same path as the Intruder without sniffing or foraging in the arena.	First frame when either mouse stops moving, or the Resident deviates from path of the Intruder.

Aggression behavioral assays provide excellent insight into the current capabilities and limitations of machine learning algorithms due to the numerous technical challenges they present. Behavioral components often include a very rapid succession of bites, lunges, pursuits, anogenital sniffing, tail rattles, lateral threats, boxing, and occasionally head shakes and prosocial behaviors such as grooming, among many others. These sequences can be further reduced to levels of granularity which may or may not convey important information (e.g., face bite versus flank bite). Furthermore, on video, animals frequently occlude each other and significantly change their body shapes as they aggress, creating difficulties for pose estimators. To our knowledge, two groups have published work using automated classifiers for evaluating aggression behaviors. Hong et al., 2015 was able to classify attack behaviors in male mice of different colors using a multi-camera apparatus, and Burgos-Artizzu et al., 2012 achieved accuracy rates of ~61% using a multi-camera apparatus with two unmarked mice of the same color. Our lab has recently released Simple Behavioral Analysis (SimBA) (Nilsson et al. 2020), an automated behavioral classification technique for aggressive, submissive, and social behaviors, that was built upon pose-estimation and the six principles outlined above.

The remainder of this review will discuss the current state of machine learning for social behavior classification in rodents in terms of aggression behavior, and the progress of the field toward meeting the goals discussed above.

3B. Supervised versus unsupervised learning

Many machine learning techniques for behavioral analysis first identify the position of animals frame-by-frame in a video via pose estimation or background subtraction. In particular, accurate and rapid pose-estimation via recently developed and generalizable convolutional neural network architectures - accessible through packages such as DeepLabCut (Mathis et al. 2018), DeepPoseKit (Graving et al. 2019), and LEAP (Pereira et al. 2019) - provide a platform for generating the pose-estimation data needed to create rich feature sets required for machine classification of complex social behavior. For example, the predicted poses may be used to estimate distances between animals, their velocities, angles and accelerations across rolling windows and this may correlate with human annotated instances of aggressive behavior and used in further modelling techniques (Table 4). These positional data are then analyzed to cluster statistically similar images that the program either identifies as a predefined behavior (supervised learning), or the cluster is studied by the experimenter who then adds behavioral labels post-hoc (unsupervised learning).

Table 4:

Recommended workflow and hardware specifications for performing machine classification of complex social behaviors in experimental animals using open-source software. For more information, see the GitHub repositories of the Sam Golden lab.

Workflow step	Description	Required Hardware	Open-source tools
(i) Record and pre-process videos	Record videos at a quality where behaviors of interest are apparent to trained human observers. Crop / clip video, down-sample resolution, and re-sample frame rates, if required.	Camera allowing recordings at ≥ 30fps and resolutions ≥640×480 (i.e., standard webcam).	FFmpeg OpenCV
(ii) Track animals	Estimate the location of the animals and their body-parts in each frame of the videos with convolutional neural network.	CPU supported by graphical processing unit (GPU), through a cloud solution (e.g., Google Colaboratory, AWS SageMaker, Microsoft Azure) or locally (e.g., NVIDEA Rtx2080Ti).	DeepLabCut DeepPoseKit LEAP YOLO mask RCNN
(iii) Supervised decision ensembles	Extract feature sets, annotate behaviors, build/tune algorithms, evaluate classification performance.	Processing time benefits from higher-end CPUs and solid-state drives (SSDs). We recommend a CPU with ≥8 cores (e.g., Intel i9–9900K).	scikit-learn imblearn OpenCV Shapely
(iv) Visualize/analyze classification results	Visually evaluate machine classifications. Perform descriptive statistics of behavior sequences, their durations and patterns.	Processing time benefits from higher-end CPUs and solid-state drives (SSDs). We recommend a CPU with ≥8 cores (e.g., Intel i9–9900K).	OpenCV FFmpeg matplotlib standard python library

Note. Accurate behavior classifications require arenas and recording conditions that allow detailed animal tracking of relevant body-parts through convolutional neural network architectures (workflow Step ii). Generally, if the recording conditions allow the behaviors of interest to be reliably observed by human annotators in recorded videos, then the behaviors of interest can also be accurately annotated by machine classification techniques.

Unsupervised learning has successfully classified behaviors in tunicates, drosophila, individual mice, and pairs of mice after cropping the animals into individual videos (Vogelstein et al. 2014; Wiltschko et al. 2015; Klibaite et al. 2017; Rudolf et al. 2019; Dolensek et al. 2020). These techniques can be tremendously powerful in identifying the inherent structure present in behavior. Because the user does not predefine behaviors of interest, these algorithms have been proposed to be less biased than supervised techniques. Thus far, unsupervised techniques have resulted in important advances in mouse ethology including the identification of facial expressions corresponding to neuronally separable emotion states (Dolensek et al. 2020), and novel sub-second behavioral structures (Wiltschko et al. 2015). The program B-SOiD has also combined unsupervised t-SNE clustering and Support Vector Machines to great success in classifying naturally occurring behaviors in single mice, within a pipeline that is readily accessible to non-specialists (Hsu and Yttri 2019). The power to identify new behavioral repertoires is alluring and has the potential to significantly advance the field of ethology. Unsupervised machine learning tools are notoriously difficult to tune and interpret, however, and we propose that they should not be a first line option for generalization to labs without significant computational experience.

Training an unsupervised behavioral classifier generally involves feeding many unlabeled video frames into a user-defined algorithm which then identifies and separates images into behavioral clusters based on user-defined mathematical requirements for differences and similarities. The de Bivort lab has conducted one of the only large-scale direct comparisons of unsupervised learning techniques in classifying animal behavior (the leg movement of flies) and provides an excellent overview of unsupervised learning techniques (Todd et al. 2017). A typical pipeline for clustering often involves data pre-processing, dimensionality reduction, and cluster assignment (Todd et al. 2017; Datta et al. 2019), though see (Wiltschko et al. 2015).

Each of these steps are modular and can be performed using any of several different algorithms. Pre-processing can include transformations such as time frequency analysis, vector normalization, and wavelet transformation, while dimensionality reduction is often accomplished via techniques such as principal component analysis (PCA) or t-distributed stochastic neighbor embedding (t-SNE). The final clustering algorithms, such as k-means clustering or Gaussian mixture modeling, are used to group the data into clusters that researchers can apply behavioral labels towards (Todd et al. 2017). The flexibility of unsupervised learning approaches at each step makes them very powerful in adapting to different datasets, but tuning parameters requires careful consideration.

Importantly, unsupervised learning techniques do not have built in performance metrics. Tuning them to high perceived accuracy (e.g., all user defined attacks are also defined by the algorithm as an attack) is in essence training the model to meet particular output benchmarks, i.e., creating a supervised algorithm (Brown and de Bivort 2018). Some classes produced by an unsupervised approach may also be difficult for users to define as recognizable behaviors. Todd et al., 2017 propose evaluating networks via parameters such as minimum dwell time within a behavior class and the reliability of the algorithm when trained multiple times on the same data set (Todd et al. 2017; Brown and de Bivort 2018). These parameters acknowledge the real-world constraints on behavior and the need for replication while imposing less bias onto the model in terms of expected output. We propose that any implementation of algorithmic features during unsupervised training should be accompanied by a description of the experimental objectives driving their necessity.

Supervised learning algorithms require greater user oversight in that users are required to annotate training data that the classification system can use to learn to correctly label images. Several pose estimators are validated for use in drosophila (Branson 2014; Berman et al. 2014; Günel et al. 2019), and tools such as DeepLabCut (Mathis et al. 2018), LEAP (Pereira et al. 2019), and DeepPoseKit (Graving et al. 2019) have been validated in mice and other non-human mammals. DeepLabCut and LEAP use variations on the supervised learning technique of deep neural networks to estimate animal pose, while the DeepPoseKit uses a novel Stacked DenseNet algorithm. For in-depth explanations of these estimators, see (Mathis et al. 2018; Graving et al. 2019; Pereira et al. 2019; Nath et al. 2019; Mathis and Mathis 2019). In many of these packages, users label body-parts at set coordinates, and these training labels are used to generalize computed labeling rules to future images. Pose estimators are powerful tools in identifying finer points of animal activity, and are able to identify phenotypes not detected with simply trajectory tracking (Hong et al. 2015). Notably, new packages can include project management and image annotation GUIs, which greatly enhances the user experience and has resulted in more widespread adoption.

Supervised learning techniques can be used for both pose estimation and behavioral classification, but unlike unsupervised algorithms, can only classify predetermined behaviors and cannot identity novel behavioral repertoires. Training accurate supervised machine learning classifiers requires the careful, frame-by-frame annotation of a behavior of interest. Human variability and annotation mistakes, such as erroneous labelling of aggression events as non-aggression events, can generate significant noise that propagate to future machine analyses (Frenay and Verleysen 2014). It is therefore important to have precise behavioral operational definitions that encompass and exclude the behavioral events and non-events, respectively, and we present examples of how such precise operational definitions may look in Table 3. While such precise descriptions are not current standard laboratory practice, and functionally translate to extra time hand-annotating videos, they can promote the introduction of standardized cross-laboratory definitions through shareable classifier repositories that ultimately increase replicability as discussed previously.

Several behavioral classifiers using supervised machine learning techniques have been validated in mice, including JAABA (Kabra et al. 2013), Autotyping (Jhuang et al. 2010), and SimBA (Nilsson et al. 2020). Each uses simple, single camera setups, with significant differences in underlying machine learning approaches to creating predictive classifiers. JAABA classifies walking and following behavior in groups of same colored mice using a boosting ensemble algorithm (Kabra et al. 2013), and Autotyping uses a combination of Hidden Markov Model and Support Vector Machines to classify a variety of home cage behaviors in single-housed mice (Jhuang et al. 2010). SimBA uses random forest classifiers to classify aggressive, defensive, and other social behaviors in mice following pose-estimation (Nilsson et al. 2020). Some programs also use multiple cameras, depth cameras, or other specialized setups for behavioral classification. The Anderson Lab has developed a random forest classification technique which can identify attack, mounting, and investigation behaviors in differently colored mice using multiple depth sensing cameras (Hong et al. 2015). Boosting ensemble algorithms have also been used to classify social movement, attack, copulation, exploratory, and drinking and eating behaviors with integrated top and side view filming (Burgos-Artizzu et al. 2012).

Whether implementing supervised or unsupervised machine learning techniques, there is a strong argument for using algorithms that are easily interpretable (Rudin 2019). Black box models have been helpful in image analysis and uncovering new behavioral patterns, but the inability to readily understand the underlying assumptions of algorithms is a detriment. Black box models - not unlike prevalent behavioral protocols - can appear to be valid and accurate (strong face validity) whilst measuring factors unrelated to the phenomenon it claims to measure (weak construct validity). Examples of such fallacies, with more trivial consequences, would include convolutional neural networks that „successfully‟ discriminate Husky dogs and wolfs based predominantly on only the presence or absence of snow in the image (Ribeiro et al. 2016), or networks generating „successful‟ gender classifications from iris texture images through the presence or absence of mascara (Kuehlkamp et al. 2017). More seriously, black box models are notorious for aggregated racial and social-economic biases (Mittelstadt et al. 2016; Rudin 2019; Obermeyer et al. 2019); while clearly a different class of concern than what is anticipated in preclinical behavioral computational neuroethology, the cautionary tale remains relevant and important. Explainable and interpretable models should thus be adopted as best scientific practice (Rudin 2019) to allow for user oversight of the parameters guiding decision processes.

While supervised learning techniques require users to predefine behaviors of interest, they are often much easier to tune and interpret than unsupervised learning techniques, particularly for labs without statisticians or computer scientists on staff. In the spirit of ease of use, generalizability, and accuracy, we propose that supervised learning techniques are a good starting point for the automated examination of aggressive behaviors.

3C. Common classifying algorithms for supervised learning

Within supervised learning, there are many different algorithms from which to choose, and no method is universally superior to the others (Hand 2006). The appropriate selection depends on the structure, noise, and biases within each dataset, as well as the threshold for acceptable training duration and willingness to troubleshoot parameters (Hand 2006; Anderson and Perona 2014; Egnor and Branson 2016; Gris et al. 2017; Akay and Hess 2019). Common techniques for behavioral and image classification include variations of Neural Networks, Support Vector Machines, Gradient Boosting Machines, Random Forests, and Hidden Markov Models. All of these algorithms have potential benefits and may outperform others on specific datasets, but we and others have found that random forest classifiers provide high accuracy classification (Breiman 2001; Liaw and Wiener 2002; Nilsson et al. 2020) with the added benefit of interpretability, being easy to tune, and robust against overfitting. Here we describe several of these algorithms and highlight the potential pros and cons they possess regarding the classification of aggression-related behaviors.

Neural Networks (Hopfield 1982; LeCun et al. 1989)

Neural networks or convoluted neural networks are typically used for large datasets with a high number (tens of thousands) of features and observations, as well as noise. Features may include the trajectory of individual body parts, Euclidean distances, body part movement over a small frame of time, etc. Users provide an input layer of data to the neural network and define the desired categories in the output layer. The network then computes additional hidden layers between the input and output layers to correctly classify input data into the appropriate output categories or classifiers. Hidden layers are advantageous as specific features are not user-selected for inclusion within the algorithm, but the cost is an absence of information on what the resulting classifications are ultimately based on. Neural networks can be challenging to interpret and tune, and similar performance can be achieved with other methods (Lietman et al. 1999; Nitze et al. 2012; Liu et al. 2013). Many programs, including the pose estimators described above, successfully use deep neural networks for pose and/or behavioral classifications (Karpathy et al. 2014; Krizhevsky et al. 2017; Mathis et al. 2018; Pereira et al. 2019). Due to the lack of information regarding feature weights, we propose that more transparent algorithms which perform at similar levels are preferable for behavioral classification derived from tracked features.

Random Forests (Breiman 2001; Liaw and Wiener 2002)

Random forests are a type of ensemble algorithm. Ensemble algorithms are composed of many independently trained weak models which are combined to make strong predictions. Random forests rely on the bootstrapping of both (i) subsets of data, and (ii) predefined features to make a powerful forest of decision trees than can then vote on the classification of a behavior. For example, a decision tree in a random forest starts with a subset of data (e.g., 500 frames of 50,000) then uses a subset of the features (e.g., 10 out of 100 features) to split the data into yes/no classifications. Random forests are useful because each branch is created using the feature which provides the most information, and by combining these data from the forest, it is possible to determine which features were most important for creating the behavioral classification. This technique is computationally rapid as the trees are independent and can be built in parallel, and they are unlikely to overfit data based on noise (Breiman 2001).

One weakness of random forests is their inability to natively support biased datasets. These datasets are common in behavioral videos, in which most frames do not contain the behavior of interest. For example, in a five-minute video filmed at 80FPS with 10 total seconds of tail rattle, this would result in 23,200 frames with no tail rattle, and 800 frames with tail rattle. Constructing a random forest with these data as-is would mostly construct trees with no or very few instances of tail rattle frames. In order to train the random forests more robustly, over-sampling and/or under-sampling techniques (Chawla et al. 2002; Batuwita and Palade 2013) can be used to balance the data.

Gradient Boosting Machines (Freund and Schapire 1997; Friedman et al. 2000; Friedman 2001)

Gradient boosting machines build decision trees iteratively, attempting to fix erroneous classifications after each node split by finding a different feature which better classifies incorrectly classified data. Gradient boosting can be more robust than random forests if tuned correctly but are more prone to overfitting data and incorrectly identifying noise, such as pose misestimation, as legitimate behavioral data. While pose estimation is often highly accurate, there are outliers that may cause overfitting when fed into a gradient boosting machine. With more accurate pose estimation and highly consistent filming conditions across labs, gradient boosting machines may provide superior classification than random forests, but it is difficult to currently meet these conditions and maintain generality of video recording conditions. While the use of gradient boosting machines is a good goal, their widespread adoption may depend on how well „base‟ models, and their underlying recording acquisition parameters, are standardized across laboratory recording environments.

Support Vector Machines (Cortes and Vapnik 1995)

Support vector machines are frequently used, and work best with, small datasets containing few outliers (Chih-Wei Hsu and Chih-Jen Lin 2002; Xu 2006). Using multiple tuning parameters (kernel, regularization, gamma, and margin), support vector machines identify clusters of similar and dissimilar data in multidimensional space and find the regression line that best separates clusters based on the training data. Understanding linear algebra and principle component analysis, in addition to having appreciable patience with tuning parameters, is helpful when using support vector machines. Like random forests, support vector machines do not work well natively with biased data sets, which can similarly be overcome through re-balancing and over- and under-sampling techniques.

There are few comparative studies of classification algorithms due to lack of generalization to other datasets, but in those that exist, support vector machines do not typically perform as well as random forests or gradient boosting machines (Caruana and Niculescu-Mizil 2006; Caruana et al. 2008). Due to the larger number of tuning parameters involved in support vector machines, and the similar or superior classification performance by random forest classifiers, we suggest that random forests may be a better starting point for creating generalizable and easily adopted classifiers.

Hidden Markov Models (HMM) (Rabiner and Juang 1986)

HMMs can be integrated with supervised or unsupervised techniques and excel at finding patterns in small sequences of data. These models have recently been extended for use in behavioral classifications due to the similarities from frame to frame of video, and the inherent structure of behavior (Carola et al. 2011; Wiltschko et al. 2015; Arakawa et al. 2017). HMMs use hidden weights to predict the probability of a transition from one state to another. For example, if a resident is biting the flank of an intruder in one frame, it is unlikely to be grooming its face in the next frame. As indicated by the name of the algorithm, behavioral classifications depend on a hidden state. Anderson & Perona, 2014 raise an interesting point that HMMs may be valuable in objectively measuring behaviors resulting from the “emotion state” of an animal. HMMs have been used in rodent aggression studies to investigate the effects of context (intruder behavior) on the aggression behavior of a home cage resident to parse species typical versus escalated aggression phenotypes (Haccou et al. 1988; Natarajan et al. 2009), and may provide an interesting path forward for understanding contextual aggression. While the hidden states can potentially be found via maximum a posteriori state estimations (Allahverdyan and Galstyan 2009), we propose that more inherently interpretable algorithms are preferable for initial behavioral classification.

3D. Promising directions

We strongly reiterate that there is no single correct pipeline for machine learning classification of animal behavior, but rather that the most successful approaches will likely use multiple pipelines in parallel while taking advantages of their pros and attempting to diminish their cons. The powerful techniques discussed above are summarized in Table 5, and the choice between them often revolves around the applicable knowledge within a lab and which algorithm best classifies data during pilot testing (Table 5). To facilitate generalizability to new labs, however, random forests seem like an excellent option due to their relatively few input parameters, robustness to noise, relative interpretability and explainability, and high performance on a wide array of data. A future goal of automated behavioral analysis is long-term real-time tracking of individuals within groups in a naturalistic setting, and many groups are currently working with these and other techniques to solve parts of this problem.

Table 5:

Overview of advantages and limitations of select opensource options for tracking and behavioral classification.

Technique	Open-source alternative(s)	Advantages	Limitations
Segmentation / identification networks for ‘fingerprinting’	idtracker.ai	• Can track large groups of unmarked individuals • Accessible GUI	• Requires GPU support • No live tracking • Only provides trajectory of animal centroid(s) • Not validated in mouse/rat video with high amounts of aggression • Users must generate their social behavior classifiers from tracking data, which may also require additional layer of pose-estimation processing.
RFID / depth camera system	Live Mouse Tracker	• Built-in battery of social behavior classifiers • Group analysis of unmarked animals (≤ 4 mice) • No model training required • Live tracking (30Hz) Accessible SQL data format True longitudinal analysis (days)	• Manufacture cost (~$2k) • Manufacturing time • Dedicated platform (cannot be used to analyze historical datasets). • Arena / RFID implementation require Animal Study Protocol submissions / revision • Validated for mice only
Pose-estimation	DeepLabCut LEAP DeepPoseKit	• Accurate tracking of user-defined body-parts • Very large user-base • Active code development • Accessible GUI’s (LEAP / DLC) • Accepts most recording environments/ and video formats.	• CNN training time • Requires GPU access • Group and live tracking features not yet available • Users generate their social behavior classifiers from tracking data
Behavioral classification	SimBA JAABA	• Accessible GUI’s • User-defined social behavior classifiers • Can accept any tracking data as input	• Requires third-party tracking (and GPU support) • No live classifications • Social classification restricted by the inability of third-party tracking solutions to track multiple similarly coat-colored animals

Note. The techniques listed above are not mutually exclusive, and the most successful behavioral analyses likely depend on synergistic implementations of disparate computational methods.

Distinguishing similarly looking animals has historically been a challenge for automated pose estimators, and ultimately problematic for the computational study of social behavior as many experiments require the use of nearly identical looking animals. Although not a problem when using outbred CD-1 mice as residents and C57 mice as intruders, due to their difference in coat color, this does preclude the generality of machine learning approaches to many rodent social behavioral assays. There are several recent programs that work to track the identity of individual and similarly appearing animals in groups over time, including ToxID, MoST, and idTracker.ai (Thanos et al. 2017; Rodriguez et al. 2017; Romero-Ferrero et al. 2019). Additionally, social tracking extensions of the program LEAP (sLEAP) and DeepLabCut have recently been released, and provide pose estimation for multiple interacting animals (Mathis et al. 2018; Pereira et al. 2019) by taking advantage of algorithms such as optical flow (Brox et al. 2006), part affinity fields (Cao et al. 2017), and deep-learning. The program idTracker uses separate segmentation and identification networks to individually identify animals and their trajectories and back propagation techniques that identify animals during partial occlusions (Pérez-Escudero et al. 2014).

Beyond image detection, there are several groups working to integrate RFID tracking and video tracking (Weissbrod et al. 2013; de Chaumont et al. 2019; Peleh et al. 2019). Live Mouse Tracker (de Chaumont et al. 2019) uses a combination of RFID tracking and depth imaging and has been validated with groups of up to 4 mice in a semi-natural environment over time. These programs are highly influential in allowing for the long-term study of group dynamics in large, enriched home cage environments and currently provide the state-of-art tracking solutions for mice of similar appearance. A commercial version of this program, RFID-Assisted SocialScan, is also available and integrates a large behavioral arena with attached nest boxes (Peleh et al. 2019). Together, these programs can track the identity of individuals while examining social hierarchies over long periods of time without excessive manual oversight or behavioral scoring. However, these approaches require significant investments in highly specialized hardware.

Several groups pioneering pose estimation are also continually working to increase the speed of their algorithms to achieve real-time results (Graving et al. 2019; Mathis and Mathis 2019). Currently, ToxID and Sensory Orientation Software are able to provide real time tracking of individual mice and their trajectories (Gomez-Marin et al. 2012; Rodriguez et al. 2017), and DeepLabCut has recently been used for real-time pose estimation (Forys et al. 2018). Extending these platforms for in depth real-time behavioral classification in mice will allow for the evaluation of behavior on the timescale of neural activity and allow fully “closed-loop” recording and manipulation studies. The Stytra package is currently validated in zebrafish larvae, and is able to both analyze pose and activity and rapidly provide stimuli to the experimental animal based on its activity (Stih et al. 2019).

These open source packages are continuing to improve and emerge, and integration between them is becoming easier. Pose estimators that can rapidly track individual animals despite multiple occlusions is an essential first step for the automation of aggression assays, and our lab has focused heavily on exploring random forest classifiers for the classification of aggression behaviors. Integrating the pose estimators above with high-accuracy and easy to use classifiers in an open source GUI allows for the generalization of these techniques across labs, while maintaining the ability to take advantage of newer tracking technologies as they emerge.

For example, our lab has recently developed an open-source graphical workflow (for more information, see the SimBA GitHub repository) for creating supervised decision ensembles from body-part tracking data generated through convolutional neural networks in accordance with the considerations and principles discussed herein (Nilsson et al. 2020). The package incorporates accessible menus for pre-processing and annotating videos, generating / deploying classification algorithms, with advanced machine learning validation, tuning and visualization tools. The models created through this software – together with extensive documentation and tutorials - are available for the scientific community through online repositories and we hope that this can spur further data sharing efforts that boosts the accuracy and scope of machine classification techniques in pre-clinical aggression research.

4. Conclusion

Aggression is an immutable force that contributes to the suffering and death for millions of people around the world (Sumner et al. 2015). While aggression can be highly rewarding and pursued despite adverse consequences (Chester and DeWall 2016; Gan et al. 2019), as well as sought after lengthy abstinence (Ducrose et al., 2014), aggression is not generally viewed in the medical profession or by the public as a component of neuropsychiatric disease states (Golden et al., 2017a; Golden and Shaham, 2018). Indeed, in the recent formulation of Research Domain Criteria (RDoC) of NIMH the term aggression does not appear under any of the research domains. Not surprisingly, no progress has been made in the treatment of pathological aggression, and little progress has been made using aggression as a diagnostic tool for neuropsychiatric comorbidity (Martin et al. 2013). We believe that drawing greater attention to the neurobiological mechanisms of maladaptive appetitive aggression within the context of neuropsychiatric disease will both act to (i) de-stigmatize aggression comorbidity in a manner similar to current approaches to substance abuse treatments and (ii) support the identification of novel mechanistic therapeutic approaches, which have presently been relegated to neuroleptic dopamine antagonists like haloperidol (Ostinelli et al. 2017) that exert their clinical efficacy through the neuroleptic‟s sedative effects (Calver et al. 2015). Therefore, we propose that neurobiological and behavioral tools used to study drug seeking and relapse should be used to study brain mechanisms of appetitive aggression, both preclinically and clinically. Further, expanding beyond reactive aggression to incorporate the full spectrum of ethologically relevant aggressive behaviors is an essential step in increasing the utility and translation of ongoing preclinical research. Machine learning tools enabling high throughput automated behavioral analysis are key to pursuing these lines of inquiry at a pace and depth necessary for modern neuroscience methods.

Significant progress has been made in developing open source software packages that are capable of tracking social behavior over long periods of time and across diverse acquisition parameters (Table 5), and we propose that a focus on increasing ease of use, generalizability, cost-consciousness, accuracy, and easy expansion of training sets will allow for their wider adoption. Such automated scoring promises to remove long-standing bottlenecks within aggression research and allow for the high-throughput experiments required for mapping the nuance of aggression phenotypes while simultaneously building a common language of aggression phenotypes through shared machine models across labs.