Emotion-based learning systems and the development of morality

Abstract

In this paper it is proposed that important components of moral development and moral judgment rely on two forms of emotional learning: stimulus-reinforcement and response-outcome learning. Data in support of this position will be primarily drawn from work with individuals with the developmental condition of psychopathy as well as fMRI studies with healthy individuals. Individuals with psychopathy show impairment on moral judgment tasks and a pronounced increased risk for instrumental antisocial behavior. It will be argued that these impairments are developmental consequences of impaired stimulus-aversive conditioning on the basis of distress cue reinforcers and response-outcome learning in individuals with this disorder.

Introduction

The field of moral development and moral reasoning has been transformed over the past twenty years. In the early 90s, there was one dominant view: moral reasoning involved decisions based on accessing conceptual domains (e.g., Colby & Kohlberg, 1987). Moral development involved the construction of these conceptual domains through some form of rational thought processes. With very few exceptions (e.g., Kagan & Lamb, 1987), there was little consideration given to any role of emotion in moral development/reasoning. Then, in the mid 1990s, the first studies to use results from psychopathy to infer the core role of emotion in moral development were conducted (e.g., R. J. R. Blair, 1995; R.J.R. Blair, Jones, Clark, & Smith, 1995). These were followed by seminal fMRI studies indicating that moral reasoning recruits brain regions implicated in emotion processing (e.g., Greene, Sommerville, Nystrom, Darley, & Cohen, 2001; Moll, De Oliveira-Souza, Bramati, & Grafman, 2002).

The aim of this paper is to consider the role of emotional learning in the development of morality. In particular, this paper will consider two forms of emotional learning: stimulus-reinforcement and response-outcome learning. It will be argued that these forms of learning are critical for the valence-based valuations on which much moral judgment (e.g., murder is bad and charitable giving is good) and decisions to commit (im)moral acts are based. But this is not to say that these two forms of emotional learning process give rise to the development of the system(s) that mediate all forms of moral reasoning. As previously noted (Nichols, 2002), emotion-based mechanisms can generate judgments of “badness” and undesirability. However, they cannot generate judgments of immorality. For example, an individual killing 10 people and a hurricane killing 10 people are both “bad” events but only the first is usually considered to be immoral. Nichols has proposed that judgments of immorality require the participant accessing semantic knowledge. The individual has to “recognize” the act as immoral based on their semantic concept of acts that are immoral. This may be the case. However, the learning/formation of conceptual structures will not be covered here.

Moreover, relatively recent theoretical work has stressed the importance of model-based reasoning/learning (Crockett, 2013; Cushman, 2013). As noted by Crockett (2013):

“The model-based system generates a forward-looking decision tree representing the contingencies between actions and outcomes, and the values of those outcomes. It evaluates actions by searching through the tree and determining which action sequences are likely to produce the best outcomes” (p. 363).

Importantly, the addition of a reference to a model-based system allows an explanation of various data regarding participants’ reasoning on complex moral reasoning tasks such as the Trolley problem (in particular, side-effect vs. means variants of the Trolley problem; Crockett, 2013; Cushman, 2013). These data cannot be explained through reference to stimulus-reinforcement and response-outcome learning that will be considered here. Notably though, the valence information that the model-based system likely relies on is determined by prior stimulus-reinforcement and response-outcome learning. Indeed, Crockett (2013) refers to a “simple model-free system” that roughly corresponds to the response-outcome learning discussed here and a “Pavlovian system” that corresponds to the stimulus-reinforcement learning discussed here.

The paper will also stress data from clinical populations – particularly individuals with psychopathic traits. As such, the paper will first consider the disorder of psychopathy. Data from work with this population and fMRI work from healthy participants and also individuals with psychopathy will then be considered with respect to two forms of emotional learning, stimulus-reinforcement learning and response-outcome learning, with respect to moral judgments and behavior. Note though, given this clinical population, the literature considered here will almost exclusively involve consideration of judgments of transgressions and antisocial behavior. Discussion of prosocial behavior can be found elsewhere (Gesiarz & Crockett, 2015).

Psychopathy

The classification psychopathy characterizes an individual who shows an increased risk for antisocial behavior that is coupled with pronounced emotional deficits (P. J. Frick, 1995; Hare, 2003). It is this emotional component reflecting reduced guilt, remorse and empathy that is critical (R. J. R. Blair, 2007a). For children with the disorder, this emotional component is typically referred to as callous-unemotional (CU) traits (Frick, Stickle, Dandreaux, Farrell, & Kimonis, 2005). These CU traits are at the core of developmental trajectory associated with psychopathy (Paul J. Frick & White, 2008). Youth with CU traits are at notably increased risk for meeting criteria for psychopathy as adults (Lynam, Caspi, Moffitt, Loeber, & Stouthamer-Loeber, 2007; L. C. Munoz & Frick, 2007).

It is important to note that the classification of psychopathy is not the same as the diagnosis of antisocial personality disorder (ASPD). ASPD and the corresponding diagnosis of conduct disorder (CD) for children are DSM diagnoses. They characterize individuals showing elevated levels of antisocial behavior and aggression. The core emotional deficits of psychopathy are not necessary to receive a diagnosis of ASPD or CD. Instead, it is only necessary to present with elevated levels of antisocial behavior.

Individuals with psychopathy are at significantly increased risk for the commission of acts of violence. In particular, criminals with psychopathy are at risk for committing instrumental (goal-directed) aggression relative to non-psychopathic criminals (though they also commit higher levels of reactive aggression; Williamson, Hare, & Wong, 1987). The groups show notably less difference for levels of (threat- or frustration-based) reactive aggression. This is especially true for the more serious offenses such as serious sexual assault or homicide. Notably, individuals with psychopathy are about twice as likely to have committed primarily instrumental homicides as non-psychopathic offenders (Woodworth & Porter, 2002). In fact, 93.3% of homicides committed by psychopathic offenders were instrumental in nature, compared with 48% of those by non-psychopathic offenders (Woodworth & Porter, 2002). In addition, individuals with psychopathy are significantly more likely to re-offend following release from prison than criminals without psychopathy (Olver & Wong, 2015).

In addition to their elevated risk of committing acts that can be considered immoral behaviors, individuals with psychopathy are compromised in their moral judgments. One of the earliest indices of moral development is the emergence of what has been termed the “moral/conventional distinction” (Smetana & Braeges, 1990; Turiel, Killen, & Helwig, 1987). This is the distinction between care-based moral transgressions (e.g., a child hitting another) and social disorder-based conventional transgressions (e.g., a child talking to another child in class). Typically developing children from the age of 3–4 years judge moral transgressions as less permissible, more serious and, critically, less rule contingent (i.e., moral transgressions remain non-permissible even in the absence of rules prohibiting them) than conventional transgressions (Smetana & Braeges, 1990; Turiel et al., 1987). Adults and youth with high psychopathic traits make significantly less of a moral/conventional distinction particularly with respect to the rule contingency judgments relative to even antisocial controls with low psychopathic traits (R. J. R. Blair, 1995, 1997). This is also seen for antisocial youth relative to comparison youth (Arsenio & Fleiss, 1996; Nucci & Herman, 1982; Smetana, 1990). Individuals with psychopathy/antisocial youth are also significantly less likely than comparison individuals to make reference to other individuals’ harm when justifying why care-based transgressions are wrong to commit (Arsenio & Fleiss, 1996; R. J. R. Blair, 1995, 1997). This does not mean that individuals with psychopathy make no distinction between moral and conventional transgressions. They judge moral transgressions as more serious than conventional transgressions (just to a lesser extent than comparison individuals) (R. J. R. Blair, 1995, 1997). Moreover, level of psychopathic traits has no predictive power for the ability to respond correctly when asked if an antisocial act causes harm (Aharoni, Sinnott-Armstrong, & Kiehl, 2014; Aharoni, Sinnott-Armstrong, & Kiehl, 2012). Similarly, level of psychopathic traits has no predictive power for the ability to select four of eight (four moral and four conventional) transgressions that are “morally wrong” (Aharoni et al., 2014; Aharoni et al., 2012).

Using other paradigms, studies have reported abnormally utilitarian moral judgments in individuals with high levels of psychopathy personality traits (Gao & Tang, 2013; Glenn, Koleva, Iyer, Graham, & Ditto, 2010), including incarcerated individuals with high psychopathy levels (Koenigs, Kruepke, Zeier, & Newman, 2011) relative to comparison individuals. In addition, higher psychopathy scores are associated with reduced severity ratings of transgressions – at least in youth in forensic institutions (Harenski, Harenski, & Kiehl, 2014) though not in adult forensic samples (Harenski, Edwards, Harenski, & Kiehl, 2014; Harenski, Harenski, Shane, & Kiehl, 2010). Work in both subclinical and clinical populations has shown that individuals with psychopathy show reduced endorsement of care-based norms (Aharoni, Antonenko, & Kiehl, 2011; Glenn, Iyer, Graham, Koleva, & Haidt, 2009). They also show an increased likelihood to allow actions that indirectly harm another (Koenigs et al., 2011) and regard accidents that harm others as more permissible than comparison individuals (Young, Koenigs, Kruepke, & Newman, 2012). Victim salience is an important determinant of permissibility for healthy individuals that has significantly less impact on the permissibility judgments of individuals with higher psychopathic traits. Specifically, Marsh and colleagues in a series of studies have reported that individuals with higher psychopathic traits judge actions that cause fear in others as significantly more acceptable than individuals with lower psychopathic traits (Cardinale & Marsh, 2015; Marsh & Cardinale, 2013).

The empirical literature on moral judgment in individuals with psychopathy began from that claim that if emotional responses, specifically aversive emotional responses to the distress of others, were necessary for the development of morality then a population with reduced responsiveness to the distress of others, individuals with psychopathy, should show disrupted moral development (R. J. R. Blair, 1995). The above literature largely supports this position. The basic idea is that emotional learning processes engender an expected (aversive) value of the transgression (R. J. R. Blair, 1995, 2007b). It is this aversive expected value that contributes to the individual’s rating of the badness of the transgression (R. J. R. Blair, 1995, 2007b). The idea is that emotional learning processes are disrupted in psychopathy and this results in an individual who lacks the emotional response underpinning the sense of badness of the transgression. The literature behind these claims will be explored below; specifically, the roles of two specific forms of emotional learning, stimulus-reinforcement and response-outcome in determining moral judgments and behavior.

Stimulus-reinforcement learning

Stimulus-reinforcement learning allows the individual to associate a valence with a stimulus; i.e., to learn whether something is good (positive) or bad (negative). The neural circuitry underlying fear and appetitive conditioning, two core forms of stimulus-reinforcement learning, is conserved across species (Johansen, Cain, Ostroff, & LeDoux, 2011). A core region is the amygdala. Within this region, information about the conditioned stimulus (CS) and unconditioned stimulus (US) converge. Additional regions that are importantly involved include the hippocampus, ventromedial prefrontal cortex (vmPFC) and the dorsal anterior cingulate cortex (dACC).

It has been argued that appropriate stimulus-reinforcement learning is a core requirement for appropriate moral judgments (R. J. R. Blair, 2007b). The individual must form an association between a representation of the transgression and an aversive unconditioned stimulus. A failure in stimulus-reinforcement learning is thought to underpin the impairments in moral judgment shown by individuals with psychopathy described above (R. J. R. Blair, 2007b). In line with this position, a body of studies has indicated that individuals with elevated psychopathic traits show impaired fear conditioning. Studies have shown that compared to healthy comparison adults, adults with psychopathy fail to acquire conditioned responses to stimuli associated with shock or unpleasant odors (Flor, Birbaumer, Hermann, Ziegler, & Patrick, 2002; Lopez, Poy, Patrick, & Molto, 2013; Rothemund et al., 2012). Consistent with the suggestion of impaired amygdala functioning in individuals with psychopathy, they also show reduced amygdala responses to CS+ relative to comparison individuals (Birbaumer et al., 2005). Perhaps, most interestingly, several studies have reported that a relative failure in the capacity to show aversive conditioning is a risk for antisocial behavior in adulthood. For example, Raine and colleagues reported that a group of 127 individuals with a criminal record at age 23 showed poorer electrodermal fear conditioning data at age 3 than a gender, ethnicity, and social adversity noncriminal comparison group (N=274) (Gao, Raine, Venables, Dawson, & Mednick, 2010)

An important thing to note about stimulus-reinforcement learning is that the stimulus with which the reinforcement is associated is the stimulus that best predicts the emergence of the reinforcement (Rescorla & Wagner, 1972). It is this fact that probably explains the relative lack of efficacy for socialization of punishment child-rearing strategies (Gershoff et al., 2010). Indeed, punishment-based child-rearing strategies are associated with negative outcomes. In their cross-cultural study, Gershoff and colleagues showed that mothers’ use of corporal punishment, expressing disappointment and yelling were significantly positively related to (likely threat-based, reactive) aggressive behavior. Giving a time out, using corporal punishment, expressing disappointment, and shaming were significantly positively related to greater child anxiety symptoms (Gershoff et al., 2010). Importantly, the best predictor of the emergence of the reinforcement (punishment) is not the transgression (which may have been conducted without anyone noticing and therefore gone unpunished) but the person delivering the punishment (and possibly an association with a representation of the punisher witnessing the transgression). The punisher may become a stimulus associated with fear rather than the transgression becoming associated with guilt (cf. relationship of child anxiety with use of punishment socialization strategies in Gershoff et al., 2010).

The victim’s distress is a form of aversive reinforcement present, at least in early childhood, whenever an individual commits a transgression that harms another individual. A series of studies have documented that the social consequences of care-based (moral) transgressions are different from those of social order-based conventional transgressions (Nucci & Nucci, 1982a, 1982b). Care-based moral transgressions are associated with the victim’s distress and are more likely to engender care-givers focusing attention on the victim’s distress to the perpetrator (Nucci & Nucci, 1982a, 1982b). These social consequences are very different from those incurred by conventional transgressions. Conventional transgressions engender references to social order and rules and are more likely to be associated with the care-giver’s anger (Nucci & Nucci, 1982a, 1982b).

The importance of care-based moral transgressions being associated with the victim’s displays of distress is: First that these displays are aversive and induce vicarious conditioning (Aniskiewicz, 1979; House & Milligan, 1976) and avoidance responses of objects associated with them (Meffert, Brislin, White, & Blair, 2015); and Second that the best predictor for the victim’s distress is the action that caused it. The individual should rapidly learn the “badness” of care-based transgressions because of the pairing of the victim’s distress with the commission (or observation of someone else committing) the care-based transgression (R. J. R. Blair, 1995, 2007b).

Valence-based learning on the basis of the emotional expressions of others is well established in the context of observational fear learning (Mineka & Zinbarg, 2006) or social referencing (Klinnert, Emde, Butterfield, & Campos, 1987) paradigms. In these paradigms, an infant observes the reaction of a caregiver to a novel object. If the caregiver reacts positively, the infant is significantly more likely to approach the object than if the caregiver shows a negative reaction/distress. Animal work has shown that social referencing is disrupted by amygdala damage (Jeon et al., 2010). Recent fMRI work has also implicated the amygdala in social referencing in humans (Meffert, Brislin, White, & Blair, 2014). In the context of social referencing, the amygdala is sensitive to expression prediction errors (i.e., the degree to which the expression induced by an object deviates from the expected emotional reaction). Learning occurs as a function of the prediction error such that greater learning (a greater change in the value associated with the object) occurs in response to greater prediction errors (Rescorla & Wagner, 1972). In addition, Meffert et al. (2014) demonstrated that the amygdala showed a differential response to objects associated with other individuals’ happiness relative to objects associated with other individuals’ fear.

Of course, this learning will only occur if the individual finds the distress of other individuals’ aversive. An assumption that individuals with psychopathy were less responsive to the distress of others drove the original tests of moral judgment in individuals with psychopathy (R. J. R. Blair, 1995; R.J.R. Blair et al., 1995). This assumption was based on early psychophysiological data indicating reduced autonomic responses to the distress of others (specifically, the sight of other individuals’ in pain) in individuals with psychopathy (Aniskiewicz, 1979; House & Milligan, 1976).

Following this, a series of studies have shown that youth with conduct problems, particularly those with psychopathic or CU traits, show deficits in expression recognition (e.g., R. J. R. Blair, Colledge, Murray, & Mitchell, 2001; Stevens, Charman, & Blair, 2001). Interestingly, this impairment in expression recognition is relatively selective. Recognition of fearful, sad and happy expressions is reduced while the recognition of disgusted and angry expressions remains intact (Dawel, O’Kearney, McKone, & Palermo, 2012; Marsh & Blair, 2007). Indeed, the impairment in the recognition of fearful expressions is seen even if the expression is presented too rapidly for eye gaze to have an influence on recognition accuracy (Jusyte, Mayer, Kunzel, Hautzinger, & Schonenberg, 2014; Sylvers, Brennan, & Lilienfeld, 2011). As such the impairment cannot be considered to be a secondary consequence of impairment in attention to particular regions of the face (cf. Dadds, Jambrak, Pasalich, Hawes, & Brennan, 2011). The impaired recognition of fearfulness and sadness is pervasive, applying also to vocal tones (R. J. R. Blair, Budhani, Colledge, & Scott, 2005; Stevens et al., 2001) and body postures (L. Munoz, 2009).

Consistent with the finding of unimpaired anger and disgusted expression recognition, individuals with psychopathy show amygdala responses to angry expressions similar to those shown by comparison individuals (the response to disgust expressions has not been tested; Carre, Hyde, Neumann, Viding, & Hariri, in press; Marsh et al., 2008; White, Williams, et al., 2012). Again consistent with the findings of impaired sad and particularly fearful expression recognition, patients with CD, particularly those with elevated psychopathic or CU traits, show reduced amygdala responses to sad (Passamonti et al., 2010) and fearful relative to neutral (Jones, Laurens, Herba, Barker, & Viding, 2009; Lozier, Cardinale, VanMeter, & Marsh, 2014; Marsh et al., 2008; Viding et al., 2012; White, Marsh, et al., 2012) expressions. Individuals with psychopathy also show reduced amygdala (and associated regions such as anterior insula and rostral medial frontal cortex) responses to pain cues (Lockwood, Bird, Bridge, & Viding, 2013; Lockwood, Sebastian, et al., 2013; Marsh et al., 2013; Michalska, Zeffiro, & Decety, 2015). Notably, the reduced amygdala response is seen even if the expression is presented too rapidly for attention to the eye region to have an influence on BOLD response (Viding et al., 2012).

Interestingly, studies have reported that elevated CU traits are associated with an increased risk for instrumental aggression (e.g., Thornton, Frick, Crapanzano, & Terranova, 2013). A recent study has shown that this association is mediated by the amygdala response to distress cues (Lozier et al., 2014). In short, reduced responsiveness to the distress of others is associated with symptoms of reduced guilt, lower empathy and increased levels of instrumental aggression.

If the amygdala is important for stimulus-reinforcement learning and stimulus-reinforcement learning is important for moral judgment, it can be predicted that the amygdala should be importantly involved when participants are making moral judgments. In line with this, fMRI research has shown that the amygdala is one of a core set of regions (that also include orbitofrontal cortex, insula, anterior cingulate cortex, precuneus and posterior cingulate cortex) consistently activated during moral judgment tasks (see, for a meta-analytic review, Boccia et al., 2016). Notably, this role of the amygdala in moral judgment appears to be disrupted in individuals with psychopathic traits (Harenski et al., 2010; Marsh & Cardinale, 2014; Marsh et al., 2011; Yoder, Harenski, Kiehl, & Decety, 2015). Indeed, while healthy individuals show a positive relationship between ratings of severity of transgressions and amygdala responsiveness, this is significantly less in individuals with psychopathy (Harenski, Harenski, et al., 2014; Harenski et al., 2010).

In short, stimulus-reinforcement learning allows the individual to learn the valence of a stimulus. It is thought that the sense of badness of care-based moral transgressions is because representations of these transgressions have been associated with an aversive unconditioned stimulus. It is argued here that this unconditioned stimulus is not pain induced by a caregiver’s physical punishments. Such punishments induce fear of the punisher rather than moral socialization (cf. relationship of child anxiety with use of punishment socialization strategies in Gershoff et al., 2010). The aversive unconditioned stimulus is the distress of the other individual. Healthy individuals learn to avoid actions/objects associated with another individual’s distress. Individuals with psychopathy show impairment in stimulus-reinforcement learning generally (at least aversive conditioning) and in their responsiveness to the distress of other individuals, functions reliant on the integrity of the amygdala. It is argued that these impairments disrupt their ability to learn the emotion-based sense of badness of care-based moral transgressions.

Response-outcome learning

There is a considerable animal and human literature on response-outcome learning and the use of response-outcome associations in reinforcement-based decision-making. An adequate review of this literature is beyond the scope of the current paper. However, useful recent reviews are available (O’Doherty, 2012; Rangel & Clithero, 2012; Schoenbaum, Takahashi, Liu, & McDannald, 2011). Regions of the brain critical for representing value in reinforcement-based decision-making include vmPFC, posterior cingulate cortex and striatum (Clithero & Rangel, 2014). Regions that appear to be particularly important in organizing avoidance responses on the basis of expected value information include anterior insula cortex and dorsomedial frontal cortex [dmFC] (Kuhnen & Knutson, 2005; White, Pope, et al., 2013). FMRI studies have implicated all of these regions (except striatum) in moral reasoning (see Boccia et al., 2016)

This paper concentrates on the regions most relevant to moral learning; i.e., the regions implicated in response-outcome learning. Both the striatum and vmPFC are thought to be critical for this type of learning (even if (ventral) striatum and vmPFC also play a role in stimulus-reinforcement learning; O’Doherty, 2012; Rangel & Clithero, 2012; Schoenbaum et al., 2011). An individual choosing to make a response relies on reinforcement expectancy information provided by these regions on the basis of prior learning. The individual will approach objects/initiate actions that are associated with positive reinforcement expectancies. The striatum is also critical for prediction error signaling (signaling the difference between the amount of reward or punishment received and the amount expected; Dayan & Balleine, 2002; O’Doherty, 2012). Prediction error signals are thought to spur reinforcement learning. The greater the prediction error, the greater the alteration in the reinforcement associated with the stimulus (Rescorla & Wagner, 1972).

Considerable data supports the suggestion of dysfunction in reinforcement-based decision-making in individuals with CD and ASPD. Patients with CD and ASPD show impairment on passive avoidance learning, reversal learning and risky choice tasks (Budhani & Blair, 2005; Budhani, Richell, & Blair, 2006; De Brito, Viding, Kumari, Blackwood, & Hodgins, 2013; Fairchild et al., 2009; Fisher & Blair, 1998; Newman & Kosson, 1986). These impairments are seen in both patients with CD with high and low levels of CU traits (White, Pope, et al., 2013); i.e., they are a risk factor for antisocial behavior rather than CU symptomatology. FMRI work suggests that these behavioral impairments relate to (i) reduced neural responsiveness to reward within both striatum and vmPFC (Cohn et al., 2015; Crowley et al., 2010; Finger et al., 2011; Rubia et al., 2009; White, Pope, et al., 2013); (ii) atypical responsiveness to punishment information within striatum and vmPFC in youth with CD (Crowley et al., 2010; Finger et al., 2011; Finger et al., 2008b; White, Pope, et al., 2013); and (iii) reduced ability to organize avoidance responses on the basis of expected value information include anterior insula cortex, dmFC and striatum in avoidance behavior (White, Fowler, et al., 2014; White, Pope, et al., 2013). With respect to emotional learning, patients with CD may show particular problems in prediction error signaling (Finger et al., 2011; White, Pope, et al., 2013).

Impairment in these regions is thought to underpin the problems shown by individuals with psychopathy on many moral judgment tasks. As noted above, moral judgment tasks are associated with activity within all these regions except striatum (Boccia et al., 2016). It is suggested that this is because a core component of moral judgment is representing the expected value of the (im)moral action to be considered. As the salience of the victim increases (i.e., the expected value of the action diminished because of representation of the cost of the victim), the action is more likely to be judged as “bad”.

Impairment in these regions is thought to underpin the increased risk for antisocial behavior. Disrupted representation of expected value, particularly when coupled with a relative failure to learn value on the basis of distress cues of others, is likely to mean that the individual is less likely to avoid actions that harm others. In addition, more generally, impaired learning about the value of actions will increase the probability of making poorer decisions leading to frustration and frustration-based aggression.

An example of how dysfunction in the representation of expected value might result in increased “immoral” (antisocial) behavior is provided by recent work with a modified version of the Ultimatum game (White, Brislin, Meffert, Sinclair, & Blair, 2013; White, Brislin, Sinclair, & Blair, 2014; White et al., 2016). In this task, the participant plays against named opponents (in reality a computer). At the beginning of each trial, one of these opponents has $20 and has to share some of this with the participant. The sharing might be fair ($10 each) or progressively unfair (up to $18 for the opponent and only $2 for the participant). In the classic ultimatum game, the participant simply chooses whether to accept or reject the offer. However, in this modified version, the participant can accept the offer or spend between $1 and $3 to retaliate. For every retaliatory $1 spent, $7 is withdrawn from the opponent. Unsurprisingly, healthy individuals increase their retaliation the more unfair the offer (White, Brislin, et al., 2013; White, Brislin, et al., 2014). But this effect of provocation is significantly greater on individuals with CD; individuals with CD punish more than healthy youth (White et al., 2016). Interestingly, retaliatory activity on this task is positively associated with reported reactive aggression in youth with CD. More interesting still, the impairment in youth with CD relates to reduced representing of the costs of retaliation within vmPFC (and reduced integrated activity of this region with the amygdala) (White et al., 2016). Indeed, this neuro-computational impairment underpins the association between retaliation on task and reported reactive aggression.

In short, response-outcome learning allows the individual to represent the value of committing a particular action. Moral judgments (and of course (im)moral behaviors) are actions. As such they rely on the individual’s ability to represent the value of the specific choices available. Regions implicated in response-outcome learning, and the representation of outcome, include vmPFC and striatum. Individuals with psychopathy show impairment in response-outcome learning. This impairment, together with the disruptions in learning the aversive value associated with actions that cause harm to others is thought to underpin their impairment on moral judgment tasks and increase their risk for instrumental antisocial behavior.

Caveats and Conclusions

The goal of this paper is to develop an argument regarding the role of emotion-based learning mechanisms in the development of moral judgment. However, before concluding it is worth considering a couple of caveats. First, as mentioned above, these emotion-based learning mechanisms can allow the individual to represent the “badness” of actions (i.e., their negative expected value based on their costs for the individual and particularly for others). However, these mechanisms alone would not allow the individual to represent the actions as immoral (cf. Nichols, 2002). As noted above, judgments of immorality require more than an aversive emotional response to the transgression. In this regard, it is worth very briefly considering the role of rostral medial frontal cortex. This region was implicated in moral judgment in the very earliest fMRI studies (Greene et al., 2001) and has continued to be implicated in more recent work (Boccia et al., 2016). This region is not typically implicated in emotional learning or the representation of expected value. However, it has been implicated in the underspecified process of self-referential processing (cf. Amodio & Frith, 2006). At least within the limitations of that underspecified process, one might consider that it is involved in accessing some of the self-relevant explicit norm information that is important for judgments of immorality (cf. Nichols, 2002). The learning of explicit norm information will involve forms of learning not covered by this paper.

Second, the current paper focused on stimulus-reinforcement learning on the basis of punishment, particularly if the “punishment” is the distress of a victim. However, there are other forms of stimulus-reinforcement learning, for example taste aversion learning that have implications for moral development. Taste aversion learning involves the individual associating the taste/sight of a certain food with symptoms caused by a toxic, spoiled, or poisonous substance (e.g., nausea). While the amygdala may play some role, the insula is particularly important for taste aversion learning; lesions of the insula block the acquisition and expression of taste aversion learning (Cubero, Thiele, & Bernstein, 1999). It is hypothesized that disgusted expressions may trigger taste aversion learning (R. J. R. Blair, 2007b). The individual will associate the aversive US of the expression with the stimulus that induced the expression. It has been argued that there are norms, often concerning matters of sexuality, that when transgressed are associated with disgust in those who hold the norm (Haidt, 2007). It appears plausible that the emotive force behind these norms may be acquired by experience of caregivers displaying disgust expressions when considering violations of these norms. This is interesting as while individuals with psychopathy show significantly reduced endorsement of care-based moral norms relative to comparison individuals, they show comparable endorsement of disgust-based norms (Aharoni et al., 2011; Glenn et al., 2009). This would suggest that the form of emotional learning underpinning the acquisition of disgust-based norms is intact in individuals with psychopathy.

Currently, the status of the functional integrity of the insula in individuals with psychopathy is unclear. Individuals with psychopathy have shown appropriate recruitment of the anterior insula and inferior frontal cortices in the context of response control tasks (Finger et al., 2008a; Marsh et al., 2011). However, youth with conduct problems generally (i.e., not youth with conduct problems and psychopathy in particular) reveal atypical responses in this region when generating avoidance responses relative to controls (White, Pope, et al., 2013). Importantly, for the position above, individuals with psychopathy generally do not show impairment in the recognition of disgusted expressions (Dawel et al., 2012; Marsh & Blair, 2008). This might mean that emotional learning on the basis of these expressions is intact in individuals with psychopathy even when emotional learning on the basis of distress cues is not. This would explain the preservation of disgust-based norms in the population in the presence of significant impairment in processing care-based norms.

Third, an aim of this paper was not to conclude that non-victim-based punishment – physical or non-physical (e.g., time outs or separation from valued resources) – was irrelevant for socialization. Such techniques may be particularly associated with the socialization of conventional (social order based; e.g., talking in class, tidying toys away after use) transgressions even if they are also used in response to care-based transgressions (Nucci & Nucci, 1982a, 1982b). Indeed, relatively intact sensitivity to angry expressions in individuals with psychopathy (Carre et al., in press; Marsh et al., 2008; White, Williams, et al., 2012) may mean that these norms are encoded in individuals with psychopathy relatively similarly to conventional transgressions (anonymous reviewer’s suggestion). This may give rise to findings that individuals with psychopathy treat care-based and conventional transgressions more similarly that comparison individuals (Arsenio & Fleiss, 1996; R. J. R. Blair, 1995, 1997; Nucci & Herman, 1982; Smetana, 1990).

Relatedly, and fourth, the paper did not consider fairness. This will be very briefly addressed here. Humans appear to show an expectation that resources will be equally distributed within the second year of life (Sloane, Baillargeon, & Premack, 2012) that at least partially reflects their experience with equitable sharing behaviors (Ziv & Sommerville, 2016). Humans show aversion to unequal resource allocations and may be will to sacrifice material payoffs to increase equality (Fehr & Schmidt, 1999). Individuals may show disadvantageous inequity aversion (DI), a desire to avoid receiving less than a peer, and advantageous inequity aversion (AI), a desire to avoid receiving more than a peer. DI emerges earlier in childhood and is seen in more cultures than AI (Blake et al., 2015). DI and AI choices are associated with comparable activations within dorsomedial frontal cortex, anterior insula cortex and dorsolateral prefrontal cortex (Guroglu, Will, & Crone, 2014). This has been argued to reflect a role of dorsomedial frontal cortex in responding to unexpected outcomes (including norm violations; cf. Alexander & Brown, 2011) and, anterior insula cortex in orchestrating potentially necessary changes in behavioral response (R. J. R. Blair & Cipolotti, 2000; Budhani, Marsh, Pine, & Blair, 2007); (see White, Brislin, et al., 2013). Interestingly, AI choices were additionally associated with activity in the ventral striatum and ventromedial prefrontal cortex regions, as noted above, linked to the computation of subjective value and reward (Guroglu et al., 2014). This potentially relates to the reward value that can be received for helping others (cf. Moll et al., 2006).

Retaliation to DI situations (even at cost to the self) is seen from the age of 4 years (McAuliffe, Blake, & Warneken, 2014). This likely reflects a desire to preserve one’s status relative to potential competitors (Crockett, Ozdemir, & Fehr, 2014; McAuliffe et al., 2014). Unfair resource allocations can be seen as disrespectful and, as such, may trigger retaliation (White, Brislin, et al., 2013). Interestingly, in healthy individuals magnitude of costly retaliatory behavior is inversely related to vmPFC activity, potentially reflecting representation of the costs of retaliation (White, Brislin, et al., 2013; White, Brislin, et al., 2014; White et al., 2016). As noted above, youth with CD (irrespective of level of psychopathy/callous-unemotional traits) show impaired representation of value within vmPFC (Crowley et al., 2010; Finger et al., 2011; Finger et al., 2008b; White, Pope, et al., 2013). In line with this, youth with CD show impaired representation of the costs of retaliation within vmPFC during retaliation to DI (White et al., 2016). Extent of this impaired representation is associated with their increased level of retaliation to DI (White et al., 2016). In short, according to the index of DI, individuals with CD (irrespective of level of psychopathy/callous-unemotional traits) show heightened levels of “fairness”.

However, at least in this culture, “fairness” relates to both DI and AI (Blake et al., 2015). Given this it is notable that in norm endorsement paradigms, individuals with psychopathy show reduced endorsement of fairness/justice-based norms (Aharoni et al., 2011; Glenn et al., 2009). I would argue that this reflects reduced sensitivity to the social reinforcement of others’ pleasure at receiving allocations and distress when these are removed. However this remains to be empirically determined.

In short, the current paper is not a complete view on the learning mechanisms necessary for moral judgment. It simply concentrates on the putative importance of stimulus-aversive conditioning on the basis of distress cue reinforcers and response-outcome learning. Impairments in these forms of emotional learning are thought to lead to the disruption in moral judgment particularly with respect to care-based transgressions and increased risk for antisocial behavior in individuals with psychopathy.