Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Sep 1.
Published in final edited form as: Biol Psychol. 2012 May 9;91(1):48–58. doi: 10.1016/j.biopsycho.2012.05.001

Characterizing the Anomalous Cognition-Emotion Interactions in Externalizing

Arielle R Baskin-Sommers 1, John J Curtin 1, Christine L Larson 2, Daniel Stout 2, Kent A Kiehl 3, Joseph P Newman 1
PMCID: PMC3407296  NIHMSID: NIHMS376452  PMID: 22579718

Abstract

Externalizing traits are characterized by exaggerated emotional (e.g., frustration, anger) and behavioral (e.g., drug seeking, reactive aggression) reactions to motivationally-significant stimuli. Explanations for this exaggerated reactivity emphasize attention, executive function, and affective processes, but the associations among these processes is rarely investigated. To examine these interactions, we measure fear potentiated startle (FPS; Experiment 1) and neural activation (Experiment 2) in an instructed fear paradigm that manipulates attentional focus, demands on executive functioning, and emotion. In both studies, exaggerated emotional reactivity associated with externalizing was specific to conditions that focused attention on threat information and placed minimal demands on executive functioning. Results suggest that a crucial cognition-emotion interaction affecting externalizing is the over-prioritization and over-allocation of attention to motivationally-significant information, which in turn, may impair executive and affective regulation.

Keywords: Externalizing, Fear Potentiated Startle, fMRI, Cognition-Emotion Interaction


The externalizing spectrum encompasses a heterogeneous mixture of behaviors, such as reactive aggression, excessive reward seeking, and low distress tolerance, which are united by overlapping genetic variance (Krueger, Markon, Patrck & Iacono, 2005). When particularly severe or persistent, these behaviors may culminate in diagnosable disinhibitory psychopathology, including conduct disorder, substance use disorders, and antisocial personality disorder. Although externalizing behaviors may result in diverse diagnoses, the essence of the externalizing construct is that these behaviors and disorders reflect a common predisposition or temperament associated with basic personality traits (e.g., low constraint, impulsivity, negative emotionality). Thus, specifying the risk associated with these predisposing personality traits has far-reaching implications for understanding and preventing a diverse array of externalizing behaviors and disinhibitory psychopathology (Iacono, Malone, McGue, 2008; Krueger et al., 2005; Gorenstein & Newman, 1980; Newman, 1997; Patrick & Zempolich, 1998; Zuckerman, 1998).

Clinical descriptions of externalizing highlight an exaggerated emotional reactivity that is presumed to underlie diverse externalizing behavior problems (Buckholtz et al., 2010a; Gorenstein & Newman, 1980; Krueger, Markon, Patrick & Iacono, 2005; Lejuez, Magidson, Mitchell, et al., 2010; Pridmore, Chambers & McArthur, 2005; Newman & Lorenz, 2003). This reactivity plays out in the context of rewards (Buckholtz et al., 2010a; Endres et al., 2011; Martin & Potts, 2004; Newman, Widom & Nathan, 1985; Nichols & Newman, 1986; Newman, Patterson, Howland & Nichols, 1990; Patterson, Kosson & Newman, 1987), stressful events (Taylor, Carlson, Iacono, Lykken et al, 1999; Verona, Patrick & Lang, 2002), and other motivationally-significant cues (i.e., drug cues; Volkow & Li, 2004).

Prominent explanations for the exaggerated reactivity associated with externalizing traits, commonly emphasize cognitive dysfunctions, such as poor executive functioning (e.g., Finn, Mazas, Justus & Steinmetz, 2002; Giancola & Tarter, 1999). Such explanations highlight the fact that limitations in cognitive control or working memory capacity can undermine a person's ability to regulate (i.e., bias) attention and, thus, increase the likelihood that motivational stimuli will hijack attention and control behavior (Bishop, 2009). In the absence of such executive control, reactions to motivationally-significant information are difficult to inhibit (e.g., negative mood, aggressive impulse, drug seeking) and become distracting regardless of their adaptive relevance (Bishop, 2009; MacCoon, Wallace & Newman, 2004). Consistent with these models, there is considerable evidence that externalizing traits are associated with executive functioning deficits (Dolan, Bechara & Nathan, 2008; Endres, Rickert, Bogg, Lucas & Finn 2011; Morgan & Lilienfeld, 2000; Ogilvie, Stewart, Chan & Shum, 2011). To the extent that this understanding of externalizing traits is correct, it follows that individuals with a high level of externalizing traits (i.e., externalizers) would be especially prone to exaggerated emotional reactivity under conditions that pit salient emotional information against a goal-directed focus (i.e., a go/no-go mixed-incentive learning task; see Endres et al., 2011; Patterson et al., 1987).

An alternative explanation for the exaggerated emotional reactivity and disinhibited behavior associated with externalizing traits involves the over-allocation of attention resources. Attention-based models of externalizing commonly assume that when these individuals encounter information in their environment that is motivationally-significant, they allocate attention to this information more strongly and thus, have more difficulty reallocating attention (Avila & Parcet, 2001; Derryberry & Reed, 1994; Farmer, Whitehead, Woolcock, 2007; Lubman, Allen, Peters & Deakin, 2008; Pessoa, 2001; Wallace & Newman, 1997). In this context, the strength of the over-allocation of attention is dependent on the interaction between the pre-existing priorities (i.e., readiness to perceive specific information) and the stimulus (i.e., what is actually seen in the environment). In other words, information that matches pre-existing priorities, like rewards, threat, or drugs, is strongly attended to, particularly when the individual is seeking or expecting those cues (see Bishop, 2009; Fox, Russo, & Dutton, 2002; Wallace & Newman, 1990 for examples of this attentional process in other populations). Consistent with this attentional-related proposal, a number of studies find strong attention allocation and difficulties re-allocating attention in the presence of motivationally-significant, goal-relevant cues among individuals high on externalizing traits (Avila & Parcet, 2001; Baskin-Sommers, Wallace, MacCoon, Curtin & Newman, 2010; Derryberry & Reed, 1994; Tiffany & Conklin, 2000; Wallace & Newman, 1997). Moreover, some theoretical accounts of this attentional response suggest that the over-prioritization and over-allocation of attentional resources, in turn, may impair executive functions that normally moderate responding, including inhibition, shifting, and control (Ellis & Ashbrook, 1988; Farmer et al., 2007; Keil et al., 2007). Thus, externalizing traits would be most associated with exaggerated emotional reactivity when anticipation of motivationally-significant stimuli increases the intensity of attention responses to environmental cues and interferes with the executive function-based regulation of this response. Importantly, such a process appears to play an important role in disrupting the inhibition of drug craving (Tiffany & Conklin, 2000; Volkow & Li, 2004) and violent responses (Blair, 2001).

In light of growing evidence that executive functioning, attentional, and emotional processes interact to yield the exaggerated emotional reactivity and disinhibited behavior associated with externalizing, it is increasingly important to clarify the specific and/or differential contributions of these factors for the disinhibited reactions associated with externalizing traits (see also Endres et al., 2011; Handley et al, 2011). A simple explanation for the emotional hyper-reactivity of externalizing individuals is that the intensity of their emotion reactions is so powerful and pervasive that it directly gives rise to this emotional reaction. However, the executive functioning perspective suggests that deficits in regulating competing demands (i.e., goal versus processing motivationally-significant information) impacts emotional reactivity, such that when an individual does not have the capacity to maintain their goal-directed focus and resolve the competition, motivationally-significant information hijacks attention and externalizers are likely to over-react to that emotional information. In contrast to these other explanations, the attention-based perspective attributes the exaggerated emotional reactions of externalizing individuals to the interaction between their motivational priorities and the occurrence of stimuli that match these priorities. When individuals with high externalizing traits encounter motivationally-significant stimuli that match their processing priorities, the stimuli attract strong attentional responses, consume resources, and yield emotional reactions that are dictated by the pre-existing processing priorities and the affective significance of the information. Although each of these psychobiological explanations is plausible, they predict different sources of emotional and behavioral disinhibition. Thus, it is important to examine the differential contributions of emotional hyper-reactivity, executive functioning, and attention to the disinhibited emotional reactions of externalizing individuals.

The goal of the present studies is to examine these alternative explanations for the exaggerated emotional reactivity associated with externalizing traits. To this end, we employed an instructed fear paradigm developed by Baskin-Sommers and colleagues (2011) (see Figure 1). In all conditions, trials contain colored boxes that are associated with the delivery of an electric shock (red box) or not (green box). Additionally, this paradigm orthogonally manipulates two factors: the focus of attention and the timing of goal-relevant stimulus presentation. In two conditions instructions are given that establish the threat information as goal-relevant and thus the primary focus of attention [Threat Focus conditions]. In two other conditions the threat information is peripheral to the primary goal (i.e., the primary goal is to categorize the case of a letter, yet the box predicting shock still appears in the trial) [Alternative Focus conditions]. The paradigm also manipulates the timing with which goal-relevant and distracting stimuli are presented (i.e., from the start of the trial (early) or after a distractor is presented (late)). Together, these manipulations result in four conditions (Early Threat Focus, Late Threat Focus, Early Alternative Focus, Late Alternative Focus). Based on this paradigm and the theoretical alternatives outlined above (i.e., general hypersensitivity to emotion, weak executive functioning, and over-allocation of attention), we investigated three hypotheses that aim to characterize and disambiguate the cognition-emotion interactions that culminate in the disinhibited emotional reactivity associated with externalizing:

Figure 1.

Figure 1

Instructed fear paradigm. The task consisted of four conditions in which focus of attention and timing of attentional focus was manipulated: (A) Early Threat Focus, (B) Late Threat Focus, (C) Late Alternative Focus, and (D) Early Alternative Focus. During Threat Focus participants attended to the color of a box predicting shock administration (red = potential shock, green = safe). For Alternative Focus, participants were instructed to attend to the case of a letter stimulus. During Early trials the stimulus to be attended to was presented first and during Late trials it was presented second. For Study 1 80 trials of each condition and for Study 2 60 trials of each condition were presented in a blocked format with the order counterbalanced across participants. For Study 1 only, white-noise startle probes were presented at 1,400 ms to measure fear-potentiated startle.

Hypothesis 1

To the extent that externalizing traits are associated with a global hyper-reactivity to motivationally-significant information, individuals high on these traits should display greater emotion in all conditions (i.e., main effect), regardless of executive functioning or attentional focus.

Hypothesis 2

To the extent that externalizing traits are associated with a fundamental deficit in executive functioning then emotional hyper-reactivity would be apparent especially under conditions that require participants to maintain a goal-directed focus of attention despite competing demands to focus attention on the highly distracting threat information (i.e., the two Alternative Focus Conditions). By contrast, in the Threat Focus conditions the goal is to focus on the threat information while ignoring an irrelevant, and actually meaningless, letter. Because the letter lacks the bottom-up salience of the threat information and does not compete meaningfully with the primary goal (i.e., focusing on the threat), the executive functioning hypothesis does not primarily predict emotional dysregulation in the Threat Focus conditions.

Hypothesis 3

To the extent that emotional hyper-reactivity in externalizing relates to over-allocation of attention to motivationally-significant stimuli, this problem would be most apparent under conditions that activate a specific motivational focus (i.e., focusing on threat) and then initially display stimuli that match that motivational focus (goal-relevant threat cues; i.e., Early Threat Focus). Paradoxically, though, when information that does not match the motivational set appears, the over-allocation of attention in anticipation of perceiving motivationally-significant stimuli would reduce the flexibility of attention, interfere with reallocation of attention to actual goal-relevant information (i.e., threat), and thus diminish emotion responses to threat-relevant stimuli. By contrast, in the Alternative Focus conditions the threat information is peripheral to the primary goal, thus participants with high and low levels of externalizing traits would be expected to allocate attention to both goal-relevant (i.e., letters) and intrinsically-relevant (i.e., colored boxes) stimuli and display comparable emotional reactivity in the Alternative Focus conditions.

To assess these alternative hypotheses, we conducted two studies. The first, used fear potentiated startle (FPS), a psychophysiological measure of fear reactivity, to examine the effect of specific cognitive-affective interactions on externalizing-related differences in emotional reactivity. The second study used functional magnetic resonance imaging (fMRI) in a different sample as a means of extending the findings of Study 1 and evaluating the neural correlates associated with these cognition-emotion interactions and reactivity.

Study 1: Fear Potentiated Startle

Methods

Participants

Participants were 84 male inmates from a maximum-security prison in southern Wisconsin. First, participants who were older then 45 years of age were eliminated because the expression of externalizing traits (i.e., impulsivity) has been found to decline with advancing age (Hare, Harpur & Hakstian, 1990). Also, some research suggests that memory and cognitive functioning begins to decline starting around age 45 (Singh-Manoux et al., 2012). Second, to increase the likelihood that participants had the intellectual aptitude to complete self-report measures and laboratory tasks, we excluded those with scores below 70 on the Shipley Institute of Learning (Zachary, 1986) estimate of intelligence. Third, based on institutional records, we disqualified individuals with clinical diagnoses of schizophrenia, bipolar disorder, or psychosis NOS. Lastly, anyone who was currently using psychotropic medications was excluded because this class of medication may interfere with startle reactivity (see Table 1 for descriptive statistics).

Table 1.

Means, Standard Deviations, and zero-order correlations with externalizing for relevant descriptive variables.

Study 1 (N=84) Study 2 (N=37)

Mean St Dev r Mean St Dev r
Demographic
Age (years) 31.82 7.21 -.18 32.76 6.66 .03
Shipley Estimated IQ 102.38 9.88 -.13 102.22 11.40 -.17
Externalizing Measures
NA-CON Composite Raw Score -21.50 29.47 -- -- -- --
ESI Raw Score -- -- -- 236.00 66.34 --
Antisocial Personality Disorder (# of symptoms) 7.63 3.49 .52* 9.16 5.19 .71*
Conduct Disorder (# of symptoms) 3.56 2.69 .46* 4.36 3.49 .67*
Adult Antisocial (# of symptoms) 4.00 1.40 .40* 3.92 1.97 .65*
Externalizing Behaviors and Characteristics
Drug Charges (supply and use) 1.79 2.41 .05 1.43 2.09 .47*
Assault Charges 1.59 2.17 .23* .97 1.28 .36*
Self-Report Alcohol/Drug Use 9.70 5.04 .15 10.00 4.50 .66*
Sensitivity to Punishment 10.27 6.07 .19 -- -- --
Sensitivity to Reward 13.51 4.80 .39* -- -- --
Digit Backwards 6.76 2.14 -.11 7.10 2.50 -.11

Note: Flagged cells indicate a significant (p<.05) zero-order correlation involving Externalizing measures (*). Negative Affect-Constraint=NA-CON (negative indicates larger constraint value than negative affect value); ESI=Externalizing Spectrum Inventory; Self-Report Alcohol/Drug Use: Michigan Assessment Screening Test for Alcohol and Drugs. Sensitivity to Punishment/Reward was measured by the Caseras, Avila & Torrubia (2002) Sensitivity to Punishment/Sensitivity to Reward- Revised scale.

Externalizing was assessed using the Multidimensional Personality Questionnaire-Brief (MPQ-B), a 155 item self-report questionnaire (Patrick, Curtin & Tellegen, 2002). The higherorder factors of Constraint (CON) and Negative Affect (NA) of the MPQ-B are related to externalizing tendencies (Eisenberg, Guthreie, Fabes et al., 2000; Krueger, Caspi & Moffitt, 2000; Miller, Greif & Smith, 2003). Those high on negative emotionality have tendencies towards stress reactivity, aggression, and alienation. Individuals low on constraint are characterized by impulsivity, low harm avoidance, and disinhibition (CON reverse scored). Here, we standardized and mean-centered NA and CON scales so that each was weighted equally. Then, we created a composite of NA and CON (i.e., high NA and low CON) as an index of externalizing psychopathology (Cronbach's alpha =.84).

Procedure

Individuals meeting the inclusion criteria (see Participants section) were invited to participate in an ongoing study. All participants provided written informed consent according to the procedures set forth by the University of Wisconsin–Madison Human Subjects Committee. Participants were tested by one of two male experimenters in a quiet and private testing room on prison grounds. Before beginning the main task, the intensity of shocks received during the experimental session was calibrated to a participant's subjective shock sensitivity (see Shock sensitivity evaluation below). Then, participants were given verbal instructions about the instructed fear-conditioning task. During the main task, participants were reminded on the computer of the instructions for each condition prior to the start of that condition.

Shock sensitivity evaluation

To control for individual differences in shock sensitivity, the intensity of shocks received during the instructed fear task was calibrated to the participants' individual subjective shock sensitivity prior to the beginning of the task. A series of electric shocks was administered in increasing intensity to the fingers of their non-dominant hand. Participants reported two intensity anchors: the first intensity that they considered uncomfortable and then the maximum intensity level that they could tolerate. The series was terminated when they reached their maximum intensity level. The shock intensity administered during the experimental session was calibrated to the reported maximum intensity level.

Instructed fear-conditioning task

Presentation of stimuli and recording of responses were controlled by DMDX (Forster & Forster, 2003) and NeuroScan Synamps2 amplifiers and acquisition software. The instructed fear task consisted of four conditions with 80 trials per condition. The task conditions crossed focus of attention and timing of goal-relevant information, yielding the Early Threat Focus, Late Threat Focus, Early Alternative Focus, and Late Alternative Focus conditions (Figure 1). At the start of each condition participants were given instructions about which stimulus was goal-relevant. In the two Threat Focus conditions, participants attended to the color of a box predicting shock administration (red = potential shock, green = safe, no chance of shock). In the two Alternative Focus conditions, participants were instructed to attend to the case of a letter stimulus and the case of the letter was unrelated to administration of electric shocks. In both condition types, the intensity of shocks was calibrated to participants' subjective tolerance (a procedure administered prior to the task). Also, each condition was presented in a blocked format with the order counterbalanced across participants (Note: Order of condition did not interact significantly with the effects reported below).

In each trial the box (red or green) and letter (uppercase “N” or lowercase “n”) were presented sequentially with the order varying according to condition. During Early trials the goal-relevant stimuli (i.e., stimuli individuals were to respond to at the end of the trial) were presented first and during Late trials goal-relevant stimuli were presented second after goal-irrelevant information was on the screen. Every trial in all conditions started with a fixation cross. The first stimulus appeared alone at 200 ms, and then the second stimulus appeared concurrently with the first at 400 ms. Following the offset of the stimuli, a blank screen appeared. Shocks were presented during the last 200 ms of the blank screen (at 1400ms). Finally, 1,600 ms after the onset of the first stimulus participants were signaled to press a button indicating whether a probe word, “Red” or “Green” for the Threat Focus conditions and “Upper or “Lower” for the Alternative Focus conditions matched the stimulus presented during that trial.

Across conditions, fifteen percent of the trials where shocked. In addition to the shocked trials, sixty-four startle-eliciting noise probes (50ms, 102dB white noise burst with near instantaneous rise time) were presented 1400ms after the onset of the first stimulus. The noise probes were equally distributed across threat/no-threat trials in all four task conditions so that each participant received 16 noise probes (8 threat and 8 no-threat) per task condition. Noise probes were also equally distributed across upper/lower case trials. Noise probes were separated by a minimum of 13s and never occurred on the same trial as shock administration.

Startle Measurement

Startle eyeblink electromyographic activity was sampled and amplified with Neuroscan Synamps2 amplifiers (Comupmedics Inc., North Carolina) at 2000Hz with a bandpass filter (30500Hz; 24dB/octave roll-off) from electrodes placed on the skin overlaying the orbicularis oculi muscle under the right eye. Data reduction and processing followed published guidelines (Blumenthal, Cuthbert, Filion, Hackley et al., 2005). Offline processing included signal epoching (-50 ms to 250 ms period surrounding noise probe), rectification and smoothing (30 Hz lowpass filter following rectification), and baseline correction. Trials with no responses (i.e., peak of startle response is less than or equal to the maximum response during the baseline period) or deflections greater than 10 μV (i.e., artifact) in the 50 ms pre-probe baseline were rejected. Prior to arriving at the final sample of 84, two participants were removed from the sample because more than 20% of their probe trials contained excessive artifact. Peak eyeblink response between 20 and 120 ms post-probe onset was scored relative to mean 50 ms pre-probe baseline. Fear potentiated startle (FPS) was scored as the difference between the threat (red box) and no threat (green box) trials.

Data Analysis

All analyses were performed with SPSS software (SPSS Inc, Chicago, Illinois) and were analyzed in a General Linear Model (GLM) with Focus and Time as within-subject categorical factors and Externalizing (Externalizing; continuous and standardized) as a between-subject quantitative factor. Fear response to threat cues was indexed by FPS, which was calculated by subtracting blink-response magnitude to probes following green (no threat) boxes from blink-response magnitude to probes following red (threat) boxes. Potential outliers were detected using studentized residuals with Bonferroni corrected p-values <.05 and were subsequently excluded from analysis.

Results

The effect of Focus on FPS was not significant (F(1,83)=.74, p=.39); nor was the effect of Time (F(1,83)=.1.87, p=.18) indicating that FPS was comparable across these task manipulations. The main effect of Externalizing was not significant (F(1,82)=.04, p=.842), indicating that Externalizing scores were not consistently related to FPS across conditions (Hypothesis 1).

However, a significant three-way interaction indicated that the Externalizing effect was moderated by both focus of attention and timing of goal-relevant information (F(1,82)=8.84, p=.004, pη2=.097). In order to unpack the significant three-way, we examine the two-way interaction between timing and Externalizing within each focus (i.e., threat and alternative). Additionally, we report the linear regression coefficients as an index of effect size within each interaction (Table 2 shows raw startle responses as a function of threat, condition, and externalizing)1.

Table 2.

General Linear Model Point Estimates for Startle Responses in the Four Conditions of the Experiment

Early alternative focus Late alternative focus Late threat focus Early threat focus
No threat Threat FPS No threat Threat FPS No threat Threat FPS No threat Threat FPS

Low 43.20 60.24 17.04 38.60 53.52 14.92 34.02 59.91 25.89 38.00 44.75 6.75
Externalizing (8.09) (11.00) (5.97) (8.13) (10.60) (4.84) (8.21) (10.63) (5.17) (7.81) (10.02) (4.40)
High 49.25 59.23 9.99 44.83 61.26 16.43 49.62 58.87 9.25 46.82 70.14 23.32
Externalizing (8.09) (11.00) (5.97) (8.13) (10.60) (4.84) (8.21) (10.63) (5.71) (7.81) (10.02) (4.40)

Condition 48.07 64.87 16.80 44.19 61.32 17.13 43.31 62.41 19.10 45.29 62.39 17.10
Mean (4.48) (6.61) (3.78) (4.72) (6.08) (2.86) (4.84) (5.19) (3.32) (4.73) (6.16) (2.79)

Note: Fear-potentiated startle (FPS) was calculated by subtracting blink-response magnitude to probes following green (no threat) boxes from blink-response magnitude to probes following red (threat) boxes in each of the four conditions. There was a significant difference between the Threat and No Threat stimuli across conditions, F(1,86)=46.72, p<.001, pη2=.35, which indicated that startle was greater in the Threat trials than the No Threat trials. Because we analyzed externalizing continuously instead of using an extreme-groups design, the values presented are point estimates (i.e., estimated using regression analyses) for low and high externalizing points (1.5 SD below and above the sample mean NA-CON total score, respectively) on the distribution. Numbers in parentheses are standard errors.

In contrast to the predictions based on the executive functioning hypothesis, there was no main effect of Externalizing in the Alternative Focus conditions (F(1,82)=.11, p=.74) (Hypothesis 2). Also, there was no difference in the relationship between Externalizing and FPS in the Early Alternative Focus versus Late Alternative Focus condition, F(1,82)=1.35, p=.249, pη2=.016. However, in line with the attentional predictions there was a significant Externalizing-related difference in FPS in the Early Threat Focus versus Late Threat Focus condition, F(1,82)=11.62, p=.001, pη2=.12 (Hypothesis 3). Moreover, the simple effect of Externalizing within the Early Threat Focus condition was significant (B=5.52 p=.027, pη2=.06), indicating that individuals high on Externalizing demonstrated significantly greater FPS in the Early Threat Focus condition (Figure 2). The relationship between Externalizing and FPS in the Late Threat Focus condition was a trend (B=-5.55, p=.084, pη2=.04), suggesting that externalizers displayed diminished FPS in this condition2.

Figure 2.

Figure 2

Fear-potentiated startle (FPS) as a function of Externalizing (±1.5 SD from the mean) and condition. Error bars represent +/- one standard error for FPS point estimates for low and high Externalizing.

Discussion

Based on previous research, we might have expected to find a global emotional hyper-reactivity in externalizing, but the present study did not produce such evidence. That is, across conditions, externalizing was not consistently associated with exaggerated reactivity to emotion stimuli. Similarly, although research frequently identifies executive functioning deficits as the underlying process responsible for externalizing dysfunction, in the current study, individuals high on externalizing traits were as able as individuals low on externalizing traits to focus on the goal-relevant information and resolve competing demands by ignoring distracting emotion cues in the Alternative Focus conditions.

The present results do, however, appear consistent with the attention hypothesis. According to this model, externalizers over-prioritize goal-relevant, motivationally-significant information and so are primed to allocate attention more strongly than other individuals to that information. Thus, when the presentation of goal-relevant information coincides with their pre-existing priorities, individuals high on trait externalizing display emotional hyper-reactivity (i.e., significantly more FPS in the Early Threat Focus condition). Conversely, when a stimulus occurs that is at odds with this goal (i.e., mismatch between pre-existing prioritization of motivationally-significant information and actual stimulus), the over-prioritization of attention toward motivationally-significant information might diminish the resources available for re-allocating attention (i.e., employing executive control) to facilitate goal-directed behavior. Ultimately, this attentional response would inhibit fluent processing, disrupt the processing of goal-directed motivationally-significant information, and undermine the threat response. In the present study, there was some evidence of the affective disruption, presumably as a consequence of the attentional response, in the trend-level effect in Late Threat Focus condition. Overall, this attentional response appears to exaggerate emotional reactivity when information is congruent with pre-existing processing priorities, but interfere with resource allocation and processing when information in incongruent with those priorities.

It is worth noting that the effect of this attention response is not evident in the Alternative Focus conditions, suggesting that the specific attentional abnormality operating in externalizing is not simply associated with an over-allocation of attention to motivationally-significant information. If this was the case, then emotion hyper-reactivity should have been apparent when the threat appeared first but was not goal-relevant (Late Alternative Focus), but there were no externalizing-related differences in this condition. Rather, the results indicate that the attentional anomaly is only apparent when motivationally-significant information conforms to goal-related expectations. Such findings suggest the match between pre-existing priorities and the presentation of information that fits those priorities is the key factor responsible for over-allocation of attention. However, it is possible that this over-prioritization of attention is also present in the Alternative Focus conditions, but fails to engender emotion hyper-reactivity in externalizing individuals. For instance, it is possible that individuals with high externalizing traits initially over-allocate attention to the goal-irrelevant threat cues (i.e., colored box), but do not dwell on the threat-relevant information when their pre-potent focus involves detecting goal-relevant, neutral cues (i.e., letter case). Thus, motivationally-significant cues may be less likely to command their attention when they are actively involved in pursuing alternative goals.

Overall, these findings specify an important dynamic underlying the dysfunctional cognitive-affective interactions associated with externalizing traits. Specifically, an over-prioritization and over-allocation of limited capacity processing resources to motivationally-significant stimuli exacerbates emotional reactions and may, paradoxically, decrease cognitive resources available for regulating executive control and emotional reactions. Further, the emotional dysregulation seems most apparent when emotion information is related to the primary goal (e.g., in this task during Threat Focus conditions, in real life possibly when drug seeking, when gambling to attain money). The fact that the response to threat mimics the exaggerated responsivity to rewards commonly seen in externalizing suggests that this attentional response may be a general mechanism operating in externalizing. Given that this is a new finding, though, it is important to replicate it and expand our understanding of the mechnisms at play.

Study 2: fMRI

Results from Study 1 support a novel framework for understanding the emotionally reactive disinhibition associated with externalizing. In light of the newness of this proposal it is important to replicate the findings and examine the neural processes implicated in this cognition-emotion interaction. The focus of Study 2 is to measure brain activity in key regions associated with affective responding, executive functioning, and attention during performance on the task that revealed externalizing-related differences in Study 1.

The amygdala and regions of the prefrontal cortex play an important role in emotional processing, and have been found to be differentially activated in individuals with externalizing psychopathology (Lindquist, Wagner, Kober, Mliss-Moreau & Barrett, in press; Phan et al., 2003; Walter, Malthia, Wiebking, Rotte, Templemann, et al., 2009). The amygdala plays a crucial role in the emotional modulation of attention (Anderson & Phelps, 2001) and the extended amygdala is a key modulator of measures such as fear potentiated startle (Anderson & Phelps, 2001; Davis, Falls, Campeau, & Kim, 1993; Davis & Whalen, 2001; Grillon, Ameli, Goddard, Woods, & Davis, 1994). While many regions of the prefrontal cortex are important for attention, executive function and detection of emotional salience, recently, a number of studies have identified the dorsomedial prefrontal cortex (dmPFC) as a key region in the allocation of attention to motivationally-significant information, particularly as it relates to the expression of fear (Etkin, Egner & Kalisch, 2011; Hanson et al., 2010; Milad, Quirk, Pitman, Orr, Fischl & Rauch, 2007; Walter, Malthia, Wiebking, Rotte, Templemann, et al., 2009). This region also aids in the monitoring and resolution of conflict in the environment (Blair, 2008; Cohen, Botvinick & Carter, 2000).

Specifically with regard to externalizing, some research highlights an externalizing-related association between reduced gray matter volumes in the right amygdala and functional amygdala hyperactivity (Benegal, Antony, Venkatasubramanian, & Jayakumar, 2007; Coccaro, et al., 2007; Raine, Meloy, Bihrle, Stoddard, LaCasse, & Buchsbaum, 1998; Sterzer, Stadler, Krebs et al., 2005). Additionally, studies often report prefrontal (i.e., dmPFC) abnormalities, both hypo-and-hyper activation, among individuals high on externalizing traits (Bufkin & Luttrell, 2005; Davidson, Pizzagalli, Nitschke & Kalin, 2003; Herpertz, Huebner, Marx, Vloet, Fink, et al., 2008; Raine, Lencz, Bihrle, et al., 2000).

In the present study, we investigated whether the pattern of activation in key regions such as the amygdala and dmPFC would track the predictions related to the attention model and results reported in Study 1. If individuals with high trait externalizing are associated with an over-allocation of attention and hyper-reactivity to goal-relevant-threat cues as found in Study 1, then they should display increased amygdala and dmPFC activation in the Early Threat Focus condition. However, in the Late Threat Focus condition amygdala and dmPFC activation would be reduced, reflecting a disruption in attentional-affective processing when information that does not match the motivational set appears. Lastly, based on the results from Study 1 and the attentional hypothesis, there should be no specific externalizing-related effects (i.e., comparable amygdala and dmPFC activation in low and high trait externalizing individuals) associated with the Alternative Focus conditions, as these conditions are more associated with executive functioning demands (i.e., competition between goal-relevant focus and peripheral goal-irrelevant threat information).

Methods

Participants

Participants were 37 male inmates from a maximum-security prison in southern Wisconsin (Note: This sample was drawn from a different institution than the sample in Study 1). In addition to the exclusion criteria listed in Study 1, participants were also excluded from the imaging study if they had contraindications for MRI (e.g., ferromagnetic or electronic implants).

The fMRI sample was collected at a different institution, where other measures of externalizing were administered. For this sample, externalizing was measured using the Externalizing Spectrum Inventory (ESI; Hall et al., 2007). The ESI is a 100-item self-report questionnaire developed to assess a broad range of behavioral (i.e., substance use) and personality characteristics (i.e., alienation, rebelliousness, impulsivity) associated with the externalizing spectrum of psychopathology. The 100-item version was derived from Krueger et al's (2007) 415-item self-report measure and is correlated r=.98 with the original measure (Krueger, Markon, Patrick, Benning et al., 2007) 3. For this sample the internal consistency (Cronbach's alpha) was .96.

Procedure

Presentation of stimuli and recording of responses were controlled by ePrime software (Psychological Software Tools). Stimuli were presented via a rear projection screen and responses were recorded with a fiber optic response box. Prior to beginning the experiment, the intensity of shocks received during the experimental session was calibrated to a participant's subjective shock sensitivity.

Shock Sensitivity

The shock sensitivity procedure was the same as Study 1. However, due to the scanner environment shocks were delivered via an AC source (Contact Precision Instruments, Boston, MA) through two Ag/AgCl electrodes placed over the right tibial nerve over the right medial malleolus (right ankle).

Instructed fear-conditioning task

The instructed fear task was essentially like the one presented in Study 1. However, in order to adapt the task to a scanner setting, each condition had only 60 trials and a fixation cross was displayed for 400 to 1200 ms (mean = 800 ms) at the beginning of each trial. Twenty percent of the trials where shocked and no white noise probes where administered in the imaging study.

Image Acquisition

Data were collected on the Mind Research Network's 1.5 T Siemens Avanto scanner with advanced SQ gradients (max slew rate 200T/m/s, 346 T/m/s vector summation, rise time us) with a 12-element head coil. EPI echoplanar functional images were collected according to the following parameters: TR/TE (2000/39 ms, flip angle 75 °, FOV 24 × 24 cm, 64 × 64 matrix, 3.75 × 3.75 mm in plane resolution, 5 mm slice thickness, 27 axial oblique slices). Head motion was limited using padding. Participants with motion greater than 3 mm were excluded from analysis. High-resolution MPRAGE structural images were obtained to cover the whole brain with 128 1-mm sagittal slices, 8° flip angle and 24 cm FOV.

Image Preprocessing

All fMRI data pre-processing and analysis was conducted with AFNI software (Cox, 1996). After image reconstruction and slice-time correction, rigid-body motion correction was done in three translational and three rotational directions, with all images registered to the fifth functional image. The amount of motion in these directions was estimated and then included as regressors in subsequent analyses. Participants with more than 3 mm displacement were excluded. Functional timeseries were deconvolved using a tent function including 8 tents spanning the 14 s following the first stimulus onset. Mean parameter estimates (beta) for tents two through five were calculated. A spatial blur of 4 mm full width half maximum was applied and images were resampled to 1 mm3 voxels. Anatomical images for each participant were nonlinearly transformed to Talairach space using the TT_N27 template and each functional run was transformed to the Talairach-transformed anatomical image. One participant did not have usable anatomical scans so the functional images were transformed to a standard Talairach (TT_N27) template. Threat minus safe difference scores were calculated for each condition for each subject. For all whole brain analyses cluster thresholding was applied to correct for multiple comparisons. Using a voxel-based p < .025 and nearest-neighbor selection criteria, clusters greater than 562 mm3 achieved a whole-brain corrected p value < 0.05.

Data Analysis

Participants were grouped into high and low externalizing traits based on a median split of externalizing scores on the ESI (N=19 individuals low on externalizing; N=18 individuals high on externalizing). First, a Group × Condition ANOVA was run. Based on this ANOVA, parameter estimates for regions relevant to the hypothesized mechanisms were extracted for significant clusters identified in the Group × Condition interaction. Analyses were focused on amygdala and dmPFC clusters, however Table 3 displays all clusters with a significant Group × Condition interaction. Lastly, simple effects (e.g. high vs. low externalizing in Threat Focus conditions) were examined. For all whole brain analyses, cluster thresholding was applied to correct for multiple comparisons using the Alphasim procedure in AFNI with the following input parameters: voxel-based p < .025, nearest-neighbor selection criteria, and estimated smoothing parameters of x = 4.066, y = 4.621, and z = 4.755. Spatial smoothness was estimated by calculating full width half max on the residual timeseries (noise) generated following deconvolution.

Table 3.

Significant clusters from the Group × Condition Interaction showing differences for the Threat Focus conditions (p < .05 corrected).

Region x y z Cluster size (voxel mm3) F value p value (corrected)
FRONTAL
 L Dorsal Medial Frontal Gyrus* BA 6 -8 -18 54 321 4.79 .004
 L Superior/Medial Frontal Gyrus* BA 10 15 64 -1 1608 3.72 .014
 L Middle Frontal Gyrus* BA 9 -39 34 27 214 4.32 .007
 L Middle Frontal Gyrus* BA 10 -38 51 -1 155 4.17 .008
 R Inferior Frontal Gyrus* BA 47 32 31 -12 131 4.20 .008
TEMPORAL
 R Superior Temporal Gyrus* BA 39 47 -49 17 281 4.44 .006
 L Superior Temporal Gyrus* BA 13 -50 -41 20 222 4.61 .006
 L Superior Temporal Gyrus* BA 41 -40 -39 11 148 4.73 .005
 R Inferior Temporal Gyrus* BA 20 62 -23 -18 167 4.12 .008
 R Middle Temporal Gyrus* BA 39 45 -69 19 147 4.09 .009
Parietal/Occiptal
 R Cuneus/Precuneus* BA 31 8 -69 18 1012 4.30 .007
 L Posterior Cingulate* BA 30 -12 -54 18 338 4.33 .007
 L Middle Occipital Gyrus* BA 18 -34 -85 4 155 4.25 .007
 L Middle Occiptal Gyrus* BA 30 -29 -58 7 123 4.07 .009
 R Lingual Gyrus/Culmen BA 19 19 -59 0 570 3.75 .013
OTHER REGIONS
 R Thalamus* 17 -26 6 349 4.40 .006
 L Thalamus* -11 -21 18 261 4.26 .007
 R Caudate* 24 -18 24 311 4.22 .007
 L Substantia Nigra* -8 -26 -11 224 4.64 .004
 R Cerebellum* 20 -54 -13 198 4.23 .007
 R Amygdala/Parahippocampal Gyrus* 15 -6 -12 98 3.84 .012

Note: The regions flagged in the table are significant contrasts based upon clusters from the Group X Condition interaction. These contrasts highlight the significant externalizing-related effects in the Threat Focus conditions.

*

denotes greater activation for high externalizers in the Early Threat Focus condition.

denotes greater activation for low externalizers in the Late Threat Focus condition.

Results and Discussion

Consistent with the FPS results, there was no main effect of externalizing on amygdala (F(1,35)=.02, p=.88) or dmPFC (F(1,35)=1.57, p=.22) activation. Significant three-way interactions in the amygdala (F(1,35)=6.77, p=.01, pη2=.16) and dmPFC (F(1,35)=4.74, p=.04, pη2=.12), respectively, indicated that the Externalizing effect was moderated by both focus of attention and timing of goal-relevant information. Next, in order to unpack the significant three-way interactions, we examine the two-way interactions between timing and Externalizing within each focus (i.e., threat and alternative). Of note, like Study 1, none of the Group × Condition interactions in the Alternative Focus conditions were significant (amygdala: F(1,35)=.40, p=.53; dmPFC: F(1,35)=.59, p=.45).

Moreover, like the FPS results in Study 1, there were significant amygdala (F(1, 35)=28.55, p<.001 uncorrected, pη2=.45; peak for Group × Condition interaction cluster: x= 15, y=-6, z=-12; a region including parahippocampal gyrus) and dmPFC (F(1,35)=13.63, p=.001, pη2=.28; peak for Group × Condition interaction cluster: x=-8, y =-18, z=54) differences found for the Early versus Late Threat Focus contrast. In the Early Threat Focus condition, individuals high on externalizing exhibited significantly more amygdala activation than individuals low on externalizing (F(1,35)=7.81, p=.008, pη2=.18). Conversely, in the Late Threat Focus condition, individuals high on externalizing exhibited significantly less amygdala activation than individuals low on externalizing (F(1,35)=8.69,p=.006, pη2=.20; Figure 3). Second, in the Early Threat Focus condition individuals high on externalizing traits also exhibited significantly more activation in the dmPFC than individuals low on externalizing (F(1,35)=9.16, p=.005, pη2=.21), whereas in the Late Threat Focus condition individuals high on externalizing exhibited significantly less activation in the dmPFC region than individuals low on externalizing (F(1,35)=9.67, p=.004, pη2=.22; Figure 4)4.

Figure 3.

Figure 3

(A) Brain image depicting significant amygdala (x= 15, y=-6, z=-12) activation for the Group x Condition interaction (B) The bar graph depicts mean threat minus safe parameter estimates for each condition based on the amygdala cluster demonstrating a significant Group x Condition interaction. Error bars represent standard error of the mean. For all brain images clusters depicted survive whole-brain correction for multiple comparisons at p < .05.

Figure 4.

Figure 4

(A and B) Brain image depicting significant dmPFC (x=-8, y =-18, z=54) activation for the Group x Condition interaction (C) The bar graph depicts mean threat minus safe parameter estimates for each condition based on the dmPFC cluster demonstrating a significant Group × Condition interaction. Error bars represent standard error of the mean. For all brain images clusters depicted survive whole-brain correction for multiple comparisons at p < .05.

The results from this imaging study are consistent with the FPS findings from Study 1 and reinforce the tenets of the attention hypothesis. Again there was no evidence for an externalizing-related difference in global emotional reactivity (i.e., main effect across conditions) or deficit in executive functioning (i.e., effects in the Alternative Focus conditions). However, like in Study 1, individuals with high externalizing traits compared to individuals with low externalizing traits displayed greater amygdala and dmPFC activation when asked to focus directly on the threat and the threat appeared first (i.e., when there was a match between pre-existing priorities and presentation of motivationally-significant, goal-relevant information). Both amygdala and dmPFC are important neural regions reflecting attention-emotion interactions (Etkin et al., 2011). Given the role of these regions in integrating attentional and affective responses, increased activation in the Early Threat Focus condition among externalizing individuals may reflect, at a neural level, the dysfunctional process that occurs when there is a match between pre-existing priorities and goal-relevant stimuli. Moreover, in the Early Threat Focus condition, brain regions implicated in the mesolimbic dopamine system (i.e., Caudate and Substantia Nigra, see Table 3), a system involved in integrating information about motivational salience, were hyper-activated in the high externalizing group. This evidence is consistent with previous work that found significantly greater neural activation in this system among externalizers (i.e., high impulsive traits) during the presentation of “behaviorally relevant environmental reinforcers” (pg. 3, Buckholtz et al., 2010b). Thus, in conjunction with the primary analyses, the overall pattern of neural activation in externalizers suggests widespread hyper-reactivity in regions related to the detection, processing, and regulation of motivational salience, particularly when the motivationally-significant stimuli match pre-existing priorities.

Conversely, in the Late Threat Focus condition, when participants were primed to focus on threat but an irrelevant distractor appeared first, deficits in allocating attention, executive control, and affective regulation were evident in high trait externalizing, as indicated by decreased amygdala and dmPFC activation. Previous studies suggest that reduction in dmPFC activation is prominent when threat processing occurs under cognitive load (e.g., due to conflict in the environment; Etkin et al., 2011). In the Late Threat Focus condition, reduced dmPFC activation may indicate that externalizing individuals are over-reacting to the presence of conflict (i.e., mismatch between goal and environmental stimuli). That is, because externalizing individuals prioritize affective stimuli, they must dedicate more resources to reorienting attention toward goal-directed information when distracting or irrelevant information is presented. Ultimately this capacity-demanding process results in a disruption in emotion responding (i.e., reflected in reduced amygdala and dmPFC activation; Blair, 2007; Corbetta, Patel & Shulman, 2008).

One issue deserving further consideration is the weak amygdala activation to threat versus safe stimuli in Early Threat Focus condition in the low externalizing group. It might be expected that when directly focusing on threat, amygdala activation in the low externalizing group would be greater to the threat than safe stimuli. However, two factors are important to consider in this case. First, some research reports decreased response to threat stimuli and postulate that this reflects the attentional demands associated with regulating affective responses (Delgado, Nearing, LeDoux & Phelps, 2008). The present task is verbally demanding and given the attentional manipulations, the demands on regulating attention and affect may have impacted the pattern of amygdala activation in the low externalizing group. Second, this pattern of amygdala activation may not be surprising given the nature of the sample. Virtually none of the previously published fMRI studies examining threat- and affect-related amygdala activation used an incarcerated control group (Birbaumer, Veit, Lotze, Erb, Hermann, Grodd. et al., 2005; Deeley, Daly, Surguladze, Tunstall, Mezey, Beer, et al., 2006; Muller, Sommer, Wagner, Lange, Taschler, Roder, et al., 2003) and fMRI studies using incarcerated samples fail to show a consistent pattern in terms of amygdala activation. Thus, it is unclear what the expected pattern of amygdala activity in incarcerated samples is, although this certainly warrants further study. Regardless, the effects within the high externalizing group are highly consistent across two studies and with a theoretical model that posits an anomalous attention response is driving emotion dysregulation in these individuals.

General Discussion

Across two studies using highly similar tasks but different prison samples, assessments of externalizing, and measures of emotional reactivity, we traced the disinhibited emotional responses associated with externalizing traits to the same dysfunctional cognitive-affective interaction. As expected, individuals with high levels of externalizing displayed exaggerated emotional reactivity, but this finding was highly specific and thus provided little support for the hypothesis that externalizing involves global hyper-reactivity to motivationally-significant stimuli (i.e., Hypothesis 1). Moreover, we found no support for the hypothesis that a deficit in executive functioning would interfere with the ability of externalizers to inhibit emotionally salient distractors while performing a task that required them to maintain a goal-directed focus on emotionally neutral stimuli (i.e., Hypothesis 2). In contrast to these more general hypotheses, the association between exaggerated emotion reactivity and externalizing traits was found to vary as a function of attentional focus and stimulus order. That is, consistent with Hypothesis 3, our results indicate that the over-allocation of attention to anticipated threat stimuli set up the dysregulation in executive functioning and emotional hyper-reactivity. This is not to deny that in certain contexts externalizers appear to have executive functioning deficits (e.g., Endres et al., 2011); but the results of the present study suggest that externalizing traits are associated with emotional hyper-reactivity largely due to their exaggerated attentional response and that this response may explain the apparent executive functioning deficits in other studies (i.e., over-allocation of attentional resources to specific stimuli would reduce the available resources for executive functions).

While it is premature to specify the exact attentional mechanism responsible for the attentional response in externalizing, it is interesting to speculate about the type of attentional processes underlying the cascade of events that culminate in emotion reactivity. In general, models of selective attention hold that information competes for limited processing resources, and particularly at later stages of attention, this competition is tightly coupled with demands on limited-capacity cognitive (i.e., executive functioning) resources (Desimone & Duncan, 1995). Emotion information appears to be privileged in this respect and typically takes-hold of perceptual and attentional resources (Pessoa, 2008). Moreover, individual differences, such as externalizing, may modulate responsivity to emotion information. Thus, in situations where motivationally-significant stimuli (e.g., threats or rewards) are particularly salient or intrinsically important to an individual, the emotion information commands processing resources and reduces the resources available for other functions (i.e., executive functions). As a result, the engagement of executive resources to control and inhibit the response to emotion information is hampered (see Wallace & Newman, 1997). Overall, the regulation of affective reactions will depend upon the application of resources associated with executive control/function, but the current findings suggest that for externalizers these downstream effects begin with an over-allocation of resources at the perceptual/attentional stage.

The present emphasis on attention as the crucial/controlling process underlying externalizing-related emotional reactivity fits well with externalizers' tendency to over-react in emotionally-charged situations. To illustrate the potential relevance of such cognitive-emotion interactions for behavior, we consider their implications for aggressive behaviors. Individuals high on externalizing are prone to display reactive aggression, which typically involves a reaction to frustration or threat (Blair, 2001). Models for reactive aggression assume roles for executive functions (i.e., cognitive control; Giancola et al., 1996) and emotion systems (Blair, 2001; Dougherty, Rauch, Deckersback, Marci et al., 2004). In essence, the combination of poor inhibitory control and an inability to process and manage emotions effectively, particularly frustration, results in the disinhibited expression of aggressive responses. One implication of the current findings is that externalizers' tendency to over-focus attention on motivationally-significant cues, such as a threat or an insult, may reduce their capacity for executive functioning and emotion regulation (i.e., not thinking before acting). Research suggests that some individuals with externalizing are hyper-sensitive to interpersonal slights and provocations (i.e., hostile attribution style; Dodge, 2006). Thus, if an individual with externalizing experiences a slight or insult in the context of an interpersonal interaction, they may over-allocate attention to that insult, have difficulty using executive functioning to regulate their reaction to the insult, and behave in a hyper-emotional way (i.e., display reactive aggression). Furthermore, this reaction may exist for any “trigger” that is over-prioritized for an externalizing individual (e.g., a substance-dependent externalizer may over-allocate attention to drug cues and display disinhibited reactions in response to these specific cues).

One limitation of the current studies is that our assessment of emotional reactivity was restricted to fear responses. Given the relationship between externalizing and hypersensitivity to rewards and drug cues, it is important to examine emotion dysregulation in the context of appetitive as well as aversive stimuli. However, in light of the more common emphasis on reward stimuli, the present results indicating dysregulated reaction in the presence of aversive stimuli suggest that the dysfunctional cognitive-affective interactions in externalizing may be more general than previously thought. Second, we used different, albeit correlated measures of externalizing in the two studies (see Footnote 3). Be that as it may, the fact that the two studies yielded consistent results speaks to the robustness of our findings across diverse externalizing measures.

In conclusion, cognition-emotion interactions involve diverse and interacting attentional, executive function, and affect-related processes. The current results suggest that a crucial cognition-emotion interaction affecting the disinhibited emotional reactivity associated with trait externalizing involves an over-prioritization and over-allocation of attention toward motivationally-significant cues. This attentional response, in turn, appears to preclude the effective utilization of executive functions (e.g., cognitive control, inhibition) and thus, ultimately culminates in the reactive disinhibition of emotion and behavior.

Highlights.

  • Studied cognition-emotion interplay in externalizing with startle and imaging.

  • Externalizing was not associated with global emotional hyperreactivity.

  • Externalizing was not associated with a primary deficit in executive functioning.

  • Externalizing was associated with an exaggerated attention response to threat.

  • Exaggerated attention responses undermine executive functions and emotion regulation.

Acknowledgments

This work was supported by grant MH053041 (Newman) and by Award Number T32-MH018931 (Baskin-Sommers) from the National Institutes of Mental Health. The imaging work was also supported by a grant from the University of Wisconsin-Madison/University of Wisconsin-Milwaukee IRIGP (Newman & Larson) and NIMH MH070539/NIDA DA026505 (Kiehl). We thank many at the Wisconsin Department of Corrections for making this research possible.

Footnotes

1

Analysis of the behavioral data revealed that there were no externalizing-related effects on behavioral performance (reaction time (F (3,246)=1.69, p=.17) or accuracy (F (3,246)=.77, p=.51)). Additionally, results for all of the primary analyses were essentially unaltered when measures of intelligence (Shipley WAIS-R estimate, Zachary, 1986) or working memory (score on digit backwards) were entered into the models.

2

Externalizing traits and psychopathy have many behavioral similarities and are often conflated in the literature. However, recent evidence suggests that the etiological underpinnings of these dimensions are different (Baskin-Sommers & Newman, 2011). In light of their symptom overlap and some overlap in participants between this and the Baskin-Sommers et al. (2011) study, it seemed important to compare and control for their respective cognitive-emotional interactions in this study. Scores on Hare's Psychopathy Checklist-Revised (2003; Hare, 2003) were entered into the statistical model as a covariate in order to control for the effect of psychopathy and estimate the unique effect of externalizing, as well as to compare the cognition-emotion interactions in psychopathy and externalizing, respectively. The unique effects of externalizing remained significant and were independent of the psychopathy-related FPS.

3

There are a variety of psychometric approaches for characterizing externalizing (e.g. using CON, Impulsive Antisociality, NA-CON difference score, ESI), and currently there is no agreed upon gold standard. There is existing evidence that the ESI and MPQ NA-CON measures tap the same underlying construct and moderately to strongly correlated (Venables & Patrick, 2011). In our dataset, using a different sample of participants who recently completed both measures, but did not complete either of the present studies, the correlation between the two measures was, r(326)=.60.

4

Due to software limitations we are not able to run a voxel-wise continuous regression-based reduction (like the analyses in Study 1). However, we recognize that some researchers caution against the use of a median split. Thus, we identified the amygdala and dmPFC clusters (region of interest pairwise approach), extracted them, and ran the simple effect analyses with externalizing as a continuous covariate. As with the analysis reported in the main text, there was a significant interaction between externalizing and time for the Threat Focus in the amygdala (i.e., Early Threat Focus versus Late Threat Focus, F(1,35)=10.70, p=.002, pη2=.23). The Early Threat Focus versus Late Threat Focus effect in the dmPFC was reduced to a trend, F(1,35)=2.93, p=.09, pη2=.08. There were no main effects of externalizing or effects of externalizing in the Alternative Focus conditions for either region of interest.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Anderson AK, Phelps EA. Lesions of the human amygdala impair enhanced perception of emotionally salient events. Nature. 2001;411:305–309. doi: 10.1038/35077083. [DOI] [PubMed] [Google Scholar]
  2. Ávila C, Parcet MA. Personality and inhibitory deficits in the stop-signal task: The mediating role of Gray's anxiety and impulsivity. Personality and Individual Differences. 2001;29:975–986. [Google Scholar]
  3. Baskin-Sommers AR, Curtin JJ, Newman JP. Specifying the attentional selection that moderates the fearlessness of psychopathic offenders. Psychological Science. 2011;22:226–234. doi: 10.1177/0956797610396227. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Baskin-Sommers AR, Wallace J, MacCoon D, Curtin JJ, Newman JP. Clarifying the factors that underlie Behavioral Inhibition System functioning in psychopathy. Personality Disorders: Theory, Research, and Treatment. 2010;1:203–217. doi: 10.1037/a0018950. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Birbaumer N, Veit R, Lotze M, Erb M, Hermann C, Grodd W, et al. Deficient fear conditioning in psychopathy: a functional magnetic resonance imaging study. Archives of General Psychiatry. 2005;62:799–805. doi: 10.1001/archpsyc.62.7.799. [DOI] [PubMed] [Google Scholar]
  6. Bishop SJ. Trait anxiety and impoverished prefrontal control of attention. Nature Neuroscience. 2009;12:92–8. doi: 10.1038/nn.2242. [DOI] [PubMed] [Google Scholar]
  7. Blair RJR. Neuro-cognitive models of aggression, the antisocial personality disorders and psychopathy. Journal of Neurology, Neurosurgery and Psychiatry. 2001;71:727–731. doi: 10.1136/jnnp.71.6.727. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Blair RJR. The amygdala and ventromedial prefrontal cortex: functional contributions and dysfunction in psychopathy. Philosophical Transactions of The Royal Society Biological Sciences. 2008;36:2557–2565. doi: 10.1098/rstb.2008.0027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Blumenthal TD, Cuthbert BN, Filion DL, Hackley S, et al. Committee report: Guidelines for human startle eyeblink electromyographic studies. Psychophysiology. 2005;42:1–15. doi: 10.1111/j.1469-8986.2005.00271.x. [DOI] [PubMed] [Google Scholar]
  10. Buckholtz JW, Treadway MT, Cowan R, Benning SD, Li R, et al. Mesolimbic dopamine reward system hypersensitivity in individuals with psychopathic traits. Nature Neuroscience. 2010a;13:419–21. doi: 10.1038/nn.2510. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Buckholtz JW, Treadway MT, Cowan R, Woodward ND, Li R, et al. Dopaminergic Network Differences in Human Impulsivity. Science. 2010b;329:532. doi: 10.1126/science.1185778. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Bufkin JL, Luttrell VR. Neuroimaging studies of aggressive and violent behavior: Current findings and implications for criminology and criminal justice. Trauma, Violence, and Abuse. 2005;6:176–191. doi: 10.1177/1524838005275089. [DOI] [PubMed] [Google Scholar]
  13. Corbetta M, Patel G, Shulman GL. The Reorienting System of the Human Brain: From Environment to Theory of Mind. Neuron. 2008;58:306–324. doi: 10.1016/j.neuron.2008.04.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Coccaro EF, et al. Amygdala and orbitofrontal reactivity to social threat in individuals with impulsive aggression. Biological Psychiatry. 2007;62:168–178. doi: 10.1016/j.biopsych.2006.08.024. [DOI] [PubMed] [Google Scholar]
  15. Cohen JD, Botvinick M, Carter CS. Anterior cingulate and prefrontal cortex: who's in control? Nature Neuroscience. 2000;3:421–423. doi: 10.1038/74783. [DOI] [PubMed] [Google Scholar]
  16. Cox RW. AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research. 1996;29:162–173. doi: 10.1006/cbmr.1996.0014. [DOI] [PubMed] [Google Scholar]
  17. Davis M, Whalen PJ. The amygdala: Vigilance and emotion. Molecular Psychiatry. 2001;6:13–34. doi: 10.1038/sj.mp.4000812. [DOI] [PubMed] [Google Scholar]
  18. Davidson RJ, Pizzagalli D, Nitschke JB, Kalin NH. Parsing the subcomponents of emotion and disorders of emotion: Perspectives from affective neuroscience. In: Davidson RJ, Scherer KR, Goldsmith HH, editors. Handbook of Affective Sciences. New York: Oxford University Press; 2003. pp. 8–24. [Google Scholar]
  19. Deeley Q, Daly E, Surguladze S, Tunstall N, Mezey G, Beer D, et al. Facial emotion processing in criminal psychopathy. Preliminary functional magnetic resonance imaging study. British Journal of Psychiatry. 2006;189:533–539. doi: 10.1192/bjp.bp.106.021410. [DOI] [PubMed] [Google Scholar]
  20. Delgado MR, Nearing KI, LeDoux JE, Phelps EA. Neural circuitry underlying the regulation of conditioned fear and its relation to extinction. Neuron. 2008;59:829–838. doi: 10.1016/j.neuron.2008.06.029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Desimone R, Duncan J. Neural mechanisms of selective attention. Annual Review of Neuroscience. 1995;18:193–222. doi: 10.1146/annurev.ne.18.030195.001205. [DOI] [PubMed] [Google Scholar]
  22. Derryberry D, Reed MA. Temperament and attention: Orienting toward and away from positive and negative signals. Journal of Personality and Social Psychology. 1194;66:1128–1139. doi: 10.1037//0022-3514.66.6.1128. [DOI] [PubMed] [Google Scholar]
  23. Dodge KA. Translational science in action: Hostile attributional style and the development of aggressive behavior problems. Developmental Psychopathology. 2006;18:791–814. doi: 10.1017/s0954579406060391. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Dolan SL, Bechara A, Nathan PE. Executive dysfunction as a risk marker for substance abuse: the role of impulsive personality traits. Behavioral Sciences and the Law. 2008;26:799–822. doi: 10.1002/bsl.845. [DOI] [PubMed] [Google Scholar]
  25. Dougherty DD, Rauch SL, Deckersback T, Marci C, et al. Ventromedial prefrontal cortex and amygdala dysfunction during anger induction positron emission tomography in patients with major depressive disorder with anger attacks. Archives of General Psychiatry. 2004;61:795–804. doi: 10.1001/archpsyc.61.8.795. [DOI] [PubMed] [Google Scholar]
  26. Eisenberg N, Guthrie IK, Fabes RA, Shepard S, Losoya S, Murphy B, et al. Prediction of elementary school children's externalizing problem behaviors from attentional and behavioral regulation and negative emotionality. Child Development. 2000;71:1367–1382. doi: 10.1111/1467-8624.00233. [DOI] [PubMed] [Google Scholar]
  27. Ellis HC, Ashbrook PW. Resource allocation model of the effects of depressed mood states on memory. In: Fiedler K, Forgas J, editors. Affect, cognition, and social behaviour. Toronto: Hogrefe; 1988. pp. 25–43. [Google Scholar]
  28. Endres MJ, Rickert ME, Bogg T, Lucas J, Finn PR. Externalizing psychopathology and behavioral disinhibition: Working memory mediates signal discriminability and reinforcement moderates response bias in approach-avoidance learning. Journal of AbnormalPsychology. 2011 doi: 10.1037/a0022501. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Etkin A, Egner T, Kalisch R. Emotional processing in anterior cingulate and medial prefrontal cortex. Trends in Cognitive Sciences. 2011;15:85–93. doi: 10.1016/j.tics.2010.11.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Finn PR, Mazas C, Justus A, Steinmetz JE. Early-onset alcoholism with conduct disorder: Go/No Go learning deficits, personality, and working memory capacity. Alcoholism: Clinical and Experimental Research. 2002;26:186–206. [PubMed] [Google Scholar]
  31. Fox E, Russo R, Dutton K. Attentional bias for threat: Evidence for delayed disengagement from emotional faces. Cognition & Emotion. 2002;16:355–379. doi: 10.1080/02699930143000527. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Giancola PR, Tarter RE. Executive Cognitive Functioning and Risk for Substance Abuse. Psychological Science. 1999;10:203–205. [Google Scholar]
  33. Giancola PR, Moss HB, Martin CS, Kirisci T, Tarter RE. Executive cognitive functioning predicts reactive aggression in boys at high risk for substance abuse: a prospective study. Alcoholism: Clinical & Experimental Research. 1996;20:740–4. doi: 10.1111/j.1530-0277.1996.tb01680.x. [DOI] [PubMed] [Google Scholar]
  34. Gorenstein EE, Newman JP. Disinhibitory psychopathology: A new perspective and a model for research. Psychological Review. 1980;87:301–315. [PubMed] [Google Scholar]
  35. Handley ED, Chassin L, Haller MM, Bountress KE, Dandreaux D, Beltran I. Do executive and reactive disinhibition mediate the effects of familial substance use disorders on adolescent externalizing outcomes? Journal of Abnormal Psychology. 2011;120:528–542. doi: 10.1037/a0024162. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Hanson J, Chung M, Avants B, Shritcliff E, Gee J, Davidson RJ, Pollak S. Early stress is associated with alterations in the orbitofrontal cortex: A tensor-based morphometry investigation of brain structure and behavioral risk. Journal of Neuroscience. 2010;30(22):7466–72. doi: 10.1523/JNEUROSCI.0859-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Hare RD. Hare Psychopathy Checklist-Revised (PCL-R) 2nd. Toronto: MHS; 2003. [Google Scholar]
  38. Herpertz SC, Huebner T, Marx I, Vloet TD, Fink GR, Stoecker T, et al. Emotional processing in male adolescents with childhood-onset conduct disorder. Journal of Child Psychology and Psychiatry. 2008;49:781–791. doi: 10.1111/j.1469-7610.2008.01905.x. [DOI] [PubMed] [Google Scholar]
  39. Iacono WG, Malone SM, McGue M. Behavioral disinhibition and the development of Early-Onset Addiction: Common and specific Influences. Annual Review of Clinical Psychology. 2008;4:325–48. doi: 10.1146/annurev.clinpsy.4.022007.141157. [DOI] [PubMed] [Google Scholar]
  40. Keil A, Bradley MM, Junghöfer M, Russmann T, Lowenthal W, Lang PJ. Cross-modal attentional capture by affective stimuli. Cognitive, Affective, and Behavioral Neuroscience. 2007;7:18–24. doi: 10.3758/cabn.7.1.18. [DOI] [PubMed] [Google Scholar]
  41. Krueger RF, Caspi A, Moffitt TE. Epidemiological personology: The unifying role of personality in population-based research on problem behaviours. Journal of Personality. 2000;68:967–998. doi: 10.1111/1467-6494.00123. [DOI] [PubMed] [Google Scholar]
  42. Krueger RF, Markon PKE, Patrick CJ, Benning SD, Kramer MD. Linking antisocial behavior, substance use, and personality: An integrative quantitative model of the adult externalizing spectrum. Journal of Abnormal Psychology. 2007;116:645–666. doi: 10.1037/0021-843X.116.4.645. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Krueger RF, Markon PKE, Patrick CJ, Iacono WG. Externalizing psychopathology in adulthood: A dimensional-spectrum conceptualization and its implications for DSM–V. Journal of Abnormal Psychology. 2005;114:537–550. doi: 10.1037/0021-843X.114.4.537. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Lejuez CW, Magidson JF, Mitchell SH, Sinha, Stevens MC, de Wit H. Behavioral and biological indicators of impulsivity in the development of alcohol use, problems, and disorders. Alcoholism: Clinical and Experimental Research. 2010;34:1334–1345. doi: 10.1111/j.1530-0277.2010.01217.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Lindquist KA, Wager TD, Kober H, Mliss-Moreau E, Barrett LF. The Brain Basis of Emotion: A meta-analytic review. Behavioral and Brain Sciences. doi: 10.1017/S0140525X11000446. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. MacCoon DG, Wallace JF, Newman JP. Self-regulation: the context-appropriate allocation of attentional capacity to dominant and non-dominant cues. In: Baumeister RF, Vohs KD, editors. Handbook of Self-Regulation: Research, Theory, and Applications. New York, NY: Guilford; 2004. pp. 422–446. [Google Scholar]
  47. Martin LE, Potts GF. Reward sensitivity in impulsivity. Neuroreport. 2004;15:1519–1522. doi: 10.1097/01.wnr.0000132920.12990.b9. [DOI] [PubMed] [Google Scholar]
  48. Miller MW, Greif JL, Smith AA. Multidimensional Personality Questionnaire profiles of veterans with traumatic combat exposure: Internalizing and externalizing subtypes. Psychological Assessment. 2003;15:205–215. doi: 10.1037/1040-3590.15.2.205. [DOI] [PubMed] [Google Scholar]
  49. Morgan AB, Lilienfeld SO. A meta-analytic review of the relation between antisocial behavior and neuropsychological measures of executive function. Clinical Psychology Review. 2000;20:113–156. doi: 10.1016/s0272-7358(98)00096-8. [DOI] [PubMed] [Google Scholar]
  50. Morgan MA, Romanski LM, LeDoux JE. Extinction of emotional learning: Contribution of medial prefrontal cortex. Neuroscience Letters. 1993;163:109–113. doi: 10.1016/0304-3940(93)90241-c. [DOI] [PubMed] [Google Scholar]
  51. Muller JL, Sommer M, Wagner V, Lange K, Taschler H, Roder CH, et al. Abnormalities in emotion processing within cortical and subcortical regions in criminal psychopaths: evidence from a functional magnetic resonance imaging study using pictures with emotional content. Biological Psychiatry. 2003;54:152–162. doi: 10.1016/s0006-3223(02)01749-3. [DOI] [PubMed] [Google Scholar]
  52. Newman JP. Reaction to punishment in extraverts and psychopaths: Implications for the impulsive behavior of disinhibited individuals. Journal of Research in Personality. 1987;21:464–485. [Google Scholar]
  53. Newman JP. Conceptual models of the nervous system: Implications for antisocial behavior. In: Stoff DM, Breiling J, Maser JD, editors. Handbook of Antisocial Behavior. New York, NU: John Wiley & Sons, Inc.; 1997. pp. 324–335. [Google Scholar]
  54. Newman JP, Lorenz AR. Response modulation and emotion processing: Implications for psychopathy and other dysregulatory psychopathology. In: Davidson RJ, Scherer K, Goldsmith HH, editors. Handbook of Affective Sciences. Oxford, England: Oxford University Press; 2003. pp. 904–929. [Google Scholar]
  55. Newman JP, Patterson CM, Howland EW, Nichols SL. Passive avoidance in psychopaths: The effects of reward. Personality and Individual Differences. 1990;11:1101–1114. [Google Scholar]
  56. Newman JP, Widom CS, Nathan S. Passive-avoidance in syndromes of disinhibition: Psychopathy and extraversion. Journal of Personality and Social Psychology. 1985;48:1316–1327. doi: 10.1037//0022-3514.48.5.1316. [DOI] [PubMed] [Google Scholar]
  57. Nichols S, Newman JP. Effects of punishment on response latency in extraverts. Journal of Personality and Social Psychology. 1986;50:624–630. doi: 10.1037//0022-3514.50.3.624. [DOI] [PubMed] [Google Scholar]
  58. Patrick CJ, Curtin JJ, Tellegen A. Development and validation of a brief form of the Multidimensional Personality Questionnaire. Psychological Assessment. 2002;14:150–163. doi: 10.1037//1040-3590.14.2.150. [DOI] [PubMed] [Google Scholar]
  59. Patrick CJ, Zempolich KA. Emotion and aggression in the psychopathic personality. Aggression and Violent Behavior. 1998;3:303–338. [Google Scholar]
  60. Patterson CM, Kosson DS, Newman JP. Reaction to punishment, reflectivity, and passive avoidance learning in extraverts. Journal of Personality and Social Psychology. 1987;52:565–576. doi: 10.1037//0022-3514.52.3.565. [DOI] [PubMed] [Google Scholar]
  61. Pessoa L. How do emotion and motivation direct executive control? Trends in Cognitive Sciences. 2009;13:160–6. doi: 10.1016/j.tics.2009.01.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Pessoa L. To what extent are emotional visual stimuli processed without attention and awareness? Current Opinion in Neurobiology. 2005;15:188–196. doi: 10.1016/j.conb.2005.03.002. [DOI] [PubMed] [Google Scholar]
  63. Phan KL, Taylor SF, Welsh RC, Decker LR, Noll DC, et al. Activation of the medial prefrontal cortex and extended amygdala by individual ratings of emotional arousal: A fMRI study. Biological Psychiatry. 2003;53:211–215. doi: 10.1016/s0006-3223(02)01485-3. [DOI] [PubMed] [Google Scholar]
  64. Raine A, Lencz T, Bihrle S, et al. Reduced prefrontal gray matter volume and reduced autonomic activity in antisocial personality disorder. Archives of General Psychiatry. 2000;57:119–127. doi: 10.1001/archpsyc.57.2.119. [DOI] [PubMed] [Google Scholar]
  65. Raine A, Meloy JR, Bihrle S, Stoddard J, LaCasse L, Buchsbaum MS. Reduced prefrontal and increased subcortical brain functioning assessed using positron emission tomography in predatory and affective murderers. Behavioral Sciences & the Law. 1998;16:319–332. doi: 10.1002/(sici)1099-0798(199822)16:3<319::aid-bsl311>3.0.co;2-g. [DOI] [PubMed] [Google Scholar]
  66. Singh-Manoux A, Kivimaki M, Glymour MM, Elbaz A, Berr C, Ebmeier KP, Ferrie JE, Dugravot A. Timing of onset of cognitive decline: results from Whitehall II prospective cohort study. British Medical Journal. 2012;344:d7622. doi: 10.1136/bmj.d7622. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Sterzer P, Stadler C, Krebs A, et al. Abnormal neural responses to emotional visual stimuli in adolescents with conduct disorder. Biological Psychiatry. 2005;57:7–15. doi: 10.1016/j.biopsych.2004.10.008. [DOI] [PubMed] [Google Scholar]
  68. Taylor J, Carlson SR, Iacono WG, Lykken DT, McGue M. Individual differences in electrodermal responsivity to predictable aversive stimuli and substance dependence. Psychophysiology. 1999;36:193–198. [PubMed] [Google Scholar]
  69. Venables NC, Patrick CJ. Validity of the externalizing spectrum inventory in a criminal offender sample: Relations with disinhibitory psychopathology, personality, and psychopathic features. Psychological Assessment. 2011 doi: 10.1037/a0024703. Advance online publication. [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Verona, Patrick CJ, Lang AR. A direct assessment of the role of state and trait negative emotion in aggressive behavior. Journal of Abnormal Psychology. 2002;111:249–258. doi: 10.1037//0021-843x.111.2.249. [DOI] [PubMed] [Google Scholar]
  71. Volkow ND, Li TK. Drug addiction: the neurobiology of behaviour gone awry. Nature Reviews Neuroscience. 2004;5:963–970. doi: 10.1038/nrn1539. [DOI] [PubMed] [Google Scholar]
  72. Wallace JF, Newman JP. Differential effects of reward and punishment cues on response speed in anxious and impulsive individuals. Personality and Individual Differences. 1990;11:999–1009. [Google Scholar]
  73. Wallace JF, Newman JP. Neuroticism and the attentional mediation of dysregulatory psychopathology. Cognitive Therapy and Research. 1997;21:135–156. [Google Scholar]
  74. Zachary RA. Shipley institute of living scale: Revised manual. Los Angeles: Western Psychological Service; 1986. [Google Scholar]
  75. Zuckerman M. Sensation seeking and psychopathy. In: Hare RD, Schalling D, editors. Psychopathic behavior: Approaches to research. New York: Wiley; 1978. pp. 165–185. [Google Scholar]

RESOURCES