Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Feb 1.
Published in final edited form as: Brain Imaging Behav. 2017 Feb;11(1):76–85. doi: 10.1007/s11682-015-9500-7

Neural Correlates of Emotion Reappraisal in Individuals with Externalizing Psychopathology

Allison J Lake a, Peter R Finn a, Thomas W James a,b,c
PMCID: PMC4959999  NIHMSID: NIHMS754737  PMID: 26809287

Abstract

Externalizing psychopathology (EXT) is characterized by regulatory deficits of behavior, cognition, and negative emotion. Previous research on EXT suggests that cognitive and affective dysregulation are highly related, such that strong affective states constrain a reduced-capacity cognitive system. Reappraisal is an effective emotional control strategy involving complex interactions between cognitive and affective brain functions and may therefore offer novel insight into the specific neural mechanisms of affective dysregulation among individuals with EXT. To evaluate these possibilities, we tested individuals with low or high EXT in a reappraisal paradigm. Neuroimaging results indicated that EXT was associated with hypo-activation in the amygdala and superior parietal lobule during both maintenance and reappraisal as well as poor modulation of the lateral occipital cortex during negative emotion reappraisal. These results suggest a general disruption of perceptual-attentional resource allocation such that high EXT individuals are characterized by poor modulation of perceptual-attentional resources during reappraisal. Subsequently, emotion reappraisal may be a useful but not adequate tool to control negative affect in EXT.

Keywords: emotion regulation, reappraisal, externalizing psychopathology


Externalizing psychopathology (EXT) reflects the covariance of multiple psychopathology domains including conduct, antisocial personality, and substance use disorders (Krueger et al., 2002; Krueger & Markon, 2006). These disorders are highly comorbid and reflect self-regulatory deficits characterized by high degrees of behavioral disinhibition, impulsivity, and poor-decision making (Bobova et al., 2009; Cantrell, Finn, Rickert, & Lucas, 2008; Finn, 2002; Finn et al., 2009; Gorenstein & Newman, 1980; Iacono, Malone, & McGue, 2008; Newman & Lorenz, 2003; Patterson & Newman, 1993), such that high EXT individuals often act impulsively without regard for consequences. Importantly, these deficits have been widely associated with limited cognitive capacity; specifically, reduced executive working memory capacity has been related to disadvantageous decision-making among individuals with EXT. (Bechara & Martin, 2004; Endres et al., 2011; Endres, Donkin, & Finn, 2014; Finn, Gunn, & Gerst, 2015; Morgan & Lilienfeld, 2000). In addition to these behavioral and cognitive problems, EXT has also been associated with affective dysregulation, such that these individuals are likely to also experience depression, anxiety, and other internalizing disorders as well as increased impulsive behavior in strong affective states (Baskin-Sommers, Wolf, Buckholtz, Warren, & Newman, 2012b; Cyders & Smith, 2007; Cyders & Smith, 2008; Krueger & Markon, 2006).

Previous research suggests that the cognitive and affective deficits associated with EXT are highly inter-related to the extent that affective reactivity is likely to constrain already reduced cognitive resources. Specifically, while affective or motivationally-relevant information typically receive priority of processing resources in healthy individuals (Ehring et al., 2010; Lim et al., 2009; Morris et al., 1998; Pessoa, 2009), individuals with EXT demonstrate over-allocation of these processing resources, resulting in an inability to adequately complete other resource-demanding tasks such as those involving cognitive control or emotion regulation (Baskin-Sommers, et al., 2012a; Baskin-Sommers et al., 2012b; Buckholtz et al., 2010; MacCoon, Wallace, & Newman, 2004; Martin & Potts, 2004). While these studies provide an integrative framework for understanding the association between the cognitive deficits observed in EXT and affective dysregulation, relatively little is known about affective dysregulation specifically among individuals with EXT. In particular, while previous research demonstrates that high levels of affective reactivity among individuals with EXT further limit reduced-capacity cognitive systems, little research has investigated the specific neural mechanisms of affective reactivity and dysregulation among individuals with EXT.

Emotion reappraisal, a cognitive emotion regulation technique, may provide unique insights into affective breakdowns associated with EXT to the extent that it has been shown to involve complex interactions between cognitive and affective processes (Ochsner & Gross, 2005; Wager, Davidson, Hughes, Lindquist, & Ochsner, 2008). Specifically, reappraisal effectively reduces strong negative emotions though reconceptualizing an emotional event to one that is emotionally neutral. For example, an individual may use reappraisal to think of a poor performance review as an opportunity to improve one's abilities, rather than indicating failure. Neural evidence supports the involvement of both cognitive and affective processes in reappraisal. Specifically, reappraisal is associated with activation in regions associated with cognitive processes including medial and lateral areas of the prefrontal cortex (PFC) thought to be involved in behavior inhibition, including the dorsolateral PFC (dlPFC) and ventromedial PFC (vmPFC). In turn, reappraisal is associated with the modulation of affect-related regions including the insula and amygdala (Gross, 2002; Koenigsberg et al., 2009; Ochsner, Bunge, Gross, & Gabrieli, 2002; Ochsner & Gross, 2008; Ochsner et al., 2009), as well as visual regions including the lateral occipital cortex (LOC; Koenigsberg et al., 2009; Ochsner et al., 2002), perhaps reflecting the strong role of perceptual processing in emotional responding (Critchley et al., 2005; Lim, Padmala, & Pessoa, 2009; Morris et al., 1998; Sabatinelli, Bradley, Lang, Costa, & Versace, 2007).

Importantly, difficulties with reappraisal have been demonstrated as a broad marker of emotion-related psychopathology (Ehring et al., 2010; Garnefski, Kraaij, & Etten, 2005; Johnstone, van Reekum, Urry, Kalin, & Davidson, 2007; Moses & Barlow, 2006) to the extent that deficient reappraisal may reflect problems in cognitive control, affective modulation, or both. While EXT-related deficits in cognition and emotion map onto the processes involved in reappraisal, the specific breakdowns in reappraisal among individuals with EXT remain unclear. To clarify the roles of the brain regions previously implicated in reappraisal on emotion regulation failures among EXT individuals, we tested low and high EXT individuals on an fMRI emotion reappraisal task. To the extent that reappraisal is a cognitively demanding task and EXT individuals are characterized by deficits in executive cognitive capacity, emotion regulatory failures may be associated with a failure to recruit adequate cognitive resources to control emotion, independent of the tendency or frequency with which reappraisal is utilized. Alternatively, difficulty with emotion reappraisal may be due to poor modulation of affective systems despite equivalent recruitment of cognitive control resources. Thus, we hypothesized that difficulty with emotion regulation among high EXT individuals would alternately be associated with under-activation of dlPFC and vmPFC, or over-activation of the amygdala and insula.

Method

Participants

Participants (n = 40, 19 women) were recruited from a larger study examining personality and cognitive factors in EXT. Participants were primarily college-aged (M = 21.15, SD = 2.26), right-handed (n = 38), and Caucasian (82.9%), with the remaining participants endorsing African American (4.9%), Asian (4.9%), and mixed ethnicities (4.9%).

Group inclusion criteria

The design included two equal groups (each n = 20) of low and high EXT individuals. To the extent that EXT is associated with varying levels of problems across a number of EXT diagnostic domains, rather than specific diagnoses, factor scores were calculated for both EXT and trait negative affectivity (NA) within a sample of 747 participants in the larger study. EXT factor scores were based on a single EXT factor using Blom transformed indicators of lifetime problems related to childhood conduct, adult antisociality, alcohol, marijuana, and other drugs (see Finn et al., 2009 for an example of the factor composition of the EXT factor) as assessed by the Semi-Structured Assessment for the Genetics of Alcoholism (SSAGA; Bucholz et al., 1994). In the larger sample, the EXT factor fit the data well, eigenvalue = .69. Low EXT participants fell within the lower tertile of the EXT factor, while high EXT participants fell within the upper tertile. That is, the low EXT group, acting here as controls, reflects healthy levels of functioning in this population with few problems in these domains, while the high EXT group reflects significant psychopathology with substantial problems across these EXT-related domains. In order to most parsimoniously establish neural differences during emotion reappraisal among EXT individuals, this tertile approach was favored over a more data-intensive dimensional approach.

Because EXT is associated with high levels NA, the low and high EXT groups were equated on NA to control for variance in this domain. This approach allowed examination of differences in emotion regulation associated with EXT without confounding levels of NA. NA factor scores were based on a single NA factor indicated by self-report, non-diagnostic measures of trait anxiety (State-Trait Anxiety Inventory-Trait Scale; Spielberger, Gorsuch, & Lushene, 1970), depression symptoms (Beck Depression Inventory-II; Beck, Steer, & Brown, 1996), and neuroticism (Eysenck Personality Questionnaire-Neuroticism Scale; Eysenck & Eysenck, 1975; see Table 1). The NA factor also fit the data well, eigenvalue = .78. Low and high EXT participants did not differ on NA, t(35) = .78, p = .44. Eligible participants were contacted by phone and screened to rule out severe head trauma, history of psychosis, or other conditions that would contraindicate fMRI testing.

Table 1.

Lifetime Problems Related to Externalizing Psychopathology (n = 37).

Low EXT High EXT

Mean Lifetime Problems (SD)
 Childhood conduct 1.89 (2.31) 13.17 (3.91)
 Adult antisocial .89 (.66) 10.89 (2.97)
 Alcohol 4.63 (5.16) 20.72 (9.36)
 Marijuana .32 (.95) 12.17 (9.92)
 Other drug .00 (.00) 11.83 (20.43)
 EXT Factor Score -1.14 (.33) .77 (.31)
EXT-Related Diagnoses, %
 Childhood conduct 0.00 55.60
 Adult antisocial 0.00 33.30
 Alcohol abuse 26.30 27.80
 Alcohol dependence 10.50 72.20
 Marijuana abuse 5.30 22.20
 Marijuana dependence 5.30 55.60
 Other drug abuse 0.00 0.00
 Other drug dependence 0.00 27.80
Mean Severity of Negative Affect (SD)
 STAI-T 39.05 (11.32) 40.39 (10.11)
 BDI 6.05 (5.51) 8.78 (4.94)
 EPQ-N 8.05 (6.75) 9.28 (5.65)
 NA Factor Score -.09 (1.16) .17 (.92)

Note. Diagnoses based on SSAGA (Bucholz et al., 1994) interview responses using DSM-IV-TR criteria. STAI-T = State-Trait Anxiety Inventory-Trait Scale (Spielberger et al., 1970); BDI = Beck Depression Inventory (Beck et al., 1996); EPQ-N = Eysenck Personality Questionnaire-Neuroticism Scale (Eysenck & Eysenck, 1975); EXT = externalizing psychopathology; NA = negative affect.

Procedure

Prior to testing, participants were required to abstain from recreational drug and alcohol use for at least 12 hours, get at least 6 hours of sleep the night before, and eat a meal within 3 hours of testing. Informed consent was obtained from all individual participants included in the study. Following informed consent, participants practiced the task outside of the scanner with feedback from the experimenter. Broadly, participants viewed affective images (aversive and neutral) while either reappraising or maintaining their natural reaction (Koenigsberg et al., 2009). Images were selected from the International Affective Picture System (IAPS; Lang, Bradley, & Cuthbert, 2008). Neutral images included people engaged in everyday activities and normal interactions; aversive images included scenes of interpersonal violence and injuries. Neutral images used in the reappraisal condition did not differ on valence ratings (M = 5.29, SD = 0.54) from those used in the maintenance condition (M = 5.15, SD = 0.51), t(46) = 0.91, p = .37; reappraisal (M = 3.62, SD = 0.70) and maintenance neutral images (M = 3.54, SD = 0.52) did not differ on arousal ratings, t(46) = 0.44, p = .66. Similarly, aversive images used in the reappraisal condition did not differ on valence ratings (M = 2.19, SD = 0.42) from those used in the maintenance condition (M = 2.32, SD = 0.56), t(46) = 0.90, p = .37; reappraisal (M = 6.21, SD = 0.67) and maintenance aversive images (M = 5.87, SD = 0.70) did not differ on arousal ratings, t(46) = 1.73, p = .09. All stimuli depicted at least one person1.

Participants were given instruction on the emotion regulation task. During maintenance, participants were instructed to maintain their natural reaction to the image without any attempts to regulate or control their response. During reappraisal, participants were instructed to reinterpret the image in a neutral way. For example, an image of interpersonal violence could be reappraised as a scene between two actors in rehearsal. The participant practiced these tasks with images similar to those used in the scanner task with experimenter feedback until the experimenter felt the participant understood the instructions.

Following practice, participants lay comfortably in the scanner with their head secured in the head coil with foam padding. Stimuli were projected onto a rear-projection screen in the scanner bore behind the participant, and viewed through a mirror attached above the head coil. On each trial, participants received a 2 s audio instruction over headphones; a “maintain” prompt indicated participants should use the maintain technique, while a “suppress” prompt indicated participants should use the reappraisal technique. An aversive or neutral IAPS image was then presented for 10 s. Participants then viewed a response screen for 4 s that displayed a Likert-scale ranging from 1 (very negative) to 5 (very positive), and then indicated via button-press their emotional reaction to the image using their right hand. While a Likert scale of this range may limit variability in responses, this procedure was in line with previous research (i.e. Koenigsberg et al., 2009) and adapted well to the scanner environment. Participants then viewed a “RELAX” screen for 4s before hearing the next audio prompt (Fig 1). Each participant completed 24 trials of each trial type (i.e. aversive-maintain [AM], aversive-reappraise [AR], neutral-maintain [NM], and neutral-reappraise [NR]) for a total of 96 trials. Trial types were pseudorandomized across the 96 trials in an event-related design. Trials were separated into four runs of 24 trials with a short break between runs. Participants received approximately $50 for completing the study, which lasted approximately two hours.

Fig 1.

Fig 1

Schematic of emotion reappraisal task

Image Acquisition

Imaging data were acquired with a Siemens Magnetom TIM TRIO 3-Tesla whole-body MRI using a 32-channel phased-array head coil. The field of view was 220 × 220 mm, with an in-plane resolution of 64 × 64 pixels and 35 axial slices of 3.8 mm thickness per volume, producing voxels that were 3.4 × 3.4 × 3.8 mm. Functional images were collected using a gradient echo EPI sequence: TE = 25 ms, TR = 2,000 ms, flip angle = 70°. Parallel imaging was not used. High-resolution T1-weighted anatomical volumes were acquired using a Turbo-flash 3-D sequence: TI = 1800 ms, TE = 2.67 ms, TR = 1800 ms, flip angle = 9°, with 160 sagittal slices of 1 mm thickness, a field of view of 256 × 256 mm, and an isometric voxel size of 1 mm3.

Data Analysis Plan

We conducted a whole-brain analysis with all of the between-subjects (high vs. low EXT) and within-subjects factors (AM, AR, NM, NR) included (Friston, Rotshtein, Geng, Sterzer, & Henson, 2006). These analyses first focused on contrasts across all participants, including a contrast to reveal brain regions activated by reappraisal (i.e. AR + NR > AM + NM) and a contrast to reveal brain regions “de-activated” by reappraisal (i.e. AM + NM > AR + NR). Second, to determine the differential activation in these processes between the two EXT groups, between-group differences in reappraisal of aversive images (i.e. AR > AM) were examined. Third, whole-brain simple main effects were examined to clarify any interactions between group and reappraisal condition. Fourth, to establish the patterns of activation associated with emotion reactivity, contrasts across all participants during aversive image maintenance (i.e. AM > NM) were performed. Fifth, to assess group differences in emotion reactivity, between-group differences in aversive image maintenance were observed. Sixth, whole-brain simple main effects were conducted to clarify interactions between group and image valence. Finally, betas from significant clusters were extracted to provide a graphical (rather than brain map) presentation of the activation profile across experimental conditions.

Data Processing

In order to minimize noise in the data, data quality was assessed in individual participants by inspecting estimated motion parameters. Participants who showed estimated motion spikes greater than 3mm and BOLD signal change greater than two standard deviations from the group mean across multiple activation clusters for both AR > AM and AM > NM contrasts, were excluded from further analyses. Based on these criteria, three participants were removed (2 high EXT, 1 low EXT) leaving 37 participants in the completed analyses.

Imaging analyses were conducted with FMRIB Software Library (FSL; http://www.fmrib.ox.ac.uk) using the FMRI Expert Analysis Tool (FEAT) tools version 5.98. Individual functional volumes were co-registered to the MNI 152 standard brain, normalized to the Montreal Neurological Institute (MNI) atlas. For each functional run for each subject, first-level analyses were performed with a General Linear Model (GLM) with explanatory variables (EVs) for each of the four task conditions constructed using boxcar functions based on the timing of the experimental protocol and convolved with a double-gamma hemodynamic response function. Second-level analyses collapsed per-run inputs from the first level into per-subject outputs as beta coefficients. Third-level analyses combined subject inputs (beta coefficients) into groups using a random-effects model and produced between-groups contrasts across dependent variables from all first-level contrasts. Maps from third-level analyses were thresholded at the cluster level with a permutation correction for multiple tests (Nichols, 2012). The second-level betas (first-level dependent variables for each subject for each voxel) were randomly permuted across voxels 1000 times to build cluster-size distributions for the voxelwise threshold of t = 2.45 (p < .01). This voxel-level criterion was chosen based on our experience with previous studies with similar effect sizes. In those studies, this level of strictness produced a good balance between type I and type II errors. For AR > AM and AM > NM contrasts, the minimum cluster size required for a type I error rate of 5% was five voxels (40 1×1×1 mm3 voxels). These thresholds (voxelwise t = 2.45, clustersize N = 5) were applied to all maps from third-level analyses. To visualize the results, 3D surface maps were created with thresholded images using the Surface Mapper (SUMA) program within Analysis of Functional Neuroimages (AFNI; Cox, 1996).

Results

Behavioral Ratings

Participants' self-report following the task indicated that they used the emotion maintenance and reappraisal techniques effectively. Specifically, there was a main effect of valence on image rating, with higher ratings (more positive reactions) following neutral (M = 3.18, SD = .15) vs. aversive images (M = 1.98, SD = .25), F(1, 33) = 605.73, p < .001. Additionally, there was a valence × technique interaction, F(1, 33) = 108.88, p < .001. For aversive pictures, ratings were higher (i.e. more neutral) following reappraise (M = 2.28, SD = .31) vs. maintain (M = 1.67, SD = .30) instructions, t(36) = 10.46, p < .001, indicating success in reducing negative reactions during reappraisal. Conversely, for neutral pictures, ratings were lower, indicating more neutral emotional reactions and successful reappraisal, following reappraise (M = 3.07, SD = .11) vs. maintain instructions (M = 3.29, SD = .25), t(36) = 5.31, p < .001. There was no significant main effect of EXT or interactions between EXT and valence or instruction on ratings (Fig 2).

Fig 2.

Fig 2

Emotional reaction ratings across four task conditions by group. Higher ratings indicate more positive evaluations. Error bars represent 95% confidence intervals

Imaging Results

Affective Reappraisal

Whole-brain analyses of neural regions activated by reappraisal in all participants largely replicated previous findings in similar reappraisal paradigms. Specifically, reappraisal was associated with increased BOLD signal in the bilateral dorsolateral prefrontal cortex (dlPFC), dorsal and rostral anterior cingulate cortex (ACC), insula, and angular gyri (Fig 3a). In turn, reappraisal modulated activity in the bilateral amygdalae, fusiform, orbitofrontal cortex (OFC), LOC, and left caudate (Fig 3b). These results are in support of previous findings that reappraisal draws largely on prefrontal areas associated with cognitive control to down-regulate subcortical areas associated with affective and perceptual processing (Online Resource 1).

Fig 3.

Fig 3

(a.) Lateral and medial views of regions activated during emotion reappraisal and (b.) regions modulated by reappraisal among all participants. (c.) Lateral and ventral views of regions activated by the interaction between task (aversive-reappraise > aversive-maintain) and group and (d.) lateral and medial views of regions deactivated by the interaction between task and group

Group differences associated specifically with the reappraisal of aversive images (i.e. AR > AM) indicated increased relative activation in the left LOC, inferior division, right LOC superior and inferior division, and left OFC during aversive image reappraisal in the high EXT group as compared to the low EXT group (Fig 3c and Online Resource 2). Additionally, the high EXT group demonstrated decreased relative activation in the left precuneus, left precentral gyrus, and right inferior prefrontal gyrus (IFG), pars triangularis and pars opercularis, bilateral OFC, right dlPFC, and right parahippocampal gyrus relative to the low EXT group (Fig 3d).

Simple main effect contrasts were conducted to better characterize the pattern of activation across conditions in the regions that showed significant interaction effects. First, to clarify the increases in BOLD signal associated with the reappraisal of negative images, the AR > AM contrast was examined in each group separately. In the high EXT group, activation was observed in the bilateral angular gyrus, superior precuneus, and dlPFC, as well as the right insula, and the superior division of the left LOC. In the low EXT group, robust activation was observed in areas that overlapped with those observed in the high EXT group. The low EXT group, however, displayed unique activation within the area of the precuneus that showed increased activation in the interaction contrast. Second, to clarify the increased activation of the left LOC, the AM > AR contrast (i.e., “deactivation” associated with reappraisal of negative images) was also examined in each group separately. In the high EXT group, deactivation was observed in the bilateral amygdala, bilateral insula, inferior temporal gyrus, and bilateral supramarginal gyrus. In the low EXT group, deactivation was observed in an area of the left LOC that overlapped with the activation from the interaction contrast (Fig 4a and 4b). Deactivation was also observed in the bilateral occipital poles, supramarginal gyrus, and insula.

Fig 4.

Fig 4

(a.) LOC activation from task (aversive-reappraise > aversive maintain) × group interaction (i.e. high EXT > low EXT). (b). Activation in the left LOC across aversive-maintain (AM) and aversive-reappraise (AR) conditions by group. (c.) Simple main effect showing greater activation in the right amygdala for the low EXT group during AM relative to the high EXT group. (d.) Activation in the right amygdala across AM, AR, neutral-maintain (NM), and neutral-reappraise (NR) conditions by group

For the sake of completeness, differences between the low and high EXT groups were examined in the AM and AR conditions. In the AM condition, increased activation was observed for the low EXT group relative to the high EXT group in the right fusiform, amygdala, IFG, and anterior LOC, as well as the bilateral superior parietal lobule (SPL; Fig 4c and 4d). Decreased activation in the low EXT group relative to the high EXT group was observed in the right OFC, bilateral IFG, and left medial prefrontal cortex (mPFC). Group differences in the AR condition revealed increased activation in the low EXT group in the bilateral amygdalae, SPL, superior temporal gyrus, and middle frontal gyrus, as well as the right fusiform and middle frontal gyrus. Decreased relative activation in the low EXT group was observed in the left IFG, bilateral OFC, and right middle temporal gyrus.

Affective Reactivity

Whole-brain analyses of neural regions activated by affective reactivity (i.e. AM > NM) in all largely replicated previous findings. Specifically, activation was observed in the bilateral amygdalae, thalamus, hippocampi, frontal operculum, supramarginal gyri, dorsal ACC, and posterior visual regions during aversive image reactivity (Fig 5a). De-activation was observed in the bilateral posterior insula, frontal poles, superior temporal gyri, precentral gyri, and parahippocampal gyri (Fig 5b).

Fig 5.

Fig 5

(a.) Lateral and medial views of regions activated during aversive image maintenance and (b.) regions modulated by aversive image maintenance among all participants. (c.) Lateral and medial views of regions activated and (d.) deactivated by the interaction between task (aversive-maintain > neutral maintain) and group

Group differences associated with reactivity to aversive images indicated increased activation in the high EXT group relative to the low EXT group in the bilateral precentral gyrus and precuneus, right supramarginal gyrus, right pars opercularis, right central operculum, left frontal operclum, and left posterior cingulate (Fig 5c). In contrast, increased activation in the low EXT group as compared to the high EXT group was observed in the bilateral SPL, LOC, right fusiform, and right temporal pole (Fig 5d).

To better clarify these interactions, simple main effects contrasts were conducted in each group separately. First, in the high EXT group, regions activated by aversive image maintenance (i.e. AM > NM) robustly replicated those regions observed across all participants reported above, including the bilateral fusiform, LOC, hippocampus, amygdala, thalamus, SPL, precentral gyrus, and right dlPFC. In contrast, “deactivation” associated with aversive image maintenance (i.e. NM > AM) in the high EXT group was observed in the right planum polare, left frontal pole, bilateral parahippocampal gyri, and superior temporal gyrus. In the low EXT group, regions activated by aversive image maintenance were largely identical to those observed in the high EXT group. However, deactivation in the low EXT group was observed in the bilateral frontal poles, superior and middle temporal gyri, precentral gyrus, postcentral gyrus, angular gyrus, precuneus, and medial prefrontal cortex (Fig 6a and 6b).

Fig 6.

Fig 6

(a.) Right SPL activation from task (aversive-maintain > neutral maintain) × group interaction (i.e. low EXT > high EXT). (b.) Activation in the right SPL across task conditions by group

Discussion

Previous models suggest that EXT is characterized by high levels of affective reactivity that overwhelm the cognitive processes that mediate emotion regulation (Baskin-Sommers et al., 2012a; Baskin-Sommers et al., 2012b; Buckholtz et al., 2010; MacCoon, Wallace, & Newman, 2004; Martin & Potts, 2004); however, little previous research has directly investigated the neural mechanisms of emotion reactivity and regulation among individuals with EXT. The present results, in contrast to predictions of cognitive-affective models of EXT, demonstrated anomalous activation primarily in regions commonly associated with affect, perception, and attention, suggesting an alternative perceptual-attentional pathway towards emotional dysregulation among high EXT individuals.

In this study, the high EXT group demonstrated lower overall amygdala activation during both maintenance and reappraisal conditions as compared to the low EXT group; however, no interaction was observed between group and condition, suggesting that both the low and high EXT groups down-regulated the amygdala during reappraisal to a similar extent. In contrast to the pattern observed in the amygdala, EXT was associated with decreased modulation of the LOC specifically during reappraisal. The LOC, a “high-level” visual cortical area, has been previously implicated in reappraisal of emotional images in other samples (Koenigsberg et al., 2009; Ochsner et al., 2002), and this effect was successfully replicated in the present study in the low EXT group. However, the high EXT group demonstrated no LOC modulation across task conditions, such that LOC activation was the same under maintenance and reappraisal conditions. This finding, in contrast to previous models of EXT, suggests a critical role of the visual system in emotion dysregulation among these individuals such that the systems that modulate and process affective input (i.e. perceptual information) are dysfunctional in EXT.

To the extent that emotional stimuli receive priority of processing resources, including perceptual resources (e.g. Pessoa & Ungerleider, 2004), the visual system is critical in both the experience and control of emotion. As such, modulation of visual input may represent an adaptive strategy for emotion reappraisal (Ehring et al., 2010; Lim et al., 2009; Mechelli, Price, Friston, & Ishai, 2004; Morris et al., 1998; Pessoa, 2009). In particular, previous work suggests that reappraisal-related visual modulation is due in part to changes in attentional allocation (Manera, Samson, Pehrs, Lee, & Gross, 2014; Ochsner & Gross, 2014). The present results, then, suggest that EXT-related emotion regulation deficits may be more primarily related to abnormal allocation of perceptual-attentional resources rather than emotional-hyperactivity, as suggested elsewhere. In particular, the results observed in the LOC may reflect cascading, down-stream effects of abnormal attentional allocation (Manera, Samson, Pehrs, Lee, & Gross, 2014; Ochsner & Gross, 2014). Consistent with this view, poor modulation was observed across conditions in the bilateral SPL in the high EXT group as compared to the low EXT group. Considering that this region of the SPL is part of the dorsal frontoparietal attention network (e.g. Corbetta & Shulman, 2002), this may suggest that high EXT individuals may be characterized by deficient perceptual-attentional modulation during both emotion reactivity and reappraisal.

While the present results cannot directly address differences in attentional deployment, the results nonetheless highlight the potential role of perceptual processes in emotion regulation and suggest that direct tests of attentional allocation of perceptual resources to emotional stimuli may be an important avenue for future research. More specifically, current theories of EXT emphasize cognitive dysfunctions typically localized to the prefrontal cortex as well as affective dysfunctions localized to limbic areas, suggesting that failures of emotion regulation in these individuals are due to poor recruitment of frontal “control” regions such as the dlPFC and vmPFC or over-activation of sub-cortical affective regions such as the amygdala and insula. These models generally imply a modular, “faculty” view of neural functioning, such that isolated regions are responsible for specific, categorical functions (Lindquist & Barrett, 2012), and that abnormal activation of these specific regions are related to psychopathology. The results here, however, failed to demonstrate differences in activation in these systems and instead suggest that activation in regions more commonly associated with perceptual processing and attentional allocation play a more critical role in the complex processes associated with affective dysregulation in EXT than previously thought (Kober et al., 2008). As such, the present results emphasize the integrated functioning of broadly distributed neural regions during emotion regulation as opposed to dysfunction of basic control vs. reflexive mechanisms in EXT.

These data provide important insight into the specific deficits associated with emotion dysregulation among individuals with EXT; nonetheless, important limitations to this study should be noted. First, the sample size presented here, although adequate to demonstrate significant effects, is relatively small and demographically homogenous. In future studies, a larger and more diverse sample may better represent EXT-related deficits. Second, while NA was controlled for across EXT groups in an effort to examine EXT-related effects without confounding levels of NA in this study, this approach is not without limitations. Specifically, to the extent that EXT is typically associated with high levels of NA, utilizing a high EXT-high NA group may reflect a more naturalistic group. However, to isolate the effects of EXT specifically, a factorial EXT × NA group design may be most ideal. Third, it is worth noting that despite robust differences in neural activation across groups, differences in behavioral ratings were not observed, thus cautious interpretation of the neural data is warranted. However, subjective ratings collected in this study may be particularly sensitive to demand characteristics or artificially limited by the 5-point scale used, and thus unreliable. Future studies may benefit from collecting psychophysiological measures of arousal such as skin conductance or pupil dilation. Finally, the present data are unable to adequately address the overall lower activation observed in the high EXT group in the regions presented here. This localized (i.e. not brain-wide) effect may be due to a number of factors including differences in arousal between groups. Alternatively, as relatively little is known about emotion reactivity and regulation in EXT, it may be possible that the mechanisms of emotion dysregulation are markedly different in this population as compared to those more widely understood (e.g. depression, anxiety). That is, emotion dysregulation in EXT may arise from dysfunction of different brain systems, including perceptual and attentional systems, than those commonly invoked in internalizing disorders, such as amygdala hyper-activation, in which case greater amygdala activation in the high EXT group would not necessarily be expected. Alternatively, as has been suggested elsewhere, emotion reactivity in EXT may be influenced by other task demands, including demands on attention and executive functions (e.g. Baskin-Sommers et al., 2012a). Continued research in emotion reactivity and regulation in EXT may help clarify these alternatives. Despite these limitations, the present study is a novel and important contribution to understanding the specific mechanisms of emotion dysregulation among individuals with EXT.

Critically, these findings provide novel insight into potential treatments for emotional problems among individuals with EXT. In particular, these findings imply that reappraisal may be a beneficial strategy for these individuals to control especially strong reactions to negative stimuli. However, to the extent that unique disruptions of a perceptual-affective system were observed during reappraisal, these data suggest that additional strategies or training may be required to fully control the emotional responses of high EXT individuals. Specifically, the findings suggest that training interventions should be targeted at the over-allocation of perceptual processing resources to aversive stimuli, rather than over-reaction of affective systems, per se.

Together, the results of the present study suggest that high EXT is characterized by deficits in regions commonly associated with attentional allocation and perceptual processing. These findings help specify the breakdowns associated with emotional problems in EXT.

Supplementary Material

11682_2015_9500_MOESM1_ESM
11682_2015_9500_MOESM2_ESM

Acknowledgments

The authors wish to thank Andrew Jahn for technical and computer programming support, Fernando Munoz for assistance with data analysis, and Ben Pruce and Colleen McCracken for assistance with data collection.

This work was supported by a grant from the National Institute of Health and National Institute on Alcohol Abuse and Alcoholism (NIH/NIAAA RO1 AA13650) to P.R.F., a National Institute on Drug Abuse training grant fellowship (NIDA T32 DA024628) to A.J.L, and by the Indiana METACyt Initiative of Indiana University, which was funded in part through a major grant from the Lilly Endowment, Inc. The authors wish to thank Andrew Jahn for technical and computer programming support, Fernando Munoz for assistance with data analysis, and Ben Pruce and Colleen McCracken for assistance with data collection

Funding: This work was supported by a grant from the National Institute of Health and National Institute on Alcohol Abuse and Alcoholism (NIH/NIAAA RO1 AA13650) to P.R.F., a National Institute on Drug Abuse training grant fellowship (NIDA T32 DA024628) to A.J.L, and by the Indiana METACyt Initiative of Indiana University, which was funded in part through a major grant from the Lilly Endowment, Inc

Footnotes

1

The following IAPS images were used: Neutral: 2020, 2210, 2215, 2221, 2230, 2235, 2270, 2271, 2357, 2372, 2381, 2383, 2385, 2389, 2393, 2394, 2410, 2435, 2440, 2441, 2480, 2485, 2487, 2493, 2495, 2499, 2514, 2516, 2518, 2570, 2575, 2579, 2580, 2595, 2635, 2745.1, 2749, 2850, 2870, 2880, 5455, 7493, 7496, 7550, 7620, 8311, 9070, 9210. Negative: 2053, 2095, 2661, 2683, 2691, 2710, 2800, 2900, 3010, 3160, 3170, 3180, 3181, 3230, 3266, 3300, 3301, 3350, 3500, 3530, 3550, 6010, 6212, 6242, 6244, 6312, 6313, 6315, 6350, 6370, 6415, 6510, 6540, 6560, 6821, 6831, 6838, 8485, 9040, 9252, 9253, 9265, 9400, 9410, 9433, 9800, 9810, 9910.

Compliance with Ethical Standards: The authors declare that they have no conflict of interest. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.

References

  1. Baskin-Sommers AR, Curtin JJ, Larson CL, Stout D, Kiehl KA, Newman JP. Characterizing the anomalous cognition–emotion interactions in externalizing. Biological Psychology. 2012a;91:48–58. doi: 10.1016/j.biopsycho.2012.05.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Baskin-Sommers A, Wolf R, Buckholtz J, Warren C, Newman J. Exaggerated attention blink response in prisoners with externalizing. Journal of Research in Personality. 2012b;46:688–693. doi: 10.1016/j.jrp.2012.08.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bechara A, Martin EM. Impaired decision making related to working memory deficits in individuals with substance addictions. Neuropsychology. 2004;18:152–162. doi: 10.1037/0894-4105.18.1.152. [DOI] [PubMed] [Google Scholar]
  4. Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory-Second edition (BDI-II) San Antonio, TX: The Psychological Corporation; 1996. [Google Scholar]
  5. Bobova L, Finn PR, Rickert ME, Lucas J. Disinhibitory psychopathology and delay discounting in alcohol dependence: Personality and cognitive correlates. Experimental and Clinical Psychopharmacology. 2009;17:51–61. doi: 10.1037/a0014503. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bucholz KK, Cadoret R, Cloninger CR, Dinwiddie SH, Hesselbrock VM, Nurnberger JI, et al. Schuckit MA. A new, semi-structured psychiatric interview for use in genetic linkage studies: a report on the reliability of the SSAGA. Journal of Studies on Alcohol. 1994;55:149–158. doi: 10.15288/jsa.1994.55.149. [DOI] [PubMed] [Google Scholar]
  7. Buckholtz JW, Treadway MT, Cowan RL, Woodward ND, Benning SD, Li R, et al. Zald DH. Mesolimbic dopamine reward system hypersensitivity in individuals with psychopathic traits. Nature Neuroscience. 2010;13:419–421. doi: 10.1038/nn.2510. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Cantrell H, Finn PR, Rickert ME, Lucas J. Decision making in alcohol dependence: Insensitivity to future consequences and comorbid disinhibitory psychopathology. Alcoholism: Clinical and Experimental Research. 2008;32:1398–1407. doi: 10.1111/j.1530-0277.2008.00714.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Corbetta M, Shulman GL. Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience. 2002;3:201–215. doi: 10.1038/nrn755. [DOI] [PubMed] [Google Scholar]
  10. Cox RW. AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research. 1996;29:162–173. doi: 10.1006/cbmr.1996.0014. [DOI] [PubMed] [Google Scholar]
  11. Critchley HD, Rotshtein P, Nagai Y, O'Doherty J, Mathias CJ, Dolan RJ. Activity in the human brain predicting differential heart rate responses to emotional facial expressions. Neuroimage. 2005;24:751–762. doi: 10.1016/j.neuroimage.2004.10.013. [DOI] [PubMed] [Google Scholar]
  12. Cyders MA, Smith GT. Mood-based rash action and its components: Positive and negative urgency. Personality and Individual Differences. 2007;43:839–850. [Google Scholar]
  13. Cyders MA, Smith GT. Emotion-based dispositions to rash action: Positive and negative urgency. Psychological Bulletin. 2008;134:807–828. doi: 10.1037/a0013341. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Ehring T, Tuschen-Caffier B, Schnülle J, Fischer S, Gross JJ. Emotion regulation and vulnerability to depression: Spontaneous versus instructed use of emotion suppression and reappraisal. Emotion. 2010;10:563–572. doi: 10.1037/a0019010. [DOI] [PubMed] [Google Scholar]
  15. Endres MJ, Donkin C, Finn PR. An information processing/associative learning account of behavioral disinhibition in externalizing psychopathology. Experimental and Clinical Psychopharmacology. 2014;22:122–132. doi: 10.1037/a0035166. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Endres MJ, Rickert ME, Bogg T, Lucas J, Finn PR. Externalizing psychopathology and behavioral disinhibition: Working memory mediates signal discriminability and reinforcement moderates response bias in approach–avoidance learning. Journal of Abnormal Psychology. 2011;120:336–351. doi: 10.1037/a0022501. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Eysenck HJ, Eysenck SBG. Manual of the Eysenck Personality Questionnaire. London: Hodder and Stoughton; 1975. [Google Scholar]
  18. Finn PR. Motivation, working memory, and decision making: A cognitive-motivational theory of personality vulnerability to alcoholism. Behavioral and Cognitive Neuroscience Reviews. 2002;1:183–205. doi: 10.1177/1534582302001003001. [DOI] [PubMed] [Google Scholar]
  19. Finn PR, Gunn RL, Gerst KR. The effects of a working memory load on delay discounting in those with externalizing psychopathology. Clinical Psychological Science. 2015;3:202–214. doi: 10.1177/2167702614542279. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Finn PR, Rickert ME, Miller MA, Lucas J, Bogg T, Bobova L, Cantrell H. Reduced cognitive ability in alcohol dependence: Examining the role of covarying externalizing psychopathology. Journal of Abnormal Psychology. 2009;118:100–116. doi: 10.1037/a0014656. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Garnefski N, Kraaij V, van Etten M. Specificity of relations between adolescents' cognitive emotion regulation strategies and internalizing and externalizing psychopathology. Journal of Adolescence. 2005;28:619–631. doi: 10.1016/j.adolescence.2004.12.009. [DOI] [PubMed] [Google Scholar]
  22. Gorenstein EE, Newman JP. Disinhibitory psychopathology: a new perspective and a model for research. Psychological Review. 1980;87:301–315. [PubMed] [Google Scholar]
  23. Gross JJ. Emotion regulation: Affective, cognitive, and social consequences. Psychophysiology. 2002;39:281–291. doi: 10.1017/s0048577201393198. [DOI] [PubMed] [Google Scholar]
  24. Gross JJ. Emotion regulation. In: Lewis M, Haviland-Jones JM, Barrett LF, editors. Handbook of Emotions. 3rd. New York, NY: Guilford Press; 2008. pp. 497–513. [Google Scholar]
  25. Iacono WG, Malone SM, McGue M. Behavioral disinhibition and the development of early-onset addiction: common and specific influences. Annual Review of Clinical Psychology. 2008;4:325–348. doi: 10.1146/annurev.clinpsy.4.022007.141157. [DOI] [PubMed] [Google Scholar]
  26. Johnstone T, van Reekum CM, Urry HL, Kalin NH, Davidson RJ. Failure to regulate: counterproductive recruitment of top-down prefrontal-subcortical circuitry in major depression. The Journal of Neuroscience. 2007;27:8877–8884. doi: 10.1523/JNEUROSCI.2063-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kober H, Barrett LF, Joseph J, Bliss-Moreau E, Lindquist K, Wager TD. Functional grouping and cortical–subcortical interactions in emotion: A meta-analysis of neuroimaging studies. Neuroimage. 2008;42:998–1031. doi: 10.1016/j.neuroimage.2008.03.059. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Koenigsberg HW, Siever LJ, Lee H, Pizzarello S, New AS, Goodman M, et al. Prohovnik I. Neural correlates of emotion processing in borderline personality disorder. Psychiatry Research: Neuroimaging. 2009;172:192–199. doi: 10.1016/j.pscychresns.2008.07.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Krueger RF, Hicks BM, Patrick CJ, Carlson SR, Iacono WG, McGue M. Etiologic connections among substance dependence, antisocial behavior and personality: Modeling the externalizing spectrum. Journal of Abnormal Psychology. 2002;111:411–424. [PubMed] [Google Scholar]
  30. Krueger RF, Markon KE. Reinterpreting comorbidity: A model-based approach to understanding and classifying psychopathology. Annual Review of Clinical Psychology. 2006;2:111–133. doi: 10.1146/annurev.clinpsy.2.022305.095213. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Lang PJ, Bradley MM, Cuthbert BN. International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical Report. 2008;A-8 [Google Scholar]
  32. Lim SL, Padmala S, Pessoa L. Segregating the significant from the mundane on a moment-to-moment basis via direct and indirect amygdala contributions. Proceedings of the National Academy of Sciences. 2009;106:16841–16846. doi: 10.1073/pnas.0904551106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Lindquist KA, Barrett LF. A functional architecture of the human brain: Emerging insights from the science of emotion. Trends in Cognitive Sciences. 2012;16:533–540. doi: 10.1016/j.tics.2012.09.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. MacCoon DG, Wallace JF, Newman JP. Self-regulation: The context-appropriate allocation of attentional capacity to dominant and non-dominant cues. In: Baumeister RF, Vohs KD, editors. Handbook of Self-Regulation: Research, Theory, and Applications. New York, NY: Guilford; 2004. pp. 422–446. [Google Scholar]
  35. Manera V, Samson AC, Pehrs C, Lee IA, Gross JJ. The eyes have it: The role of attention in cognitive reappraisal of social stimuli. Emotion. 2014;14:833. doi: 10.1037/a0037350. [DOI] [PubMed] [Google Scholar]
  36. Martin LE, Potts GF. Reward sensitivity in impulsivity. Neuroreport. 2004;15:1519–1522. doi: 10.1097/01.wnr.0000132920.12990.b9. [DOI] [PubMed] [Google Scholar]
  37. Mechelli A, Price CJ, Friston KJ, Ishai A. Where bottom-up meets top-down: Neuronal interactions during perception and imagery. Cerebral Cortex. 2004;14:1256–1265. doi: 10.1093/cercor/bhh087. [DOI] [PubMed] [Google Scholar]
  38. Morgan AB, Lilienfeld SO. A meta-analytic review of the relation between antisocial behavior and neuropsychological measures of executive function. Clinical Psychology Review. 2000;20:113–136. doi: 10.1016/s0272-7358(98)00096-8. [DOI] [PubMed] [Google Scholar]
  39. Morris JS, Friston KJ, Büchel C, Frith CD, Young AW, Calder AJ, Dolan RJ. A neuromodulatory role for the human amygdala in processing emotional facial expressions. Brain. 1998;121:47–57. doi: 10.1093/brain/121.1.47. [DOI] [PubMed] [Google Scholar]
  40. Moses EB, Barlow DH. A new unified treatment approach for emotional disorders based on emotion science. Current Directions in Psychological Science. 2006;15:146–150. [Google Scholar]
  41. Newman JP, Lorenz AR. Response modulation and emotion processing: Implications for psychopathy and other dysregulatory psychopathology. In: Davidson RJ, Scherer K, Goldsmith HH, editors. Handbook of Affective Sciences. New York, NY: Oxford University Press; 2003. pp. 904–929. [Google Scholar]
  42. Nichols T. Multiple testing corrections, nonparametric methods, and random field theory. Neuroimage. 2012;62:811–815. doi: 10.1016/j.neuroimage.2012.04.014. [DOI] [PubMed] [Google Scholar]
  43. Ochsner KN, Bunge SA, Gross JJ, Gabrieli JDE. Rethinking feelings: An fMRI study of the cognitive regulation of emotion. Journal of Cognitive Neuroscience. 2002;14:1215–1229. doi: 10.1162/089892902760807212. [DOI] [PubMed] [Google Scholar]
  44. Ochsner KN, Gross JJ. The cognitive control of emotion. Trends in Cognitive Sciences. 2005;9:242–249. doi: 10.1016/j.tics.2005.03.010. [DOI] [PubMed] [Google Scholar]
  45. Ochsner KN, Gross JJ. Cognitive emotion regulation insights from social cognitive and affective neuroscience. Current Directions in Psychological Science. 2008;17:153–158. doi: 10.1111/j.1467-8721.2008.00566.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Ochsner KN, Gross JJ. The neural bases of emotion and emotion regulation: A valuation perspective. In: Gross JJ, editor. Handbook of Emotion Regulation. New York, NY: Guilford Press; 2014. pp. 23–42. [Google Scholar]
  47. Ochsner KN, Ray RR, Hughes B, McRae K, Cooper JC, Weber J, et al. Gross JJ. Bottom-up and top-down processes in emotion generation common and distinct neural mechanisms. Psychological Science. 2009;20:1322–1331. doi: 10.1111/j.1467-9280.2009.02459.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Patterson CM, Newman JP. Reflectivity and learning from aversive events: Toward a psychological mechanism for the syndromes of disinhibition. Psychological Review. 1993;100:716–736. doi: 10.1037/0033-295x.100.4.716. [DOI] [PubMed] [Google Scholar]
  49. Pessoa L. How do emotion and motivation direct executive control? Trends in Cognitive Sciences. 2009;13:160–166. doi: 10.1016/j.tics.2009.01.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Pessoa L, Underleider LG. Neuroimaging of attention and the processing of emotion-laden stimuli. Progress in Brain Research. 2004;144:171–182. doi: 10.1016/S0079-6123(03)14412-3. [DOI] [PubMed] [Google Scholar]
  51. Sabatinelli D, Bradley MM, Lang PJ, Costa VD, Versace F. Pleasure rather than salience activates human nucleus accumbens and medial prefrontal cortex. Journal of Neurophysiology. 2007;98:1374–1379. doi: 10.1152/jn.00230.2007. [DOI] [PubMed] [Google Scholar]
  52. Simons JS, Carey KB, Gaher RM. Lability and impulsivity synergistically increase risk for alcohol-related problems. The American Journal of Drug and Alcohol Abuse. 2004;30:685–694. doi: 10.1081/ada-200032338. [DOI] [PubMed] [Google Scholar]
  53. Spielberger CD, Gorsuch RL, Lushene RE. Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists; 1970. [Google Scholar]
  54. Wager TD, Davidson ML, Hughes BL, Lindquist MA, Ochsner KN. Prefrontal-subcortical pathways mediating successful emotion regulation. Neuron. 2008;59:1037–1050. doi: 10.1016/j.neuron.2008.09.006. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

11682_2015_9500_MOESM1_ESM
11682_2015_9500_MOESM2_ESM

RESOURCES