FUNCTIONAL NEURAL CORRELATES OF FACIAL AFFECT RECOGNITION IMPAIRMENT FOLLOWING TBI

Abstract

Facial affect recognition deficits following traumatic brain injury (TBI) have been well documented, as has their relationship with impairment in several other cognitive domains. However, little is known about the neurobiological mechanisms underlying affect recognition deficits, in particular mechanisms underlying different aspects of facial affect recognition (e.g., perceptual and interpretive processes). In the current study, 33 adults with moderate-to-severe TBI and 24 demographically matched healthy comparison (HC) participants completed an fMRI facial affect recognition study. While in the scanner, participants were asked to match the affect of a target face to either (a) one of two faces differing in affect (perceptual condition) or (b) one of two written affect labels (interpretative condition). In both groups we found activations in regions typically involved in affect recognition. Our results revealed that in the perceptual condition individuals with TBI tended to activate the left dorsolateral prefrontal cortex less than HCs, and within the HC group individuals with higher perceptual affect recognition scores showed higher levels of activation in the same brain region. Individuals with TBI who were specifically impaired at interpretative affect recognition showed less activation than HCs in the right fusiform gyrus. Moreover, in the labeling condition individuals with TBI tended to de-activate medial prefrontal regions less than HCs. A region of interest analysis revealed that individuals with TBI showed significantly less activation than HCs in the FFA for all the contrasts of interest. Our results suggest involvement of several brain regions in facial affect recognition impairment post TBI, and provide neurobiological support for the notion that distinct aspects of facial affect recognition can be differentially impaired following TBI.

Introduction

Impairments in recognizing facial affect have been well-documented in individuals with moderate-to-severe traumatic brain injury (TBI) (Bornhofen & McDonald, 2008; Croker & McDonald, 2005; Green, Turner, & Thompson, 2004; S. McDonald et al., 2011; Rigon, Turkstra, Mutlu, & Duff, 2016, In press; Rosenberg, Dethier, Kessels, Westbrook, & McDonald, 2015; Rosenberg, McDonald, Dethier, Kessels, & Westbrook, 2014; Ylvisaker, Turkstra, & Coelho, 2005), little work has investigated the underlying mechanisms of these deficits (Neumann, Keiski, McDonald, & Wang, 2014; Neumann, McDonald, West, Keiski, & Wang, 2015). In particular, relatively little is known about the neurobiological underpinnings of facial affect recognition impairment, and about how injury-related functional and structural brain damage can lead or contribute to behavioral deficits.

Facial affect recognition abilities and their neural correlates have been extensively studied in healthy populations: A meta-analysis of over one hundred functional magnetic resonance imaging (fMRI) studies found that the brain regions involved included visual perceptual regions (primary and secondary visual areas, and in particular the fusiform face area, FFA), frontal and temporal limbic regions important for affective experience and processing (insula, amygdala and para-hippocampal areas), as well as dorsal and ventral prefrontal regions, which play a role in affect categorization and contextual understanding (Sabatinelli et al., 2011). In particular, a study from Hariri and colleagues (2000) focused on teasing apart the neural correlates of two different types of affect recognition skills: perceptual skills (i.e., the ability to detect salient facial features carrying emotional information) and more interpretative skills, requiring the verbal categorization of said perceptual features (Hariri, Bookheimer, & Mazziotta, 2000). Perceptual and verbal categorization skills have been found to be related but separable, with the perceptual detection of facial features carrying emotionally salient information preceding the verbal categorization stage. In the study by Hariri and colleagues, perceptual skills were tested using an affect matching task (i.e., participants saw a target face and had to choose which of two additional faces showed the same emotion), while interpretative skills were tested using an affect labeling task (i.e., participants saw a target face and had to choose which of two labels correctly described the facial affect). The authors reported that although both conditions were associated with activation in visual and fronto-temporal affective and higher-order regions, the main difference between the two conditions was that during the labeling condition the amygdala and prefrontal activations were more negatively correlated (with an increase in prefrontal cortex activation and a decrease in amygdala activation). This study revealed that tasks requiring different types of affect recognition strategies elicit different types of neural response, providing evidence for the idea that different systems underlie each sub-skill in healthy individuals.

To date, two studies have used fMRI to examine the neural underpinnings of facial affect recognition deficits following TBI. One study (Rigon, Voss, Turkstra, Mutlu, & Duff, 2017) measured resting-state functional connectivity (Biswal, Yetkin, Haughton, & Hyde, 1995; Fox & Greicius, 2010) as the temporal correlation of brain activity among distinct brain regions during wakeful rest. The study examined the relationship between performance on an affect labeling task and functional connectivity within a network of brain regions supporting facial affect processing, such as prefrontal and medial temporal limbic regions, as well as secondary visual areas related to facial processing. Results indicated that individuals with TBI had weaker functional connectivity within this network compared to healthy individuals, but that greater functional connectivity was associated with better affect recognition performance within the TBI group.

In the second fMRI study to examine the underpinnings of affect recognition, Neumann and colleagues (Neumann et al., 2015) employed an event-related design to elicit affect labeling processes, and compared brain activation of healthy comparison participants vs. individuals with TBI who were either impaired and unimpaired in affect labeling. The authors’ findings revealed that activation in the right fusiform gyrus was modulated by both injury and facial affect labeling performance, with individuals with TBI who were not impaired in the task showing levels of activation that were more similar to healthy participants. These findings align with structural gray matter analyses, which have found significant correlations between gray matter values in secondary visual areas and affect recognition performance in TBI samples (Genova et al., 2015).

While the study by Neumann and colleagues brought new insight to the neural mechanisms of affect labeling in individuals with TBI, to our knowledge no study has used fMRI to identify neural correlates of different aspects of affect recognition deficits (i.e., perceptual facial-affect recognition vs. verbal categorization and interpretation of affective features, which have been found to have different neural correlates in healthy individuals) (Hariri et al., 2000). A number of studies have examined different types of affection recognition in TBI (Green et al., 2004; Ietswaart, Milders, Crawford, Currie, & Scott, 2008), and recent work carried out by our group (Rigon, Voss, Turkstra, Mutlu, & Duff, 2018) found that although individuals with TBI underperformed healthy individuals on both labeling and matching affect recognition tasks, they made significantly more errors on a labeling task than on a matching one, suggesting that difficulties in affect recognition in adults with TBI are predominantly driven by deficits in interpretative abilities. We do not know, however, how these error patterns relate to different neural mechanisms. Impaired facial affect recognition has been linked to profound negative sequelae in adults with TBI (e.g., problems in social communication and affect regulation, socially inappropriate behavior, poor social integration), and a deeper understanding of its underlying mechanisms can help shed light on the causes of impairment, as well as benefit clinicians and rehabilitators by informing development of effective treatments (Knox & Douglas, 2009; Neumann et al., 2014; Spikman et al., 2013). Indeed, given the promising results shown by lifestyle interventions and neuro-modulatory treatments targeting the functionality of specific brain networks (as opposed to the impaired behaviors) a deeper knowledge of the systems involved in affect recognition processes in individuals with TBI holds the potential to inform new therapeutic protocols (Barbey et al., 2015; Wang et al., 2014).

The aim of the current study was to examine the relationship between brain activation and affect recognition impairment following TBI. In particular, here we sought to explore for the first time the differential patterns of activation in response to tasks that require different affect recognition abilities: purely perceptual vs. requiring verbal categorization. Based on previous work, we hypothesized that individuals with TBI would display lower levels of activation in the FFA than healthy comparison (HC) participants when engaging in an interpretative emotion recognition (labeling task). As this was the first study to examine perceptual emotion recognition in TBI using fMRI, we carried out whole brain exploratory analyses to investigate the presence of abnormal patterns of activation during a matching task. Our analyses techniques allowed us to identify brain regions involved in facial affect recognition abilities following TBI, and to further our understanding of the neurobiological systems underlying different emotion recognition sub-skills.

Methods

Participants

Thirty-two individuals with moderate-severe TBI and 23 healthy comparisons (HC) were tested for this study. Participants were recruited through the University of Iowa community and through the University of Iowa Brain Injury Registry (Rigon, Turkstra, et al., 2016; Rigon, Voss, Turkstra, Mutlu, & Duff, 2016; Rigon et al., 2017). The sample was part of a larger behavioral study (Rigon et al., 2018). For the affect recognition task sample, the groups were not significantly different for age (t(36.37)=.95, p>.05), education(t(53)=−.6, p>.05), or sex (Χ²(1, N=55)=1.05, p>.05) (See Table 1). Due to a power outage during the scanning session, one participant with TBI was able to complete the affect recognition task but not the localizer task. Participant demographics for the localizer sample are described in table S1.

Table 1.

Demographics for the functional neuroimaging sample (affect recognition task)

	N	AGE (Mean±SD)	SEX (Females)	EDUCATION (Mean±SD)	CHRONICITY (Months, Mean±SD)
HC	23	49.17±20.12	9	15.39±2.13	N/A
TBI	32	55.78±13.76	17	15.03±2.22	116.38(±130.78)
Group Differences (p)	N/A	.35	.31	.55	N/A

Note: HC=Healthy comparison participants, TBI=Traumatic brain injury, p=p-value, SD=Standard Deviation, N/A=Not Applicable.

Inclusionary criteria for individuals with TBI were (1) history of moderate to severe TBI, (2) chronic post-injury phase (all participants were > 12 months post injury), (3) aphasia quotient higher than 93.8 on the Western Aphasia Battery (WAB) (Shewan & Kertesz, 1980). Language deficits were ruled out to ensure that participants were able to correctly understand and follow instruction, and that poor performance on the tasks administered was not due to language deficits. Participants sustained their TBI a minimum of 17 months and a maximum of 509 months before testing (Mean=116.38, SD=130.78). One participant had sustained two separate TBIs. Causes of injury were falls (17), motor vehicle accidents (12), assaults (2), and non-motor vehicle accidents (2). Cause of injury for the participant who could not complete the localizer task was a motor vehicle accident. TBI severity was assessed using the Mayo Classification System (Malec et al., 2007). In the current sample, information on GCS was available for 13 participants, on LOC for 28 participants, information on retrograde or anterograde amnesia on 23 participants, and on CT or MRI findings for 28 participants. For the participant who did not complete the localizer task, information of LOC and PTA were available.

fMRI acquisition parameters

Neuroimaging data were collected at the University of Iowa Magnetic Resonance Facilities, on a 3T whole-body GE MR750W scanner with a 32-channel RF head receive coil. High-resolution T1-weighted brain images were acquired using a 3D Brain Volume (BRAVO) protocol with 256 interleaved coronal slices, inversion time (TI)=450ms, echo time (TE)=3.25 ms, repetition time (TR)=8.46 ms, field of view (FOV)=256mm², voxel size=1mm³, and flip angle=12º.

T2*-weighted functional data were collected using a fast echo planar imaging (EPI sequence) with BOLD contrast (TR=2000ms, TE=30ms, 32 slices acquired in ascending order, voxel size=3.4x3.4x4.4mm, 64×64 matrix, flip angle=77º). A total of 255 volumes were collected for the affect recognition task, while 120 volumes were collected for the FFA localizer (see below).

FFA localizer

The purpose of the localizer task was to locate the region in the secondary visual areas showing the highest levels of activation in response to the presentation of facial stimuli. In the localizer task, participants saw blocks of either faces with a neutral expression or objects, and were instructed to attend to them without having to make a response. Pictures of faces were selected from the Radboud Faces Database, while pictures of objects were selected from the Bank of Standardized Stimuli (Brodeur, Dionne-Dostie, Montreuil, & Lepage, 2010; Dotsch, Wigboldus, Langner, & van Knippenberg, 2008). The structure of the localizer task was adapted from a previous study (Kanwisher, McDermott, & Chun, 1997). Each block included pictures of 8 different stimuli each presented twice in random order (see Figure S1A). The order of stimuli presentation was the same for all participants. Face blocks were alternated with Object blocks, for a total of six blocks (Faces–Objects–Faces–Objects–Faces–Objects). For each block, each item was on screen for 1 second, followed by a 250 millisecond fixation cross, for a total of 20 seconds per block. Between blocks, participants saw a fixation cross for 20 seconds. The task lasted 4 minutes, and was administered at the end of the 30-minute fMRI session. All stimuli were in black and white. In all Face blocks, half of the faces were male and half of the faces were female.

fMRI affect recognition task

The affect recognition task was adapted from Hariri and colleagues (2000) (Hariri et al., 2000). The task included three different conditions: Control, Match, and Label. For the Control condition, participants saw a target shape on the top of the screen, and were instructed to select the shape that perfectly matched it between two options on the bottom of the screen. For the Match condition, participants saw a face showing one of two possible affects (anger or fear) and were instructed to select which one of two faces on the bottom of the screen showed the same affect. The two possible answers were always faces of different individuals, and displayed either fear or anger (as in the original task). Faces that were distractors in one trial were used as target in other trials. In the Label condition, participants saw a target face on the top of the screen, and were instructed to choose the label that best described the affect displayed on the bottom of the screen. The labels were always ‘Afraid’ or ‘Angry’ (See Figure 1). For each condition and each block, the correct response was on the right side of screen on 50% of trials. All facial stimuli were selected from the Karolinska Direct Affect Database (Lundqvist, Bruce, & Ohman, 2015).

Fig 1.

Facial affect recognition task Facial affect recognition task (within the scanner). A. Timeline of the task, adapted by Hariri and colleagues (2000).

All participants were predominately right handed, and they selected the correct response using an MRI compatible response box and by pressing the index finger if the correct response was on the left side of the screen, and the middle finger if it was on the right side. For each trial, participants had 4 seconds to select the correct response, as opposed to 5 seconds in the original task. The duration was shortened based on pilot data suggesting that individuals with TBI, on average, responded to similar items within 3 seconds, and to shorten the length of the task and thus reduce the possibility of fatigue acting as a confound. The stimuli remained visible for 4 seconds regardless of how early participants made a response, or whether they were not able to respond within the timeframe assigned. If participants changed their mind and pressed another finger within the 4 seconds, the second answer was collected. Each block included 8 trials. In the Match and Label conditions, each block included 4 trials in which anger was the target affect, and 4 trials in which fear was the target affect. Between each trial, a fixation cross of 250 milliseconds appeared, resulting in blocks of 34 seconds. Between blocks participants were asked to attend to a fixation cross for 20 seconds. The fixation cross became larger two seconds before the following block was scheduled to start, thus warning participants of its beginning. Overall, the task lasted 8.5 minutes, and participants saw each condition 3 times, always in the same order: Label-Match-Control (x 3). The decision to keep the order of the conditions fixed between participants was made to prevent differential order of stimuli presentation from acting as a potential confound in correlational analyses. Before entering the scanner, participants practiced at least one block for each condition of the task, or until they felt satisfied with their grasp of the instructions.

Statistical analysis of in-scanner facial affect recognition behavioral data was carried out using a mixed effect analysis of variance (ANOVA), with group as between-factor and condition (Match, Label and Control) as within factor. Paired and independent sample t-tests were used for post-hoc tests.

Out-of-scanner affect recognition task

The affect recognition tasks were administered as part of a larger battery that followed the MRI session. The tasks consisted of a facial affect matching task to test perceptual affect recognition skills, and a facial affect labeling task to examine the ability to verbally categorize affects. The tasks were adapted from Palermo and colleagues, and were chosen over other emotion recognition tasks because of their ability to detect subtle inter-individual differences in emotion recognition abilities even within healthy populations (Palermo et al., 2017; Palermo, O'Connor, Davis, Irons, & McKone, 2013). As the tasks were later employed for correlational analyses, we chose instruments that could detect sufficient variance both within our population of interest and within the comparison group.

The matching task used is an odd-expression-out task, in which participants saw three faces displaying emotional expression. Two faces of different individuals showed the same expression, and a third shows a different expression (for a total of three different actors for each trial). Participants were asked to indicate the face displaying a different affect from the other two. The target affect and distractor affects were always one of the six basic affects (anger, happiness, disgust, fear, sadness and surprise). The version employed here consisted of 100 10s trials, in which participants saw all three faces simultaneously for 5 seconds. After 5 seconds, faces disappeared, and participants were given 5 more seconds to make their choice by pressing the button corresponding to the face showing a different affect. Participants could choose to select the answer that they thought correct whenever they preferred during the 10 seconds. To prevent poor performance due to fatigue, every twenty trials, participants were given the opportunity to take a break. We ensured that each participant was able to produce a motor response in the allotted time.

The labeling task consisted of 144 trials, and each trial lasted a total of 10 seconds, and consisted of a face present in the center of the screen for 2 seconds, followed by a blank screen that lasted 8 seconds. In order to decrease the working memory demand, for the whole 10 seconds, labels of the six basic affects remained visible at the bottom of the screen, and participants were asked to select the one corresponding to the affect shown by the stimulus. Participants could select the correct answer any time in the 10 seconds available. As soon as participants selected their answer, the task moved to the next trial. For both tasks, participants could not change their answer after making a decision. The matching task was always administered before the labeling task. This was decided following the procedure adopted by the creators of the task (Palermo et al., 2013). For further information about the development and psychometric features of out-of-scanner matching and labeling tasks and the way they were modified for the current study, see Palermo and colleagues, 2013, as well as Rigon and colleagues, 2018.

Results of the out-of-scanner tasks (but not of the in-scanner task) were used as the behavioral variable of interest when linking brain activation to affect recognition performance, as well as to divide individuals with TBI in impaired vs. unimpaired groups (See below under the Affect recognition task section for further information, as well as Rigon et al., 2018). We did this because the out-of-scanner-tasks were expressly created to test a wide range of affects, as well as to uncover individual differences, and thus a better representation of participants’ affect recognition abilities relative to each other. On the other hand, the in-scanner task was markedly easier to limit difficulty as a confound in brain activation differences, and thus designed to elicit brain activation corresponding to different affect recognition skills, with fewer trials (24 per condition) and several participants performing to ceiling. Therefore, performance on the in-scanner task is not ideal for behavioral analysis designed to capture individual differences in affect recognition sub-skills. Statistical analysis of out-of-scanner tasks was carried out using one-tailed independent sample t-tests, with the hypothesis that the TBI group would underperform the HC group.

fMRI data analysis

Quality check: Motion correction and group comparison

On average, in the affect recognition task individuals with TBI moved significantly more than HCs both when it came to frame-to-frame relative motion (TBI Mean=.13±.1, HC Mean=.08±.04; t(40.04)=2.22, p<.05), and absolute motion from the middle reference volume (TBI Mean=.42±.22, HC Mean=.26±.13; t(51.38)=3.44, p<.01). For the FFA localizer, individuals with TBI did not move significantly more than HCs when it came to relative motion (TBI Mean=.16±.21, HC Mean=.08±.04; t(32.44)=2.03, p>.05), but they did when it came to absolute motion (TBI Mean=.4±.52, HC Mean=.16±.13; t(34.63)=2.4, p<.01). In order to prevent group differences in motion from affecting the neuroimaging results, a data-driven method that isolates motion-related signal in the data (i.e., ICA AROMA) was employed (see below) (Pruim, Mennes, Buitelaar, & Beckmann, 2015; Pruim, Mennes, van Rooij, et al., 2015). In addition, participants were excluded for excessive motion based on three different criteria: (1) if absolute motion exceeded the size of one functional voxel (3.4mm), (2) if average relative motion exceeded .5 mm, and (3) if visual inspection of the timeseries revealed motion highly correlated with the task blocks or with a specific condition (Gordon et al., 2015). Following these criteria, no participants were excluded from the facial affect recognition sample.

FFA localizer

Data processing was carried out in FSL 5.0.4 using FEAT (Smith et al., 2004; M. W. Woolrich et al., 2009). Individual–level analysis included motion correction using MCFLIRT, brain extraction using BET, and spatial smoothing with a 6 mm FWHM Gaussian kernel (Smith, 2002). Following these preprocessing steps data were denoised using the software package ICA-AROMA, a data-driven method that allows identification of motion-related independent components from fMRI data (Pruim, Mennes, Buitelaar, et al., 2015; Pruim, Mennes, van Rooij, et al., 2015). ICA-AROMA has been employed for motion related denoising in several task-related fMRI studies (Nieuwhof et al., 2017; Parada, Gerard, Larcher, Dagher, & Binik, 2016; Samu et al., 2017; Sood & Sereno, 2016; Yang et al., 2017). Briefly, ICA-AROMA relies on the Multivariate Exploratory Linear Decomposition into Independent Components (MELODIC) to extract independent components from the data and classifies them as motion-related based on whether they exceed one of three criteria: (1) a decision boundary combining the edge fraction and maximum realignment parameters correlation, (2) a cerebrospinal fluid fraction higher than 10%, or (3) a high-frequency content larger than 35%. For the localizer within the TBI group, an average of 34.35 components were extracted (SD=6.17), and an average or 20.26 components were classified as noise (SD=6.51), with an average ratio of noise components to total components of .58 (SD=.099). Within the HC group, an average of 32.43 components were extracted (SD=4.49), and an average or 17.91 components were classified as noise (SD=3.93), with an average ratio of noise components to total components of .54 (SD=.08). The two groups did not significantly differ in the number of total or noise components extracted (all t(52)<1.54, all p>.05). The ratio of noise to total components was typical for this type of denoising procedure (Kelly et al., 2010; Pruim, Mennes, van Rooij, et al., 2015). A non-aggressive denoising procedure was employed (i.e., partial component regression).

Following denoising, data were high-pass filtered (.0125 Hz), convolved with the double gamma hemodynamic response function, and registered to standard space using the non-linear registration tool FNIRT and boundary based registration (BBR) (Andersson, Jenkinson, & Smith, 2007a, 2007b).

Group-level analysis was carried out focusing on the contrast faces>objects (see figure S1B), as well as the group-by-condition interaction for said contrast. We additionally used FEAT’s outlier de-weighting procedure to detect outlier data points in each participant’s statistical map with respect to all other subjects and de-weight them in the group statistical maps (M. Woolrich, 2008). Z statistic images were thresholded using clusters determined by Z>3.1 and cluster significance threshold of p<.05 (Worsley, 2001). As there were no significant results at the group-level for the faces>objects contrast, a structural region of interest (ROI) was employed to determine peak voxels at the subject level. The structural map was selected by using a combination of the bilateral masks of the temporal and temporo-occipital fusiform cortex from the Harvard-Oxford cortical Atlas distributed with FSL, and then eroding the resulting mask by two 2mm voxels in order to avoid peaks too close to the mask edge (Desikan et al., 2006; Frazier et al., 2005; Goldstein et al., 2007; Makris et al., 2006). Spherical FFA ROIs (18mm diameter) centered on the peak coordinates were then created and used as individual FFA ROIs for task analyses.

Affect recognition task

Preprocessing

For the affect recognition task, individual–level analysis included motion correction using MCFLIRT, brain extraction using BET, spatial smoothing with a 6 mm FWHM Gaussian kernel, and denoising using ICA-AROMA (Jenkinson, Bannister, Brady, & Smith, 2002; Pruim, Mennes, van Rooij, et al., 2015; Smith, 2002). For the emotion recognition task within the TBI group, an average of 56.06 components were extracted (SD=6.22), and an average or 33.91 components were classified as noise (SD=6.82), with an average ratio of noise components to total components of .6 (SD=.09). Within the HC group, an average of 55.74 components were extracted (SD=8.69), and an average or 31.48 components were classified as noise (SD=4.77), with an average ratio of noise components to total components of .57 (SD=.09). The two groups did not significantly differ in the number of total or noise components extracted (all t(53)<1.47, all p>.05). As for the FFA localizer, a non-aggressive denoising procedure was employed. Following denoising, data were high-pass filtered based on the total cycle time between the same condition (.006 Hz), convolved with the double gamma hemodynamic response function, and registered to standard MNI space using the nonlinear registration tool FNIRT and BBR (Andersson et al., 2007a, 2007b).

Task activation analysis

Traditional regional activation analysis was performed to examine differences in magnitude of activation across the contrasts of interest between the two groups. Group maps for the contrasts of interests (Label>Control and Match>Control, as well as their interaction with group) were created using the FSL outlier de-weighting procedure and thresholded using clusters determined by Z>3.1 and a cluster threshold of p<.05 (Worsley, 2001). Neuroanatomical location of significant clusters was determined using the Harvard-Oxford Cortical and Subcortical Atlases (Desikan et al., 2006; Frazier et al., 2005; Goldstein et al., 2007; Makris et al., 2006).

Group-level regional activation analyses were carried out examining group differences for each of the two main contrasts of interest, Label>Control and Match>Control. Two different types of group comparisons were carried out: (1) The first analysis included a two-group comparison of TBI vs. HC; (2) The second analysis was a three-group comparison. Similar to the methods employed in previous studies on TBI populations, for the three-group comparison, individuals with TBI were defined as either impaired (TBI-I) or non-impaired (TBI-NI) in each of the two out-of-scanner affect recognition tasks (matching and labeling) based on how they performed compared to the average of the HC group (Neumann et al., 2015). As there are no normative data for the tasks that would cover the age range of participants of the current study, we used the HC’s scores to determine whether individuals with TBI were considered impaired or not in either the labeling or matching task. Participants who performed 2 standard deviations below the HC average or worse were considered impaired, leading to TBI-I=14 and TBI-NI=18 for the labeling task, and TBI-I=3 and TBI-NI=29 for the matching task. Examining the difference in demographic variables revealed that, as expected given the negative correlation between affect recognition performance and age, for labeling TBI-I were significantly older than TBI-NI (t(27.77)=3.75, p<.01), and than HCs (t(31.27)=2.74, p<.05). There were no significant differences in education between TBI-I and TBI-NI (t(30)=−.39, p<.05), or between TBI-I and HC (t(35)=−.69, p>.05), nor in sex composition between TBI-I and TBI-NI (Χ²(1, N=32)=.1, p>.05) or between TBI-I and HC (Χ²(1, N=37)=.42 p>.05). For matching there was no significant age difference between TBI-I and TBI-NI (t(30)=1.04, p>.05), and between TBI-I and HCs (t(24)=.31, p>.05), although it should be noted that on average TBI-I tended to be older than both TBI-NI (Mean TBI-I=61,67, SD=10.69, Mean TBI-NI=52.97, SD=13.93) and HCs (Mean TBI-NC=49.17, SD=20.11), and that the lack of statistical significance was likely due to the small sample size. There were no significant differences in education between TBI-I and TBI-NI (t(30)=−1.41, p<.05), or between TBI-I and HC (t(24)=−1.57, p>.05), nor in sex composition between TBI-I and TBI-NI (Χ²(1, N=32)=.5, p>.05) or between TBI-I and HC (Χ²(1, N=26)=.04 p>.05). For labeling, we examined how impairment in affect recognition was related to brain activity during an affect recognition task following TBI using an analysis of covariance (ANCOVA) and effect coding with the HC group as the reference group. Given the age difference between TBI-I and HC, as well as the correlations between age and affect recognition performance, age was added as a covariate. The contrast of interest was Label>Control, with a focus on the HC>TBI-I and TBI-I>HC contrasts. The same was done for the matching task, although it should be noted that only three individuals with TBI were impaired in the task, and that the main contrast of interest was Match>Control.

Next, correlational analyses between out-of-scanner performance on matching and labeling tasks and brain activation for, respectively, the Match>Control contrast and the Label>Control contrast were carried out. Correlational analyses were performed only using the two-group approach (i.e., TBI vs. HC). Two types of designs were used: within group correlational analysis for both the TBI and HC groups, always adding sex and age as covariates, given how both factors are related to affect recognition abilities both within healthy and brain injury populations (Montagne, Kessels, Frigerio, de Haan, & Perrett, 2005; Rigon, Turkstra, et al., 2016); and interaction analysis, examining the between-group difference in the linear association between brain activity and performance on the affect recognition tasks. Both analyses were whole brain and exploratory (Z>3.1, p<.05).

Lastly, we examined group differences in brain activation by extracting percent signal change (PSC) from specific ROIs. In particular, we focused on two regions that have been found to be important in affect recognition in both TBI and other populations: the FFA (subject-specific) and the left and right amygdalae (using the ROIs described above). For each ROI, PSC was extracted using Featquery, and independent t-tests were used to examine group differences.

Results

Within-scanner and out-of-scanner affect recognition tasks–behavioral results

Within the scanner, a repeated measure ANOVA revealed both significant main effects of group (F(1,53)=8.85, p<.01) and condition (F(1,53)=80.18, p<.001), and a significant group-by-condition interaction (F(2,108)=4.34, p<.05). The interaction was driven by the fact that individuals with TBI made significantly more errors than HCs on both the Match (t(53)=−2.22, p<.05) and Label (t(53)=−2.9, p<.05) conditions of the affect recognition task. However, there was no significant difference on the Control condition (t(47.66)=−1.43, p>.05), suggesting that impairment was specific to the emotion recognition conditions. Within the TBI group, participants were significantly more accurate in the Control condition than in the Match and Label conditions (all t(31)>8.59, all p<.001), but there was no significant difference in number of correct responses for the Match and Label condition (t(31)=−2.24, p>.01).The same pattern repeated in the HC group, with participants significantly more correct in the Control condition than in the Match and Label conditions (all t(22)>5.41, all p<.001), but with no significant difference in number of correct responses for the Match and Label condition (t(22)=−1.62, p>.05).

Analysis of reaction times revealed a significant main effect of group (F(1,53)=11.79, p<.01), with individuals with TBI’s responses significantly slower than HCs in all three conditions (all t(53)>2.92, all p<.01). There was also a significant main effect of condition (t(1,53)=460.74, p<.001), with participants significantly slower in the Match and Label conditions than in the Control conditions (all t(22)>−19.95, all p<.001), and faster in the Label condition than in the Match condition (t(22)=9.96, p<.001). However, the group-by-condition interaction was non-significant (t(2,108)=.13, p>.05).

For the out-of-scanner affect recognition tasks, individuals with TBI performed significantly worse than HCs both in the matching (t(53)=1.95, p<.05, Mean TBI=66.71±10.54, Mean HC=71.87±8.36) and in the labeling tasks (t(44.36)=−3.25, p<.01; Mean TBI=69.88±12.98, Mean HC=78.20±5.46). These results reflect previously published sample for the larger study sample (Rigon et al., 2018).

fMRI – Group comparison

HC vs. TBI

Although there was not a significant difference between individuals with TBI and HCs for the Label>Control condition, there was a significant difference for the Match>Control condition. HCs showed higher activation in a cluster in the left dorsolateral prefrontal cortex, due to a combination of higher activation in the TBI group during the Control condition, and lower activation within the TBI group during the Matching condition (Figure 2, table 2).

Fig 2.

Group comparison, match>control contrast. A) Peak activation map for the Match>Control contrast, HC>TBI. B) Percent signal change for the Match>Control contrast.

Table 2.

Activation peaks for the contrasts of interest

Contrast	Peak z	Coordinates			Cluster size	Anatomical location
		x	y	z
Match>Control, HC>TBI	4.16	−42	0	28	497	L Precentral gyrus
	4.13	−36	12	22		L Inferior frontal gyrus
	3.95	−42	0	42		L Precentral gyrus
	3.82	−42	6	36		L Middle frontal gyrus
	3.79	−36	6	22		L Precentral gyrus
	3.75	−44	8	40		L Middle frontal gyrus
Label>Control, HC>TBI-I	4.03	42	−68	−28	221	R Cerebellum
	4.02	34	−76	−20		R Occipital fusiform gyrus
	3.92	26	−70	−22		R Cerebellum
	3.75	34	−64	−28		R Cerebellum
	3.64	28	−62	−34		R Cerebellum
	3.55	34	−72	−26		R Cerebellum
Label>Control, TBI>HC	5.12	10	34	−6	2529	R Anterior Cingulate gyrus
	5.07	20	36	−6		R Frontal medial cortex
	5	4	24	0		R Subcallosal cortex
	4.94	4	10	6		R Lateral ventricle
	4.69	26	24	−10		R Orbito-frontal cortex
	4.51	18	28	−4		R Cerebral white matter
Matching>Control, HC>TBI	4.22	−38	46	32	206	L Frontal pole
	3.84	−26	60	30		L Frontal pole
	3.75	−28	52	32		L Frontal pole
	3.26	−22	64	30		L Frontal pole

Note: R = right, L = left. Coordinates are presented in MNI space. Anatomical regions were identified using the FSL Harvard-Oxford cortical and subcortical atlases.

Labeling ANCOVA (TBI-I vs. HC, TBI-NI vs. HC)

Although there are widespread differences in activation between groups (Figure 3A), the only statistically significant difference between groups revealed that individuals with TBI-I (individuals with TBI who were impaired in the affect labeling task) showed significantly lower activation than HCs in a cluster including the posterior part of the fusiform face area and the cerebellum for the Label>Control contrast (figure 3B, table 2). We further explored the data by extracting PSC, and found that although individuals with TBI who were not impaired at labeling (TBI-NI) displayed differences in activation between Control and Label similar to HCs, individuals with TBI who were impaired (TBI-I) tended to activate equally in both conditions (Figure 3C).

Fig 3.

Differential activation in the Label>Control contrast for individuals with TBI impaired at affect recognition. Activation maps for TBI- Impaired (TBI-I), TBI Not Impaired (TBI-NI) and healthy comparisons (HC). B) Significant cluster for the HC>TBI-I contrast, Label > Control contrast. C) Percent signal changes (PSC) across the three conditions.

Matching ANCOVA

There was no significant difference between HCs and individuals with TBI-I at matching for the Match>Control contrast (although it should be noted that only three individuals with TBI performed less than 2 standard deviations below average).

Interaction analyses

An exploratory interaction analysis revealed that HC individuals showed significantly stronger correlation between activation in a region of the ventral and medial prefrontal cortex for the Label>Control contrast and performance in the labeling task (Figure 4A; Table 2). Further investigation of the interaction revealed that within the TBI group there was no significant correlation between activation in the medial prefrontal cortex and performance on the labeling task (partial correlation correcting for sex age, r=−.12, p>.05), while within the HC group, individuals with higher labeling scores tended to activate the medial prefrontal cortex less for the Label>Control contrast (r=−.62, p<.01) (Figure 4B).

Fig 4.

Group-differences in the linear relationship between brain activity and labeling performance. A, B) Significant difference in the linear relationship between labeling performance and activation in the medial prefrontal cortex for the Label>Control contrast. C) Individuals with TBI de-activated this region significantly less than HCs.

We examined PSC across subjects for the Label>Control contrast, and an independent sample t-test revealed that individuals with TBI showed significantly less PSC than HCs (t(53)=1.93, p<.05), which was driven by the fact that they tended to deactivate less in the Label condition compared to the Control condition (Figure 5C). Thus, on average individuals with TBI tended to deactivate less in the medial prefrontal cortex for the Label>Control contrast, and participants who deactivated more (i.e., participants who were more similar to HCs) tended to perform better on the out of the scanner labeling task.

Fig 5.

Group-differences in the linear relationship between brain activity and matching performance. A, B) Significant difference in the linear relationship between matching performance and activation in the left dorsolateral prefrontal cortex for the Match>Control contrast. C) Individuals with TBI activated this region significantly less than HCs.

Conversely, HCs showed a stronger correlation between activation in a cluster in the left prefrontal cortex on the Match>Control contrast and performance on the out of the scanner matching task. The interaction was driven by a positive correlation between matching performance and PSC within the left dorsolateral prefrontal cortex within the HC group (r=.62, p=.001), while there was no significant correlation within the TBI group (r=−.23, p>.05) (Figure 5A; Table 2). Further examination showed that on average individuals with TBI showed lower levels of activation in the dorsolateral prefrontal cortex than for the Match>Control contrast (t(53)=−2.2, p<.05), which was mainly driven by lower levels of activation in the Match conditions (Figure 5B and C). In other words, participants with TBI tended to activate less than HCs in the dorsolateral prefrontal cortex, and healthy participants who activated more tended to perform better at matching.

ROI analysis

An ROI analysis was carried out to examine group differences in PSC in brain regions hypothesized to play a role in facial affect recognition. For the Label>Control contrast, there was no significant group difference for either the left (t(53)=−.12, p>.05) nor the right amygdala (t(53)=.25, p>.05). Similarly, there was no significant difference for the Match>Control contrast for either the left (t(53)=.199, p>.05) nor the right amygdala (t(53)=−2.1, p=.05).

Regarding the FFA, individuals with TBI showed lower activation than HCs both for the Label>Control contrast (t(52)=−2.57, p<.01) and for the Match>Control contrast (t(52)=2.02, p<.05) (Figure 6A & C). Interestingly, further inspection of the data revealed that for both contrasts, the group difference was mainly driven by individuals who were impaired at affect recognition, with TBI-I showing significantly less activation than HCs for Label>Control (t(34)=−3.63, p<.001), while TBI-NI did not (t(39)=−1.58, p=.12) , and TBI-I showing significantly less activation than HCs for Match>Control (t(24)=−4.82, p<.001) while TBI-NI did not (t(49)=−1.65, p=.12) (Figure 6B & D).

Fig 6.

ROI analysis results. A & C) Significant differences in FFA activation between the HC and TBI group; B & D) For both matching and labeling, the differences were driven by those individuals who were impaired at affect recognition.

Discussion

In this study, we used fMRI to identify neural correlates of facial affect recognition in individuals with moderate-to-severe TBI. For the first time, based on the notion that affect recognition is not a unitary cognitive skill and on previous findings that individuals with TBI show significantly greater impairment on affect labeling tasks than affect matching tasks (Rigon et al., 2018), we focused on teasing apart perceptual and interpretative facial affect recognition processes. Individuals with TBI showed lower activation in the left dorsolateral prefrontal cortex while engaging in affect matching, and those who were impaired at affect labeling displayed lower activation in the right fusiform gyrus. We also found a difference in the linear relationship between behavior and brain activation: within the TBI group, labeling performance and deactivation of the medial prefrontal cortex during labeling were not significantly correlated; however, there was a significant negative correlation between labeling performance and medial prefrontal cortex deactivation within the HC group. Similarly, while there was a significant positive correlation between activation in the left dorsolateral prefrontal cortex and matching performance within the HC group, there was no such correlation in the TBI group. We next discuss each of our findings and their respective implications.

The primary significant difference in brain activation between the TBI and HC groups was for the Match>Control condition, with HCs showing higher levels of activation in the left dorsolateral prefrontal cortex. Interestingly, the left dorsolateral prefrontal cortex was also the area showing a significantly different linear relationship between brain activation and matching performance between the two groups. Although Hariri and colleagues (2000) did not report left prefrontal activation for the Match>Control task, other studies adapting the same experimental design have (Aupperle et al., 2012; Hariri et al., 2000). It should be noted that for this specific task the working memory load is much higher for the Match condition than the Label condition: in the former, participants see a target face and have to keep that face in mind as they scan the two options on the bottom. By contrast, in the labeling task, participants already know what the two labels will be (angry vs. afraid), and are only required to locate the position of the correct label. The fact that the Match condition is, in terms of working memory and visual search requirement, more demanding than the Label condition is supported by data showing that reaction times for the inside scanner Match condition were higher in both groups than for the Label condition. Given the working memory imbalance between the two conditions, and considering that individuals with TBI tend to present working memory problems, it is possible that our results reflect, in part, group differences in working memory. While the literature on working memory and fMRI has reported both instances in which deficits in working memory are accompanied by prefrontal hyper-activation (Kasahara et al., 2011; B. C. McDonald, Saykin, & McAllister, 2012) and hypo-activation (Kasahara et al., 2011; Newsome et al., 2008; Sanchez-Carrion et al., 2008), often depending on the specific region examined, in the current study it appears that the latter is the case.

Although there were no significant clusters showing less activation than in HCs for the participants who were impaired at affect matching, it should be noted that only three participants with TBI fit this criterion, and thus the TBI-I sample was likely too small to reveal differences. Indeed, it is noteworthy that the largest between-group differences were noted for the Match condition, when only three individuals with TBI were impaired at the task, and thus fewer group differences at the imaging level would be expected than for the Label condition. One possibility, that warrants further study, is that this is due to successful compensatory strategies by individuals with TBI when engaging in emotion matching.

We also found that individuals with TBI who were impaired at affect labeling tended to activate less in a small cluster in the right fusiform gyrus relative to healthy individuals, as well as in neighboring clusters in the cerebellum. The cerebellum is involved in the processing of facial affects (Ferrucci et al., 2012), while the differential FFA activation closely mirrors the reports of Neumann and colleagues. This finding, coupled with structural findings from other reports, suggests a role for more perceptual brain regions in the processes leading to labeling impairment (Neumann et al., 2015). As previously discussed by Neumann and colleagues, considering the evidence showing that the FFA is involved in holistic face processing (Kanwisher & Yovel, 2006), this finding supports the account that deficits on emotion labeling tasks might be due to a breakdown in the ability to integrate separate features into a configural unit (Maurer, Grand, & Mondloch, 2002), with important treatment implications. Indeed, while successful completion of the matching task does not necessary require holistic face processing (as specific features, such as the smile, can be isolated and matched individually), it is possible that for emotion labeling tasks these types of demands are higher.

Perhaps the most interesting findings were the significant differences in linear correlations between performance and activation in the two groups. As mentioned, HCs with better performance on the matching task also showed more activation in the left dorsolateral prefrontal cortex, and HCs as a group activated the left dorsolateral prefrontal cortex more on average. On the other hand, there was no significant correlation between activation in this region and performance within the TBI group, which suggests a disruption of mechanisms underpinning perceptual facial affect recognition. In particular, given the nature of the Match condition in the task, it is possible that lower DLPFC activation reflects an impairment in the ability to ‘hold in mind’ the abstract representation of the features of the target face carrying salient affective information (Frith & Dolan, 1997). Similarly, HCs with better performance on the labeling task also showed more deactivation in the ventromedial and sub-callosal prefrontal cortex, and HCs as a group activated less the left ventromedial prefrontal cortex on average, but there was no significant correlation within the TBI group. Structural gray matter analyses have linked the subcallosal cortex with the recognition of negative affects (such as the stimuli used in the current task) (Kumfor, Irish, Hodges, & Piguet, 2013), and sadness induction has been associated with higher levels of subcallosal activation (Phan, Wager, Taylor, & Liberzon, 2002), suggesting a role of these regions in emotion regulation and emotion processing. While future research to further investigate the role of these regions in affective deficits following TBI is warranted, our findings reveal possible cortical regions whose functionality might be targeted by behavioral and lifestyle interventions, to improve specific functions.

While we chose the emotion recognition task used here because of its widespread use and its length, it has a few limitations. First, the Control condition was perceptual in nature, and thus did not provide an ideal baseline for the Label condition. While other studies have opted for including blocks containing a more appropriate baseline (such as a gender labeling condition) (Gee et al., 2012), here we decided to use the original version of the task to minimize its duration and to be able to compare our results with a larger body of literature. Moreover, as with any matching task, without further experimental manipulations it is impossible to state the specific cognitive strategies used by participants to respond correctly (e.g., although it would be the more cognitively time-consuming strategy, it is possible that participants were labeling the target face and the two options, and then choosing their answer.)

Although the localizer task was designed based on the literature (i.e., using objects as the non-face condition in order to mimic as closely as possible the Control condition of the affect recognition task) the face>objects condition was surprisingly null (Berman et al., 2010; Kanwisher et al., 1997). Though this null finding might reflect our use of more conservative analysis procedures than those in previous studies, which we did to mitigate concerns related to statistical thresholding, it is notable that no clusters survived at the group level (Eklund, Nichols, & Knutsson, 2016). This was observed both within the HC and the TBI group, and it is unlikely to be due purely to a loss of FFA specificity to faces within the TBI group. Inspection of the individual maps revealed for both groups that when thresholding for Z>2.3. only about half of the participants exhibited peaks within the fusiform gyrus or other secondary visual areas. A possible explanation lies in the large age range of the participants of the current studies, as a loss of neural specificity for perception of visual stimuli is often reported in the aging literature (Owsley, 2011). Nevertheless, future studies focusing on the FFA in individuals with TBI might benefit from the use of a scene as the non-face condition (Berman et al., 2010).

Conclusion

In the current study, we compared patterns of activation elicited by perceptual vs. interpretative affect recognition tasks. Our findings reveled that individuals with TBI showed disrupted activation in lateral prefrontal regions (in particular for perceptual skills) and in both medial prefrontal and occipital regions (in particular for verbal categorization skills). The findings support the idea that facial affect recognition is comprised of subcomponents that depend on distinct neural substrates, and thus advances knowledge of the neurobiology of affect recognition more broadly. These findings have the potential to inform the development of new modulatory and lifestyle interventions for adults with TBI, targeting specific neurobiological mechanisms of facial affect recognition impairment.