The ventromedial prefrontal cortex plays a crucial role in regulating emotion and social behavior, yet the precise mechanisms underlying this function remain unclear. Using eye-tracking in patients with brain lesions, Wolf et al. show that ventromedial prefrontal cortex is critical for directing visual attention during facial emotion recognition.
Keywords: attention, emotion, prefrontal cortex, social cognition, lesion studies, eye tracking
Abstract
The ventromedial prefrontal cortex is known to play a crucial role in regulating human social and emotional behaviour, yet the precise mechanisms by which it subserves this broad function remain unclear. Whereas previous neuropsychological studies have largely focused on the role of the ventromedial prefrontal cortex in higher-order deliberative processes related to valuation and decision-making, here we test whether ventromedial prefrontal cortex may also be critical for more basic aspects of orienting attention to socially and emotionally meaningful stimuli. Using eye tracking during a test of facial emotion recognition in a sample of lesion patients, we show that bilateral ventromedial prefrontal cortex damage impairs visual attention to the eye regions of faces, particularly for fearful faces. This finding demonstrates a heretofore unrecognized function of the ventromedial prefrontal cortex—the basic attentional process of controlling eye movements to faces expressing emotion.
Introduction
Beginning with the landmark case of Phineas Gage (Harlow, 1868), and corroborated by a series of neurological cases throughout the 20th century, it has been well-established that ventromedial prefrontal cortex (PFC) damage precipitates marked changes in personality and emotional function, including lack of empathy, social disinhibition, and impaired decision-making (Blumer and Benson, 1975; Eslinger and Damasio, 1985; Barrash et al., 2000). Neuropsychological studies aimed at elucidating the specific functions of the ventromedial PFC have traditionally examined the effect of ventromedial PFC damage on behavioural choices resulting from deliberative value-based decision-making processes, as in tasks involving risky gambles (Bechara et al., 1997; Camille et al., 2004), moral judgement (Ciaramelli et al., 2007; Koenigs et al., 2007; Young et al., 2010), probabilistic reinforcement learning (Fellows and Farah, 2003; Wheeler and Fellows, 2008), economic exchange (Koenigs and Tranel, 2007; Krajbich et al., 2009), and simple binary item preference (Henri-Bhargava et al., 2012). Accordingly, theoretical accounts of the ventromedial PFC’s critical role in social and affective function have focused on higher-order cognitive processes related to representations of emotion and value (Damasio, 1996; Fellows, 2011; O'Doherty, 2011; Myers-Schulz and Koenigs, 2012; Roy et al., 2012). However, there is a possibility that the ventromedial PFC may also contribute to more rapid, precursory stages of visual processing of socially and emotionally meaningful stimuli, on a timescale similar to the amygdala (Kawasaki et al., 2001; Pessoa and Adolphs, 2010). The ventromedial PFC is densely and reciprocally connected with the amygdala (Barbas, 2000; Ghashghaei and Barbas, 2002; Roy et al., 2009), where damage has been shown to impair the allocation of visual attention to the eye region of faces during emotion recognition (Adolphs et al., 2005). Moreover, ventromedial PFC damage has been associated with deficits in identifying facial expressions of emotion (Hornak et al., 2003; Heberlein et al., 2008; Tsuchida and Fellows, 2012), and at least one case report shows an effect of orbital and medial PFC damage on performance in a cued visual attention task (Vecera and Rizzo, 2004). Considered together, this combination of findings suggests that the ventromedial PFC, like the amygdala, may be critical for the allocation of visual attention to the emotionally expressive regions of faces. To test this hypothesis, we used eye tracking in a sample of neurological patients with focal, bilateral ventromedial PFC lesions during a test of facial emotion recognition.
Materials and methods
Participants
The target lesion group consisted of three neurosurgical patients with extensive bilateral parenchymal changes, largely confined to the ventromedial PFC, where ventromedial PFC is defined as Brodmann areas 11, 12, 25, 32, and the medial portion of 10 below the level of the genu of the corpus callosum (Fig. 1 and Supplementary Fig. 1). All three patients had large anterior cranial fossa meningiomas with vasogenic oedema. Their clinical presentations were subtle or obvious personality changes over at least several months preceding surgery. Each patient underwent gross total tumour resection without any intraoperative or postoperative complications. On post-surgical MRI, although vasogenic oedema largely resolved, there were persistent circumscribed bilateral ventromedial PFC lesions in each patient.
Ten neurosurgical patients who had focal lesions outside of ventromedial PFC comprised a brain-damaged comparison group, which included eight patients who had undergone tumour resections and two patients who had undergone surgery for aneurysm clipping following subarachnoid haemorrhage. Lesions in the brain-damaged comparison group involved ventral or lateral anterior temporal cortex (n = 5), dorsomedial frontal cortex (n = 3), lateral frontal and temporal cortex (n = 1), and cerebellum (n = 1) (Supplementary Fig. 2). All ventromedial PFC and brain-damaged comparison patients’ neurosurgeries were performed in adulthood, and all experimental data were collected at least 3 months after surgery, during the chronic phase of recovery. The inclusion of these brain-damaged comparison patients allows us to rule out the possibility that the pattern of eye fixations observed in the ventromedial PFC lesion group could be due to anatomically non-specific effects of brain damage or history of related medical issues (e.g. craniotomy, oedema, seizure, past medications, etc.). All neurosurgical patients (ventromedial PFC and brain-damaged comparison) were recruited through a patient registry established through the University of Wisconsin Department of Neurological Surgery.
Twenty-one neurologically healthy adults also participated as a normal comparison group. The normal comparison participants had no history of brain injury, neurological or psychiatric illness, or current use of psychoactive medication. Normal comparison participants were between the ages of 50 and 64 (the age range of ventromedial PFC patients; see Table 1 for group demographic and neuropsychological data). One normal comparison participant was excluded as a statistical outlier based on deficient emotion recognition performance (see task description below), and one was excluded due to technical difficulties with eye tracking. This resulted in a final normal comparison group size of 19. Normal comparison participants were recruited through community advertisement. All participants had normal or corrected to normal vision.
Table 1.
Group/ patient | Age | Education | Sex | IQ | PSI | Picture Completion | Trail A | Trail B − A | BDI | Trait Anxiety |
---|---|---|---|---|---|---|---|---|---|---|
vmPFC-A | 58 | 12 | F | 109 | 100 | 11 | 47.0 | 18.0 | 3 | 21 |
vmPFC-B | 50 | 12 | M | 88 | 81 | 13 | 32.7 | 65.9 | 9 | 40 |
vmPFC-C | 64 | 20 | M | 117 | 102 | 8 | 31.5 | 40.3 | 10 | 42 |
vmPFC (n = 3) | 57.3 (7.0) | 14.7 (4.6) | 2 M/1 F | 104.7 (15.0) | 94.3 (11.6) | 10.7 (2.52) | 37.1 (8.6) | 41.4 (24.0) | 7.3 (3.8) | 34.3 (11.6) |
BDC (n = 10) | 52.4 (10.8) | 14.5* (1.7) | 4 M/6 F | 102.3* (7.7) | N/A | N/A | 31.7 (9.3) | 31.9 (15.1) | 13.1*(8.3) | 43.6*(10.6) |
NC (n = 19) | 58.3 (3.3) | 16.6 (2.5) | 11 M/8 F | 110.2 (6.6) | N/A | N/A | N/A | N/A | 5.1 (5.4) | 31.4 (6.4) |
Age = age of participant at time of testing (years); Education = years of education completed; IQ = IQ estimated by the Wide Range Achievement Test 4 (Wilkinson and Robertson, 2006), Blue Reading subtest; PSI = Processing Speed Index from the WAIS (Wechsler, 2008) (standardized mean = 100, SD = 15); Picture completion = scaled score for subject’s age group on the picture completion subtest of the WAIS (standardized mean = 10, SD = 3); Trail A = Trail Making Test (Reitan and Wolfson, 1985) Part A time to completion (s), completion times > 78 s are considered deficient; Trail B−A = Trail Making Test Part B minus Part A times to completion (s); BDI = Beck Depression Inventory score (Beck et al., 1996); Trait Anxiety = score on Trait Anxiety items from the State-Trait Anxiety Inventory (Spielberger et al., 1983). For group data, means are presented with SD in parentheses.
*Significant difference from normal comparison group (P < 0.05).
BDC = brain-damaged comparison; NC = normal comparison; vmPFC = ventromedial PFC.
Lesion segmentation and image normalization
Ventromedial PFC patients’ lesions were visually identified and manually segmented on a high-resolution (1 mm3) T1-weighted anatomical MRI image. Lesion boundaries were drawn to include areas with evidence of gross tissue damage or abnormal signal characteristics. A T2*-weighted FLAIR anatomical image was used to identify additional damage surrounding the core lesion area not apparent on the T1-weighted image (tissue with signal characteristics differing from healthy grey or white matter, e.g. hyperintensity). All structural MRI data were obtained at least 3 months after surgery. T1-weighted anatomical images were preprocessed with the FreeSurfer image analysis suite (http://www.nmr.mgh.harvard.edu/freesurfer) to remove non-brain tissue, as previously described (Segonne et al., 2004). The resulting skull-stripped anatomical images were diffeomorphically aligned to the Montreal Neurological Institute (MNI) coordinate system using a Symmetric Normalization algorithm (Avants and Gee, 2004) with constrained cost-function masking to prevent warping of tissue within the lesion mask (Brett et al., 2001). A lesion overlap map (Fig. 1) was created by computing the sum of lesion masks for all subjects in MNI space.
Facial emotion recognition task
Stimuli were chosen from the Karolinska Directed Emotional Faces set (Lundqvist et al., 1998). Ten male and 10 female actors, each depicting two emotions out of happiness, sadness, anger, fear, disgust, and neutral, comprised the stimuli for the recognition task. Face stimuli were converted to greyscale, cropped to remove hair and ears, and matched for size and luminance. Before beginning the task, participants were instructed that on each trial a face would appear onscreen for several seconds, during which time they should try to identify the emotion of the face. Trials began with a fixation cross presented for 4 ± 1 s, followed by a 3-s face presentation. Faces were presented such that the tip of the nose appeared at the same point on the screen as the fixation cross. Faces subtended 11.5° visual angle. After viewing the face, participants had unlimited time to use a computer mouse to rate the expression’s valence (‘How positive or negative was that facial expression?’) on a 9-point scale and to identify the emotion from the six possibilities presented. Faces were onscreen during both the valence rating and emotion choice components of the task, to minimize any working memory demands.
Visual attention tasks
All ventromedial PFC and brain-damaged comparison patients completed several tests to ensure intact basic elements of visual processing. These tests included a neurological exam, which tests for gross deficits in visual fields, eye movements, and spatial attention; eye tracker calibration, which requires voluntary eye movements to locations spanning the entire stimulus presentation screen (i.e. each corner, each edge, and centre); and Trails A, which measures visual search and scanning. All ventromedial PFC and brain-damaged comparison patients exhibited normal performance on each of these tests. Additionally, ventromedial PFC patients completed the Wechsler Adult Intelligence Scale-IV (WAIS) Picture Completion, which measures detailed visual perception and recognition (Wechsler, 2008), and WAIS processing speed index (consisting of Coding and Symbol Search subtests), which measures visual perception, scanning speed, and visual working memory. All ventromedial PFC patients exhibited normal performance on these tests (Table 1).
Eye tracking
Participants’ eye movements were tracked at 60 Hz with an ASL D6 desk-mounted eye tracker (Applied Science Laboratories). Participants were seated ∼64 cm away from the monitor. All participants underwent a 9-point calibration before beginning the experimental task. Head tracking software was used to account for head movements in real time. Fixations were defined as gaze coordinates remaining inside 1° visual angle for 100 ms or longer (Lambert et al., 1974; Karsh and Breitenbach, 1983), and identified offline using automated software.
Each face stimulus was divided into three areas of interest for analysis. The vertical bounds of the ‘eye’ area of interest were just superior of the corrugator muscle and the inferior orbit, and the horizontal bounds were the lateral corners of each eye. The vertical bounds of the ‘mouth’ area of interest were the middle of the philtrum and just inferior of the lower lip, and the horizontal bounds were points just beyond the corners of the lips. The ‘face’ area of interest was a rectangle the maximum height and width of the face stimulus.
For all analyses performed on eye tracking data, individual trials were excluded if eye tracking failed for >10% of samples (300 ms) during the face presentation. This threshold was set to reduce the impact of eye tracking artefacts introduced by excessive blinking and head movement. Seventy-nine of 1280 trials (6.2%) were excluded on this basis. Groups did not significantly differ with respect to the total number of trials excluded (P = 0.94). To account for interindividual variability in overall frequency of visual fixations and to facilitate comparison of data to previous lesion patient research (Adolphs et al., 2005), we used proportion of fixations made to a given area of interest (out of the total number fixations during each 3-s face presentation) as our primary dependent measure.
For our main statistical analyses, we performed non-parametric tests because of the small sample size of bilateral ventromedial PFC lesion patients. Specifically, we used a two-tailed Kruskal-Wallis test, followed by between-group comparisons with Mann-Whitney U tests, to test the hypothesis that, compared to the normal comparison and brain-damaged comparison groups, the ventromedial PFC lesion group would exhibit fewer visual fixations to the eye regions of the faces.
Results
Proportion of fixations to the eyes
Overall, there was a significant effect of group on the proportion of fixations to the eye region of faces per trial (X2 = 6.07, P = 0.048). Between-group comparisons indicated that the ventromedial PFC group (mean = 13.03%, SD = 4.42) made a smaller proportion of fixations to the eyes per trial than did the normal comparison group (mean = 28.28%, SD = 15.90; U = 7.00, P = 0.040) or brain-damaged comparison group (mean = 37.92%, SD = 21.23; U = 2.00, P = 0.028; Figs 2 and 3A), whereas there was no significant difference between the normal comparison and brain-damaged comparison groups (U = 71.00, P = 0.27). This finding confirms our main study hypothesis regarding visual fixations.
As a follow-up analysis to this primary result, we examined eye tracking data with respect to individual emotion categories. There was a significant effect of group on the proportion of fixations to the eye region of fear faces (X2 = 6.53, P = 0.038). Between-group comparisons indicated that the ventromedial PFC group (mean = 12.40%, SD = 2.64) made a significantly smaller proportion of fixations to the eyes of faces showing fear than did the normal comparison group (mean = 31.76%, SD = 18.80; U = 7.00, P = 0.040) or brain-damaged comparison group (mean = 42.79%, SD = 22.15; U = 1.50, P = 0.022; Fig. 3A), whereas there was no significant difference between the normal comparison and brain-damaged comparison groups (U = 68.00, P = 0.22). For faces showing disgust and neutral, there were also trends toward group effects on the proportion of fixations made to the eyes (disgust: X2 = 5.89, P = 0.053; neutral: X2 = 5.86, P = 0.053). No significant group effects were detected for happy, sad, or angry faces (all P’s > 0.17).
Proportion of fixation time to the eyes
To confirm that the fixation deficit observed in the ventromedial PFC group was not due to them making fewer, but longer, fixations to the eye region than the comparison groups, we also investigated group differences in the average proportion of total eye fixation time, relative to total fixation time per trial. Consistent with the main fixation analyses, there was a trend towards a group effect of eye fixation time when collapsing across emotion categories (X2 = 5.18, P = 0.075). Between-group comparisons indicate that the ventromedial PFC group (mean = 11.96%, SD = 7.57) spent a smaller proportion of total fixation time looking at the eye region than did the normal comparison group (mean = 26.94%, SD = 16.62; U = 11.00, P = 0.094) or brain-damaged comparison group (mean = 38.02%, SD = 22.30; U = 3.00, P = 0.043), whereas the normal comparison and brain-damaged comparison groups did not significantly differ (U = 67.00, P = 0.20). Also consistent with the main fixation analyses, there was a significant group effect of eye fixation time specific to fear faces (X2 = 6.94, P = 0.031). Between-group comparisons indicate that the ventromedial PFC group (mean = 11.35%, SD = 5.20) spent a smaller proportion of total fixation time looking at the eye region of fear faces than did the normal comparison group (mean = 30.34%, SD = 19.32; U = 10.00, P = 0.077) or brain-damaged comparison group (mean = 43.11%, SD = 23.58; U = 2.00, P = 0.028). There was also a trend towards the normal comparison group spending less time looking at the eyes than the brain-damaged comparison group (U = 57.00, P = 0.081). Thus, the pattern of results with respect to time spent looking at the eye region of the face largely complements the pattern observed when analysing proportion of fixations made to the eye region of the face, and rules out the possibility that ventromedial PFC patients made fewer, but longer, fixations to the eye regions relative to comparison groups.
General measures of visual exploration
To examine whether the deficits observed in the ventromedial PFC group’s attention to the eye region of faces was because of a general lack of visual exploration or eye movement, we tested for group differences on a variety of visual exploration metrics. The groups did not significantly differ with respect to total number of fixations per trial (P = 0.51), nor did the groups significantly differ with respect to proportion of fixations to the mouth per trial (P = 0.94). As an indicator of the total distance of eye movements made during a trial, we summed the distances between consecutive fixations within each trial. This value yields a gross measure of total eye movement during the task. Groups did not differ with respect to total distance between fixations across all trials (P = 0.34) or for fear trials specifically (P = 0.30). To test whether the observed fixation abnormalities were due to the ventromedial PFC patients making multiple fixations near the point of initial fixation (i.e. the nose), we looked for group differences in average distance of participants’ furthest fixations from the nose. This was calculated by using, for each trial, the length of the longest line segment formed between the nose and the coordinates of each fixation and then averaging across trials. No group differences were detected for average distance of the furthest fixation from the nose across all trials (P = 0.25) or within fear trials specifically (P = 0.12). Hence, the fixation deficit in the ventromedial PFC group appeared to be specific to the eye region of the face and not attributable to a more general deficit in generating eye movements or exploring stimuli.
Epoch analysis
To further elucidate the time-course of the fixation deficit observed in the ventromedial PFC group, we repeated the analyses of proportion of fixations to the eye region of faces showing fear after binning each 3-s trial into three 1-s epochs. Within the interval of 0 to 1 s post-stimulus onset, there was a significant group effect with respect to proportion of fixations made to the eyes of fear faces (X2 = 7.69, P = 0.021; Fig. 4). Between-group comparisons indicated that the ventromedial PFC group (mean = 0.00%, SD = 0.00) made a smaller proportion of fixations to the eyes of fear faces within this interval than did the normal comparison group (mean = 28.08%, SD = 20.85; U = 3.00, P = 0.014) or brain-damaged comparison group (mean = 36.67%, SD = 26.49; U = 0.00, P = 0.011), whereas the normal comparison and brain-damaged comparison groups did not significantly differ (U = 74.50, P = 0.35). Within the interval of 1 to 2 s post-stimulus onset, there was no significant group effect with respect to proportion of fixations made to the eyes of fear faces (X2 = 4.49, P = 0.11). Similarly, within the interval of 2–3 s post-stimulus onset, there was no significant group effect with respect to proportion of fixations made to the eyes of fear faces (X2 = 4.17, P = 0.12).
Facial emotion recognition
Although the ventromedial PFC group had lower overall emotion recognition accuracy (mean = 76.67%, SD = 2.89) than the normal comparison group (mean = 85.79%, SD = 8.82) and brain-damaged comparison group (mean = 84.94%, SD = 4.52), these differences were not statistically significant (X2 = 4.23, P = 0.12; Fig. 3B). To determine if the ventromedial PFC group’s fixation deficit to the eye region of fear faces was accompanied by a deficit in fear recognition, we tested for group differences in fear recognition. The ventromedial PFC group had lower fear recognition accuracy (mean = 52.38%, SD = 43.64) than the normal comparison (mean = 77.44%, SD = 22.50) and brain-damaged comparison groups (mean = 69.52%, SD = 20.72); however, these differences were not statistically significant (X2 = 2.02, P = 0.36).
Discussion
Our results show that ventromedial PFC damage impairs visual attention during facial emotion identification. This study is the first to use eye tracking in ventromedial PFC lesion patients to assess visual attention during facial emotion recognition. The importance of this function for adaptive social behaviour has been demonstrated by behavioural studies of autism, which have linked abnormal visual attention to faces (Pelphrey et al., 2002; Dalton et al., 2005; Hernandez et al., 2009) with deficits in recognizing facial expressions of emotion (Jemel et al., 2006; Harms et al., 2010). Studies of human neurological lesion patients have played an integral role in elucidating the neurocircuitry supporting these processes. Building on initial neuropsychological studies that associated amygdala damage with marked deficits in identifying facial expressions of emotion, particularly fear (Adolphs et al., 1994, 1999; Young et al., 1996), a subsequent eye tracking study demonstrated that the root cause of this amygdala-dependent deficit is the failure to attend to the eye region of the face during visual inspection (Adolphs et al., 2005). The present results indicate that ventromedial PFC also plays a critical role in mediating visual attention to the eye region of the face, particularly for fearful expressions. Considering the substantial degree of structural and functional interconnection between ventromedial PFC and amygdala (Barbas, 2000; Ghashghaei and Barbas, 2002; Roy et al., 2009), we propose that these two regions may comprise part of a neural circuit responsible for endogenously controlling visual attention to the eye regions of faces. One possibility is that reciprocal or coincident activity between the ventromedial PFC and amygdala may signal the social-emotional salience of the stimulus, and direct subsequent eye movements accordingly. Human functional MRI data have shown that amygdala activity discriminates between fearful and happy faces, even when the faces are backward-masked and presented for <50 ms (Whalen et al., 1998). In addition, single-neuron recordings from the ventromedial PFC in humans have shown short-latency (<200 ms from stimulus onset) discrimination between fearful and happy faces (Kawasaki et al., 2001). Anatomical data are consistent with this proposed function. Ventromedial PFC and amygdala share robust bidirectional projections with each other; both regions receive projections from high-level visual areas in temporal cortex; and both regions interconnect densely with areas of posterior lateral orbital cortex, which in turn project to the lateral frontal eye fields that control eye movement (Barbas, 2000; Cavada et al., 2000). This proposed early detection/attention-allocation function is consistent with previous neuropsychological data showing that bilateral amygdala damage specifically impairs visual attention to the eye region of faces for the first fixation following stimulus onset (Kennedy and Adolphs, 2010), as well as our own follow-up analyses with ventromedial PFC lesion patients demonstrating the most pronounced deficit of eye fixations during the first second of face viewing. To further explore the putative relationship between ventromedial PFC and amygdala for this function, an important follow-up study in ventromedial PFC patients will be to determine whether some type of exogenous direction of attention to the eye region of the face rescues the observed deficits in emotion recognition, as was the case for a patient with bilateral amygdala lesion (Adolphs et al., 2005). A follow-up study involving gaze manipulation would also serve to more clearly establish whether abnormalities in visual fixations are causally linked to emotion recognition performance in ventromedial PFC lesion patients. Moreover, future studies could determine whether the attention deficit following ventromedial PFC damage is specific to the eye regions of faces, or if it applies more generally to socially or emotionally salient information in visual or other sensory modalities. Another alternative is that this deficit may apply broadly to voluntary shifts in attention, regardless of social or emotional significance (Vecera and Rizzo, 2004). However, our follow-up analyses, which show no significant group differences in total fixations, distance between fixations, or maximum eccentricity of fixation, argue against this possibility.
Although our study is the first to use eye tracking in ventromedial PFC lesion patients to assess visual attention to emotional faces, a number of previous studies have assessed emotion recognition accuracy in this patient population. The findings of these studies have been somewhat mixed, with several reporting no significant overall impairment among patients with ventromedial PFC damage (Hornak et al., 2003; Shamay-Tsoory et al., 2003, 2007; Shaw et al., 2005), but others showing clear deficits (Hornak et al., 1996; Heberlein et al., 2008; Tsuchida and Fellows, 2012). There may be at least three possible reasons for these ostensibly conflicting results. One reason may be the lesion characteristics of the ventromedial PFC patients. Each of the previous studies included patients with unilateral ventromedial PFC damage, potentially allowing for preservation of function by the intact hemisphere. Varying degrees of unilateral versus bilateral damage in the ventromedial PFC patient samples between studies could potentially account for the mixed results. Notably, in one study that specifically examined the performance of the subset of patients with bilateral ventromedial PFC lesions, it was found that three of the five bilateral cases had significant impairment in facial emotion recognition (Hornak et al., 2003). A second reason may be the sensitivity of the recognition test. Two of the studies showing deficits in ventromedial PFC patients used tests requiring the detection of subtle differences in facial expressions of emotion (Heberlein et al., 2008; Tsuchida and Fellows, 2012), rather than categorical identification of more exaggerated expressions (Hornak et al., 1996, 2003; Shamay-Tsoory et al., 2003, 2007). The task that we used here (categorization of exaggerated stereotypical facial expressions of emotion) is well-suited for eye-tracking (Adolphs et al., 2005), but it is not an especially sensitive measure of recognition accuracy, as indicated by the high overall recognition performance (ceiling effect) in all subject groups (Fig. 3B). Moreover, because we only had seven trials for each category of emotion and the response accuracy was dichotomous (correct/incorrect) and near ceiling, we did not have a sufficiently variable or continuous distribution of accuracy scores to perform a valid correlation analysis relating eye-tracking results to recognition accuracy. Future studies with this patient population could use more sensitive emotion recognition tests. We expect that these future studies will yield more pronounced recognition deficits than we observed here. The third reason may be consideration of individual emotions. As amygdala damage has been associated with a selective deficit in recognizing negative emotions, especially fear (Adolphs et al., 1994, 1999), there is preliminary evidence that ventromedial PFC damage may also be particularly associated with deficits in recognizing negative emotions (Heberlein et al., 2008). Our eye tracking data are consistent with this proposal; fixation deficits in the ventromedial PFC group seemed to be greatest for fear faces. However, we should also note that ventromedial PFC lesions were associated with an overall deficit in eye fixations across all emotions, including a trend-level effect for neutral faces, suggesting that the observed deficits in visual attention are related to face viewing during emotion categorization in general, rather than as a response to viewing particular emotional faces per se.
There are several features of the study design that warrant further discussion. First is the limited sample size of ventromedial PFC lesion patients (n = 3). For this study, we used extremely stringent selection criteria for our target group; lesions had to involve substantial portions of the ventromedial PFC bilaterally, but could not extend significantly outside the ventromedial PFC. Limiting the ventromedial PFC lesion patient group to these criteria increases lesion homogeneity and reduces the likelihood of preservation of function by a single hemisphere. This patient selection strategy is distinct from typical ventromedial PFC lesion studies, which often include patients with lesions that are exclusively or primarily unilateral and/or lesions that extend beyond the boundaries of ventromedial PFC (e.g. into adjacent dorsomedial PFC, lateral PFC or anterior temporal lobe). To our knowledge, no previous ventromedial PFC patient study has limited its sample to bilateral, yet selective, ventromedial PFC lesions. By analogy, studies of patients with bilateral, yet selective, amygdala damage have included no more than one or two patients (Adolphs et al., 1994, 2005). Therefore, although our sample size may be small by conventional ventromedial PFC lesion patient standards (which typically feature 5–12 ventromedial PFC lesion patients), it is unique with respect to the uniformity of selective bilateral ventromedial PFC lesions.
To obtain patients with lesions meeting these stringent inclusion criteria, we selected patients who had all undergone surgical resection of large orbital meningiomas. Other lesion aetiologies rarely result in focal bilateral ventromedial PFC lesions. For example, ischaemic strokes typically yield unilateral lesions whereas bilateral lesions resulting from subarachnoid haemorrhage or traumatic brain injury are typically not confined to ventromedial PFC. Although intracranial meningiomas can result in compression of surrounding brain tissue, in each of our patients there was no evidence of chronic distal or diffuse tissue damage following meningioma resection. Neuroradiology reports following surgery indicate no abnormal findings outside the ventromedial PFC, whereas neuropsychological testing (Table 1) reveals normal performance in measures of processing speed and general visuospatial attention (such as WAIS Coding, Symbol Search, Picture Completion, and Trail Making Tests A and B) that are sensitive to diffuse brain injury (Salmond and Sahakian, 2005). We believe the unprecedented uniformity of lesion characteristics in this ventromedial PFC patient sample likely contributes to the remarkable consistency of the individual results. As can be seen in Fig. 3A, the overall proportions of eye fixation for each ventromedial PFC patient were similar to one another, and all were more than one standard error below the mean of each comparison group (normal comparison and brain-damaged comparison groups). This pattern was especially pronounced for fear faces (Fig. 3A), where the ventromedial PFC patients made nearly identical proportions of eye fixations, all of which were more than two standard errors below the comparison group means. Nonetheless, it will be important to replicate the present findings in larger samples of ventromedial PFC lesion patients.
A second feature of our study that warrants further consideration is the suboptimal matching of our comparison groups. Although the ventromedial PFC lesion group included two males and one female, the brain-damaged comparison and normal comparison groups included roughly equal numbers of males and females. To ensure that the observed group differences in visual fixation data were not due to a greater proportion of male subjects in the ventromedial PFC group, we examined whether there were any gender differences in visual fixation data within the normal comparison group (Supplementary Fig. 3). As male and female normal comparison participants exhibited similar proportions of fixations to the eye region [F(1,17) = 0.19, P = 0.67] and emotion recognition accuracy [F(1,17) = 0.06, P = 0.80], we conclude that gender did not likely play a role in the observed group differences. Likewise, although the brain-damaged comparison patients had greater levels of depression and anxiety than the other two groups (Table 1), neither eye movement nor emotion recognition were related to these variables among comparison subjects (all P-values > 0.22; Supplementary Fig. 4), suggesting that differences in depression and anxiety did not drive the observed differences between ventromedial PFC and comparison groups in visual attention patterns and emotion recognition accuracy. Furthermore, lesion size and laterality differed between the ventromedial PFC and brain-damaged comparison groups. Whereas all of the three ventromedial PFC lesion patients had bilateral lesions (Fig. 1), nearly all of the brain-damaged comparison patients had unilateral lesions (Supplementary Fig. 2). Although the ventromedial PFC and brain-damaged comparison patients underwent a similar craniotomy procedure and skull-base neurosurgical approach (thus allowing us to rule out the possibility that the pattern of eye fixations observed in the ventromedial PFC lesion group could be a result of anatomically non-specific effects of brain damage or history of related medical issues), lesion size and laterality were not well-matched between patient groups. To address this issue, we briefly examine here the eye-tracking data of the one brain-damaged comparison patient with a bilateral lesion, which involved dorsomedial frontal cortex. Relative to normal comparison subjects, this brain-damaged comparison patient exhibited an entirely normal pattern of fixations; her overall proportion of eye fixations was 28.23% (slightly greater than the normal comparison group mean of 26.94%), whereas her proportion of eye fixations to fear faces was 41.67% (also greater than the normal comparison group mean of 30.34%). Although limited to only one brain-damaged comparison subject, these data indicate that a bilateral medial brain lesion per se, is not sufficient to yield the pattern of reduced eye fixations observed in the ventromedial PFC lesion group.
Finally, we consider the implications of these findings for models of ventromedial PFC function. The study results broaden the understanding of ventromedial PFC function to include not just higher-order cognitive processes like value computation and emotion regulation, but also the basic attentional process of controlling eye movement to the socially or emotionally salient features of the environment. As ventromedial PFC dysfunction is believed to play a key role in numerous psychiatric disorders (Blair, 2007; Etkin and Wager, 2007; Levy and Glimcher, 2012; Myers-Schulz and Koenigs, 2012), a clearer understanding of the function of this brain area could help elucidate the psychobiological mechanisms underlying various forms of mental illness.
In sum, through a novel application of eye tracking in human lesion patients with bilateral ventromedial PFC damage, we have demonstrated a previously unknown role for this brain area in mediating visual fixations during the recognition of facial expressions of emotion.
Supplementary Material
Acknowledgements
The authors thank Ian Carroll and Maia Pujara for assistance with participant recruitment and data collection.
Glossary
Abbreviation
- PFC
prefrontal cortex
Funding
This work was funded by a grant from NIMH (MH086787), with additional support from NIH grants T32GM008692, T32GM007507, T32MH018931, as well as a fellowship from the NSF (DGE-1256259).
Supplementary material
Supplementary material is available at Brain online.
References
- Adolphs R, Gosselin F, Buchanan TW, Tranel D, Schyns P, Damasio AR. A mechanism for impaired fear recognition after amygdala damage. Nature. 2005;433:68–72. doi: 10.1038/nature03086. [DOI] [PubMed] [Google Scholar]
- Adolphs R, Tranel D, Damasio H, Damasio A. Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala. Nature. 1994;372:669–72. doi: 10.1038/372669a0. [DOI] [PubMed] [Google Scholar]
- Adolphs R, Tranel D, Hamann S, Young AW, Calder AJ, Phelps EA, et al. Recognition of facial emotion in nine individuals with bilateral amygdala damage. Neuropsychologia. 1999;37:1111–7. doi: 10.1016/s0028-3932(99)00039-1. [DOI] [PubMed] [Google Scholar]
- Avants B, Gee JC. Geodesic estimation for large deformation anatomical shape averaging and interpolation. Neuroimage. 2004;23(Suppl 1):S139–50. doi: 10.1016/j.neuroimage.2004.07.010. [DOI] [PubMed] [Google Scholar]
- Barbas H. Connections underlying the synthesis of cognition, memory, and emotion in primate prefrontal cortices. Brain Res Bull. 2000;52:319–30. doi: 10.1016/s0361-9230(99)00245-2. [DOI] [PubMed] [Google Scholar]
- Barrash J, Tranel D, Anderson SW. Acquired personality disturbances associated with bilateral damage to the ventromedial prefrontal region. Dev Neuropsychol. 2000;18:355–81. doi: 10.1207/S1532694205Barrash. [DOI] [PubMed] [Google Scholar]
- Bechara A, Damasio H, Tranel D, Damasio AR. Deciding advantageously before knowing the advantageous strategy. Science. 1997;275:1293–5. doi: 10.1126/science.275.5304.1293. [DOI] [PubMed] [Google Scholar]
- Beck AT, Steer RA, Brown GK. BDI-II, Beck depression inventory: manual. 2nd edn. San Antonio, TX: Psychological Corporation; 1996. [Google Scholar]
- Blair RJ. The amygdala and ventromedial prefrontal cortex in morality and psychopathy. Trends Cogn Sci. 2007;11:387–92. doi: 10.1016/j.tics.2007.07.003. [DOI] [PubMed] [Google Scholar]
- Blumer D, Benson DF. Personality changes with frontal and temporal lesions. In: Benson DF, Blumer D, editors. Psychiatric aspects of neurological disease. New York: Stratton; 1975. pp. 151–70. [Google Scholar]
- Brett M, Leff AP, Rorden C, Ashburner J. Spatial normalization of brain images with focal lesions using cost function masking. Neuroimage. 2001;14:486–500. doi: 10.1006/nimg.2001.0845. [DOI] [PubMed] [Google Scholar]
- Camille N, Coricelli G, Sallet J, Pradat-Diehl P, Duhamel JR, Sirigu A. The involvement of the orbitofrontal cortex in the experience of regret. Science. 2004;304:1167–70. doi: 10.1126/science.1094550. [DOI] [PubMed] [Google Scholar]
- Cavada C, Company T, Tejedor J, Cruz-Rizzolo RJ, Reinoso-Suarez F. The anatomical connections of the macaque monkey orbitofrontal cortex. A review. Cereb Cortex. 2000;10:220–42. doi: 10.1093/cercor/10.3.220. [DOI] [PubMed] [Google Scholar]
- Ciaramelli E, Muccioli M, Ladavas E, di Pellegrino G. Selective deficit in personal moral judgment following damage to ventromedial prefrontal cortex. Soc Cogn Affect Neurosci. 2007;2:84–92. doi: 10.1093/scan/nsm001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dalton KM, Nacewicz BM, Johnstone T, Schaefer HS, Gernsbacher MA, Goldsmith HH, et al. Gaze fixation and the neural circuitry of face processing in autism. Nat Neurosci. 2005;8:519–26. doi: 10.1038/nn1421. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Damasio AR. The somatic marker hypothesis and the possible functions of the prefrontal cortex. Philos Trans R Soc Lond B Biol Sci. 1996;351:1413–20. doi: 10.1098/rstb.1996.0125. [DOI] [PubMed] [Google Scholar]
- Eslinger PJ, Damasio AR. Severe disturbance of higher cognition after bilateral frontal lobe ablation: patient EVR. Neurology. 1985;35:1731–41. doi: 10.1212/wnl.35.12.1731. [DOI] [PubMed] [Google Scholar]
- Etkin A, Wager TD. Functional neuroimaging of anxiety: a meta-analysis of emotional processing in PTSD, social anxiety disorder, and specific phobia. Am J Psychiatry. 2007;164:1476–88. doi: 10.1176/appi.ajp.2007.07030504. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fellows LK. Orbitofrontal contributions to value-based decision making: evidence from humans with frontal lobe damage. Ann N Y Acad Sci. 2011;1239:51–8. doi: 10.1111/j.1749-6632.2011.06229.x. [DOI] [PubMed] [Google Scholar]
- Fellows LK, Farah MJ. Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. Brain. 2003;126(Pt 8):1830–7. doi: 10.1093/brain/awg180. [DOI] [PubMed] [Google Scholar]
- Ghashghaei HT, Barbas H. Pathways for emotion: interactions of prefrontal and anterior temporal pathways in the amygdala of the rhesus monkey. Neuroscience. 2002;115:1261–79. doi: 10.1016/s0306-4522(02)00446-3. [DOI] [PubMed] [Google Scholar]
- Harlow JM. Recovery from the passage of an iron bar through the head. Publ Mass Med Soc. 1868;2:327–47. [Google Scholar]
- Harms MB, Martin A, Wallace GL. Facial emotion recognition in autism spectrum disorders: a review of behavioral and neuroimaging studies. Neuropsychol Rev. 2010;20:290–322. doi: 10.1007/s11065-010-9138-6. [DOI] [PubMed] [Google Scholar]
- Heberlein AS, Padon AA, Gillihan SJ, Farah MJ, Fellows LK. Ventromedial frontal lobe plays a critical role in facial emotion recognition. J Cogn Neurosci. 2008;20:721–33. doi: 10.1162/jocn.2008.20049. [DOI] [PubMed] [Google Scholar]
- Henri-Bhargava A, Simioni A, Fellows LK. Ventromedial frontal lobe damage disrupts the accuracy, but not the speed, of value-based preference judgments. Neuropsychologia. 2012;50:1536–42. doi: 10.1016/j.neuropsychologia.2012.03.006. [DOI] [PubMed] [Google Scholar]
- Hernandez N, Metzger A, Magne R, Bonnet-Brilhault F, Roux S, Barthelemy C, et al. Exploration of core features of a human face by healthy and autistic adults analyzed by visual scanning. Neuropsychologia. 2009;47:1004–12. doi: 10.1016/j.neuropsychologia.2008.10.023. [DOI] [PubMed] [Google Scholar]
- Hornak J, Bramham J, Rolls ET, Morris RG, O'Doherty J, Bullock PR, et al. Changes in emotion after circumscribed surgical lesions of the orbitofrontal and cingulate cortices. Brain. 2003;126 (Pt 7):1691–712. doi: 10.1093/brain/awg168. [DOI] [PubMed] [Google Scholar]
- Hornak J, Rolls ET, Wade D. Face and voice expression identification in patients with emotional and behavioural changes following ventral frontal lobe damage. Neuropsychologia. 1996;34:247–61. doi: 10.1016/0028-3932(95)00106-9. [DOI] [PubMed] [Google Scholar]
- Jemel B, Mottron L, Dawson M. Impaired face processing in autism: fact or artifact? JAutism Dev Disor. 2006;36:91–106. doi: 10.1007/s10803-005-0050-5. [DOI] [PubMed] [Google Scholar]
- Karsh R, Breitenbach FW. Looking at looking: the amorphous fixation measure. In: Groner R, Menz C, Fisher DF, Monty RA, editors. Eye movements and psychological functions: International views. Hillsdale, NJ: Lawrence Erlbaum Associates; 1983. [Google Scholar]
- Kawasaki H, Kaufman O, Damasio H, Damasio AR, Granner M, Bakken H, et al. Single-neuron responses to emotional visual stimuli recorded in human ventral prefrontal cortex. Nat Neurosci. 2001;4:15–6. doi: 10.1038/82850. [DOI] [PubMed] [Google Scholar]
- Kennedy DP, Adolphs R. Impaired fixation to eyes following amygdala damage arises from abnormal bottom-up attention. Neuropsychologia. 2010;48:3392–8. doi: 10.1016/j.neuropsychologia.2010.06.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Koenigs M, Tranel D. Irrational economic decision-making after ventromedial prefrontal damage: evidence from the Ultimatum Game. J Neurosci. 2007;27:951–6. doi: 10.1523/JNEUROSCI.4606-06.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Koenigs M, Young L, Adolphs R, Tranel D, Cushman F, Hauser M, et al. Damage to the prefrontal cortex increases utilitarian moral judgements. Nature. 2007;446:908–11. doi: 10.1038/nature05631. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Krajbich I, Adolphs R, Tranel D, Denburg NL, Camerer CF. Economic games quantify diminished sense of guilt in patients with damage to the prefrontal cortex. J Neurosci. 2009;29:2188–92. doi: 10.1523/JNEUROSCI.5086-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lambert RH, Monty RA, Hall RJ. High-speed data processing and unobtrusive monitoring of eye movements. Behav Res Methods Instrum. 1974;6:525–30. [Google Scholar]
- Levy DJ, Glimcher PW. The root of all value: a neural common currency for choice. Curr Opin Neurobiol. 2012;22:1027–38. doi: 10.1016/j.conb.2012.06.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lundqvist D, Flykt A, Öhman A. Psychology section, Karolinska Institutet; 1998. The Karolinska Directed Emotional Faces—KDEF [CD ROM]: CD ROM from Department of Clinical Neuroscience. [Google Scholar]
- Myers-Schulz B, Koenigs M. Functional anatomy of ventromedial prefrontal cortex: implications for mood and anxiety disorders. Mol Psychiatry. 2012;17:132–41. doi: 10.1038/mp.2011.88. [DOI] [PMC free article] [PubMed] [Google Scholar]
- O'Doherty JP. Contributions of the ventromedial prefrontal cortex to goal-directed action selection. Ann N Y Acad Sci. 2011;1239:118–29. doi: 10.1111/j.1749-6632.2011.06290.x. [DOI] [PubMed] [Google Scholar]
- Pelphrey KA, Sasson NJ, Reznick JS, Paul G, Goldman BD, Piven J. Visual scanning of faces in autism. J Autism Dev Disor. 2002;32:249–61. doi: 10.1023/a:1016374617369. [DOI] [PubMed] [Google Scholar]
- Pessoa L, Adolphs R. Emotion processing and the amygdala: from a ‘low road’ to ‘many roads’ of evaluating biological significance. Nat Rev Neurosci. 2010;11:773–83. doi: 10.1038/nrn2920. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Reitan RM, Wolfson D. The Halstead-Reitan neuropsychological test battery: theory and clinical interpretation. Tucson, AZ: Neuropsychology Press; 1985. [Google Scholar]
- Roy AK, Shehzad Z, Margulies DS, Kelly AM, Uddin LQ, Gotimer K, et al. Functional connectivity of the human amygdala using resting state fMRI. Neuroimage. 2009;45:614–26. doi: 10.1016/j.neuroimage.2008.11.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Roy M, Shohamy D, Wager TD. Ventromedial prefrontal-subcortical systems and the generation of affective meaning. Trends Cogn Sci. 2012;16:147–56. doi: 10.1016/j.tics.2012.01.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Salmond CH, Sahakian BJ. Cognitive outcome in traumatic brain injury survivors. Curr Opin Crit Care. 2005;11:111–6. doi: 10.1097/01.ccx.0000155358.31983.37. [DOI] [PubMed] [Google Scholar]
- Segonne F, Dale AM, Busa E, Glessner M, Salat D, Hahn HK, et al. A hybrid approach to the skull stripping problem in MRI. Neuroimage. 2004;22:1060–75. doi: 10.1016/j.neuroimage.2004.03.032. [DOI] [PubMed] [Google Scholar]
- Shamay-Tsoory SG, Tibi-Elhanany Y, Aharon-Peretz J. The green-eyed monster and malicious joy: the neuroanatomical bases of envy and gloating (schadenfreude) Brain. 2007;130(Pt 6):1663–78. doi: 10.1093/brain/awm093. [DOI] [PubMed] [Google Scholar]
- Shamay-Tsoory SG, Tomer R, Berger BD, Aharon-Peretz J. Characterization of empathy deficits following prefrontal brain damage: the role of the right ventromedial prefrontal cortex. J Cogn Neurosci. 2003;15:324–37. doi: 10.1162/089892903321593063. [DOI] [PubMed] [Google Scholar]
- Shaw P, Bramham J, Lawrence EJ, Morris R, Baron-Cohen S, David AS. Differential effects of lesions of the amygdala and prefrontal cortex on recognizing facial expressions of complex emotions. J Cogn Neurosci. 2005;17:1410–9. doi: 10.1162/0898929054985491. [DOI] [PubMed] [Google Scholar]
- Spielberger CD, Gorsuch RL, Lushene R, Vagg PR, Jacobs GA. Manual for the state-trait anxiety inventory. Palo Alto, CA: Consulting Psychologists Press; 1983. [Google Scholar]
- Tsuchida A, Fellows LK. Are you upset? Distinct roles for orbitofrontal and lateral prefrontal cortex in detecting and distinguishing facial expressions of emotion. Cereb Cortex. 2012;22:2904–12. doi: 10.1093/cercor/bhr370. [DOI] [PubMed] [Google Scholar]
- Vecera SP, Rizzo M. What are you looking at? Impaired ‘social attention’ following frontal-lobe damage. Neuropsychologia. 2004;42:1657–65. doi: 10.1016/j.neuropsychologia.2004.04.009. [DOI] [PubMed] [Google Scholar]
- Wechsler D. Wechsler adult intelligence scale. 4th edn. San Antonio, TX: Pearson; 2008. [Google Scholar]
- Whalen PJ, Rauch SL, Etcoff NL, McInerney SC, Lee MB, Jenike MA. Masked presentations of emotional facial expressions modulate amygdala activity without explicit knowledge. J Neurosci. 1998;18:411–8. doi: 10.1523/JNEUROSCI.18-01-00411.1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wheeler EZ, Fellows LK. The human ventromedial frontal lobe is critical for learning from negative feedback. Brain. 2008;131(Pt 5):1323–31. doi: 10.1093/brain/awn041. [DOI] [PubMed] [Google Scholar]
- Wilkinson GS, Robertson GJ. WRAT4: wide range achievement test. Lutz, FL: Psychological Assessment Resources; 2006. [Google Scholar]
- Young AW, Hellawell DJ, Van De Wal C, Johnson M. Facial expression processing after amygdalotomy. Neuropsychologia. 1996;34:31–9. doi: 10.1016/0028-3932(95)00062-3. [DOI] [PubMed] [Google Scholar]
- Young L, Bechara A, Tranel D, Damasio H, Hauser M, Damasio A. Damage to ventromedial prefrontal cortex impairs judgment of harmful intent. Neuron. 2010;65:845–51. doi: 10.1016/j.neuron.2010.03.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.