Brain Mechanisms for Inferring Deceit in the Actions of Others

Abstract

During social interactions, it is important to judge accurately whether a person is honest or deceitful. We often use nonverbal cues to infer whether others are trying to deceive us. Using functional magnetic resonance imaging, we studied subjects watching videos of actors lifting a box and judged whether or not the actors were trying to deceive them concerning the real weight of the box. When the subjects judged the actions as reflecting deceptive intention, there was activation of the amygdala and rostral anterior cingulate cortex. These areas were not activated when subjects made judgements about the beliefs rather than the intentions of others. We suggest that these activations reflect the observers' judgements of social intentions toward themselves and might reflect an emotional response to being deceived.

Introduction

Social exchange is a critical component of everyday life. It depends on an ability to predict and infer the mental states of others such as their desires, beliefs, and intentions. In social and economic interactions, it is important to judge accurately whether a person is honest or deceitful. It has been repeatedly shown that it is not easy to detect deception (DePaulo et al., 1980; Ekman and O'Sullivan, 1991; Vrij et al., 2000) and that people have a bias in judging the behavior of other people as being truthful (Levine et al., 1999; Vrij, 2000).

Nonetheless, it is possible to infer deception from nonverbal behavior (Bond et al., 1992; Frank and Ekman, 1997; Vrij et al., 2000), and indeed the people who are best at detecting deceit are those who attend more to nonverbal rather than verbal cues (Ekman and O'Sullivan, 1991). These cues can be subtle. When presented with dynamic point-light displays, that is kinematic patterns, people can judge not only the type of actions performed (Dittrich, 1993) and the actor's emotion (Dittrich et al., 1996) and expectation but also whether there is an intention to deceive (Runeson and Frykholm, 1983).

The present study explores the brain mechanisms involved in detecting deception when observing the nonverbal dynamic behavior of actors. We scanned subjects using event-related functional magnetic resonance imaging (fMRI) while they viewed actions performed on videos. The videos showed actors picking up boxes. When preparing these videos, in all trials, the actors were correctly informed of the weight of the box, but in some of them, the actors were instructed to pretend that the box had a different weight, so as to create trials in which the actors had deceptive intentions. We required the subjects to judge whether or not the actors were trying to deceive concerning the weight of the box. By comparing these trial types, we aimed to identify the neural circuits that are involved in the detection of deceptive intention from the observation of nonverbal behavior. To determine whether the activations were specific to detecting deception, we compared the results with those of a previous study (Grèzes et al., 2004). In that study, we also presented videos of actors picking up a box, which varied in weights from trial to trial. The difference was that the subjects were asked to make judgements about the actor's expectation of the weight. Thus, we showed similar videos in both studies and in both studies, the subjects made judgment about mental states. However, only in the present study did the observers make judgments about the intention of the actor to deceive observers.

Materials and Methods

Subjects. Eleven right-handed subjects (four men and six women, ranging 20-35 years of age) with no neurological or psychiatric history participated in the imaging study. All gave informed consent according to procedures approved by the Joint Ethics Committee of the National Hospital for Neurology and Neurosurgery (University College of London Hospital National Health Service Trust) and Institute of Neurology (University College of London).

Stimuli. Eight actors (four men and four women) were videotaped over 50 trials of lifting and carrying a box of varying weights (1, 6, 12, or 18 kg). The same box was used in all conditions. Each recorded act began with the actor off-screen to the left and the box placed in the middle of the scene and ∼1 m from the right edge of the table. The actor entered from the left, lifted the box by its side handles, walked toward the table, placed the box on the table, and went out of the room. The entire act usually lasted 6-9 sec. During the first part of filming session, the experimenter gave the exact weight of the box to the actor. In the second part of the filming session, the experimenter gave the exact weight of the box to the actor who was instructed to pretend that the box had a different weight. In this way, we created videos in which the actors had deceptive intentions. Eight actors were used to assess variability in observer performance because of individual difference in actor styles of lifting and carrying. We edited video recordings of 3.5 sec concentrating on the lifting phase. Using the After-effect software, we blurred the face of the actors on each movie to hide facial and emotional expression, so that only bodily information was provided.

The stimuli used during the fMRI experiment consisted of color video recordings of the actors performing the action of lifting and carrying a box of different weights. The videos were viewed by projection onto a mirror mounted onto the head-coil in the scanner. Before starting the experiment, subjects were told that in some of the videos the actors were pretending that the box was heavier or lighter than it was. In the scanner, the subjects were required to observe the videos (3.5 sec each) and to decide (forced choice) whether or not the actor was trying to deceive the observer about the weight of the box. The subjects made their responses with a key-press with their right finger. The responses were given at the end of each video during the presentation for 2.5 sec of a screen on which were displayed the words “Real” and “Faked.” The location of the two words (on the right and left parts of the screen) was randomized (Fig. 1).

Figure 1.

The subjects watched videos of actors lifting a box and were instructed to judge whether or not the actor was trying to deceive the observer about the weight of the box. Each trial corresponds to the presentation of a color movie of 3.5 sec followed by a screen of 1.5 sec on which the words Real and Faked were displayed. The position on the screen of the two words was randomized across trials to avoid motor preparation during the observation phase. The subjects were required to decide (forced choice) whether the actor had a deceptive intention by making key-press with the right finger.

We purposely delayed the responses to ensure that the subjects responded at the same time at the end of the observation period. Furthermore, by varying the left-right position of the response cues (Real and Faked), we deliberately prevented the subjects from preparing their responses during the observation period. As a result, our design was not appropriate for detecting any reaction time advantage in making judgements about deceptive intentions. The study was performed in a single continuous scanning session. One hundred and twenty-eight video films intermixed with 72 null events were presented in random order. Fifty percent of the videos involved deception.

MRI scanning. A 2 T VISION system (Siemens, Erlangen, Germany) was used to acquire 36 T₂ ^*-weighted transverse echoplanar (EPI) images [matrix size = 64 × 64; in-plane resolution of 3 × 3 mm²; echo time (TE) = 40 msec] with blood oxygenation level-dependent (BOLD) contrast. EPIs comprised 2.4 mm thick axial slice taken every 3.6 mm, acquired sequentially in a descending direction and continuously during a 22 min session. An automatic shimming procedure was performed before each session. A total of 483 functional volumes was collected for each subject within the single scanning session with an effective repetition time (TR) of 2.74 sec/volume. The first five volumes were discarded to allow for T1 equilibration effects. T1 anatomical volume images (magnetization prepared rapid gradient echo sequence with a high-resolution isovoxel acquisition of 1 × 1 × 1 mm³) were collected for each subject.

Data analysis. We used SPM99 software (Wellcome Department of Imaging Neuroscience, London, UK) for image processing and analysis. The first five volumes were discarded to allow for T1 equilibration effects. The remaining 380 image volumes were realigned to the first volume by rigid body transformation, sinc interpolated over time to correct for phase advance during acquisition, normalized to the Montreal Neurological Institute (MNI) reference brain, and spatially smoothed by a Gaussian kernel of full width half maximum 8 mm (Ashburner and Friston, 1997). Statistical parametric maps of t-statistics were calculated for condition-specific effects within a general linear model.

The effects of interest were defined as follows: two represented the trials where subjects perceived real (genuine) actions [correct (RC) and incorrect (RI) responses] and two represented the trials where subjects perceived faked actions [correct (FC) and incorrect (FI) responses]. The BOLD response to the stimulus onset for each event type was modeled by a box-car waveform of 6.5 sec (2.38 TR) convolved with the hemodynamic response function and its temporal derivative to account for any temporal shifts in the response of the stimuli (Friston et al., 1998). Also included for each session were six covariates to capture residual movement-related artifacts (the three rigid-body translations and the three rotations determined from initial registration) and a single covariate representing the mean (constant) over scans. The data were high-pass filtered with a frequency cutoff at 140 sec.

We performed a random effects analysis. Images of parameter estimates for each contrast of interest were created for each subject (first-level analysis). Two contrasts of interest were calculated: (1) main effect of perceiving faked versus real actions, independently of the responses of the subjects, [(FC + FI) - (RC + RI)], where F is films of faked action, R is films of real actions, C is correct, and I is incorrect responses; (2) main effect of judging an action as reflecting a deceptive intention versus judging an action as being real [(FC + RI) - (RC + FI)], where F is films of faked action, R is films of real actions, C is correct, and I is incorrect responses. These contrasts were then entered into a one-sample t tests in which subjects were treated as a random variable and subsequently transformed into statistical parametric maps (SPMs) of the Z statistic. The statistical parametric maps were thresholded at p < 0.001 (uncorrected for multiple comparisons). These maps were overlaid on the MNI template and on the normalized structural images of each subject and labeled using the atlas of Duvernoy (1999).

To see whether the activations were specific to detecting deception, we compared the results with those of a previous study (Grèzes et al., 2004) in which different subjects (n = 6) looked at videos of actors picking up a box that varied in weight from trial to trial and were asked to make judgements about the actor's expectation of the weight. When making the videos, the expectation of the actor was manipulated: the experimenter told the actor the weight of the box; in the majority of trials, the experimenter gave the correct weight but on a minority of trials an incorrect weight. The observer watched an equal number of trials in which the actor had a correct or false expectation. The trials were divided on the basis of the judgment made by the subjects, that is, the trials on which the subjects judged the actor to have a correct expectation and those on which the subjects judged the actor to have a false expectation.

A random effects analysis was performed over the two studies. Images of parameter estimates for each contrast of interest were created for each subject and for each study and were then entered into a second-level analysis using a one-way ANOVA with the two hemodynamic response functions comprising a factor. The contrast of interest for the present study consisted of the comparison between actions that were judged as reflecting deceptive intention versus trials that were judged as being real. The contrast of interest for the previous study consisted of the comparison between actions that were judged as reflecting false expectation versus those judged as reflecting correct expectation. SPMs of the T-statistic were constructed using a generalized Greenhouse-Geiser correction.

To reveal the common network associated with both judgements, we performed a conjunction analysis (thresholded at p < 0.001, uncorrected for multiple comparisons). For brain regions found to be activated in the present study at p < 0.001, but that did not appear in the conjunction analysis, a two-tailed t test was performed between the results of the two studies.

Results

Behavioral analysis

The subjects were able to detect deception above a chance level. The mean judgements per subject were converted into a measure of sensitivity [d'] independent of any response bias (Macmillan and Creelman, 1991), and on this measure, the subjects were significantly sensitive to deception (d' = 0.89; t test; p < 0.0001). The subjects also tended to judge actions more often as being real (mean correct, 73%) than faked (mean correct, 60%) (two-sample t test; p = 0.006).

We also tested for the main effects on accuracy of action type (Real, Faked) and difference in weight between the real and pretended one (6, 12, and 18 kg). There was a significant effect of action type (p < 0.01). The effect of weight did not reach significance (p = 0.083), and there was no significant interaction between action type and weight difference (p = 0.297).

Neuroimaging results

We first analyzed the data to detect differences between presentations of the videos showing real or faked actions. This analysis will highlight activations in part because of kinematic differences between the actions shown in these two sets of videos. The main effect of perceiving faked versus real actions, independently of the subject's judgements, was calculated as: (FC + FI) - (RC + RI), where F is films of faked action, R is films of real actions, C is correct, and I is incorrect rating. This analysis showed right hemispheric activations located to the superior temporal sulcus (STS) and temporoparietal junction (Fig. 2a,b). The full list of activations is given in Table 1.

Figure 2.

Regions showing amplitude difference when subjects observed actions that were actually faked compared with actions that were actually real. a, Group (n = 11) average activation of the right temporoparietal junction superimposed on a sagital section of the MNI brain. b, Fitted responses at the local maxima in the right temporoparietal junction showing amplitude difference between the perception of faked actions (correct and error responses) versus real actions (correct and error responses).

Table 1.

Brain regions showing amplitude effects for the observation of faked actions

	Talairach coordinates
Brain regions	x	y	z	Z score	Cluster
R medial frontopolar gyrus	14	66	24	4.42	15
R temporoparietal junction	64	−44	22	3.77	117
R inferior frontal gyrus	54	14	8	3.41	14
R superior temporal sulcus	50	−32	−6	3.97	28

p< 0.001, noncorrected. R, Right.

We next analyzed the data according to the judgements made by the subjects. The main effect of judging an action as reflecting a deceptive intention versus judging an action as being real was calculated as: (FC + RI) - (RC + FI), where F is films of faked action, r is films of real actions, C is correct, and I is incorrect rating. Thus, both FC and RI were judged to be faked. This analysis highlighted activations in the amygdala (Fig. 3b), rostral anterior cingulate cortex (Fig. 3a), and superior temporal sulcus (Fig. 3c) as well as lateral orbitofrontal cortex (Fig. 3c,d). The full list of activations is given in Table 2.

Figure 3.

Regions showing amplitude difference when subjects judged the actions to reflect deceptive intention.a, Group (n=11) average activation of the rostral anterior cingulatesulcus superimposed on a sagital section of the MNI brain. b, Group (n = 11) average activations of bilateral amygdala superimposed on a horizontal section of the MNI brain. c, Group (n = 11) average activation of the orbitofrontal cortex and of the superior temporal sulcus in the right hemisphere superimposed on a sagital section and of the orbitofrontal cortex bilaterally on a horizontal section of the MNI brain. d, Fitted responses at the local maxima in the right orbitofrontal cortex showing amplitude difference when subjects judged the actions to reflect deceptive intention (Faked Correct and Real Error) compared with when subjects judged the actions to reflect no deceptive intention (Real Correct and Faked Error).

Table 2.

Brain regions showing amplitude effects when subjects judged the actions to reflect deceptive intention

	Talairach coordinates
Brain regions	x	y	z	Z score	Cluster
R Superior temporal sulcus/ascending posterior segment	52	−56	34	3.79	14
L Superior temporal sulcus, ascending posterior segment	−42	−60	30	3.75	16
Anterior cingulate sulcus, rostral cingulate zone, anterior part	−8	42	20	3.83	45
R inferior frontal gyrus, pars triangularis	52	32	6	3.88	20
R lateral orbital gyrus	50	46	−6	3.82	81
L lateral orbital gyrus	−46	34	−10	3.49	28
L amygdala	−26	−4	−20	3.37	6
R amygdala	26	0	−22	3.64	14
L lateral occipitotemporal sulcus/fusiform gyrus	−46	−50	−22	4.11	33
L cerebellum	−24	−80	−24	3.92	33
L anterior inferior temporal sulcus	−42	8	−34	3.87	16

p < 0.001 non-corrected. L, Left; R, right.

Finally, we performed an analysis to determine which activations were specific to the judgment of intentions and deceit. As explained previously, we implemented this by statistically comparing the above results with those from a previous study (Grèzes et al., 2004) in which the observers made judgements about whether the actors in the videos had a correct or incorrect expectation as to the weight of the box they were going to pick up.

We first performed a conjunction analysis across the two studies to identify the areas that were activated in common to the two studies. This conjunction between actions that were judged as reflecting deceptive intention and actions that were judged to reflect false expectation revealed common activations bilaterally in the superior temporal sulcus (Fig. 4a), lateral orbitofrontal cortex (Fig. 4b), dorsomedial frontal cortex, and cerebellum. There was no activity in the conjunction analysis in the amygdala and anterior cingulate cortex. Next, to determine whether the amygdala and the anterior cingulate cortex were activated to a greater extent during the detection of deceptive intention compared with the detection of false beliefs, we performed a two-tailed t test between the data for the two studies. The anterior cingulate cortex (Fig. 4c) and the left amygdala (Fig. 4d) were only activated for detecting deception but not for detecting false expectation (p < 0.005). This was also true for the right amygdala at a lower significant level (p < 0.02).

Figure 4.

Commonalities and differences between inferring false belief and inferring deceptive intention from other people's actions. a, Left, Histograms representing the percentage of signal change at the local maxima in the right superior temporal sulcus across studies; the first histogram corresponds to false belief, the second to deceptive intention. Right, Group average activation of right superior temporal sulcus superimposed on a sagital section of the MNI brain. b, Left, Histograms representing the percentage of signal change at the local maxima in the right orbitofrontal cortex across studies; other conventions as in a. Right, Group average activation of right orbitofrontal cortex superimposed on a sagital section of the MNI brain. c, Left, Histograms representing the percentage of signal change at the local maxima in the rostral anterior cingulate cortex across studies; other conventions as in a; ^*p < 0.005. Right, Group average activation of the rostral anterior cingulate cortex superimposed on a sagital section of the MNI brain. d, Left, Histograms representing the percentage of signal change at the local maxima in the anterior left amygdala across studies; other conventions as in a; ^*p < 0.005. Right, Group average activation of the left amygdala superimposed on a coronal section of the MNI brain.

Discussion

Detecting deception

The aim of the present study was to identify the neural events that mediate people's perception that they are being deceived. Our critical finding was enhanced activity in the amygdala and anterior cingulate cortex when subjects judged that the actions of others reflected deceptive intentions. These activations cannot simply be taken to reflect judgements about mental states in general because they were specific to the situation in which the observers judged the intention of actors to deceive the observer. Moreover, these activations cannot be explained as simply reflecting kinematic differences between the actions in the videos because they were found only when the data were analyzed in terms of the judgements made by the observer and not when the data were analyzed in terms of the presentation of videos that actually depicted faked rather than real actions.

When subjects judged that they detected deception, two other areas were also activated, the STS and the orbitofrontal cortex. These areas, together with the cingulate cortex, are all interconnected with the amygdala (Stefanacci and Amaral, 2002). Thus, they form a network of areas that are activated in relation to judgment of deception. One may speculate that the visual input to this system when subjects view biological actions comes from the STS.

The amygdala is a crucial part of a neural circuitry by which stimuli trigger emotional responses that reflect an appraisal of value (Adolphs, 1999; Dolan, 2002). This is true whether the stimuli are social (Aggleton and Passingham, 1981; Meunier et al., 1999) or not social (LeDoux, 2000). The emotional responses serve to modulate and bias behavior, depending on the valence of the perceived stimuli (Dolan, 2002). In the present study, the judgements were of social deception. The amygdala has also been shown to be activated when subjects view static faces and judge whether they are trustworthy (Winston et al., 2002). Critically, lesions of the amygdala cause patients to be more likely than control subjects to judge that faces are trustworthy (Adolphs et al., 1998). The assumption is that in these patients, untrustworthy faces are less likely to trigger an emotional response. What is different in the present experiment is that the observers were specifically required to judge intention. In a recent fMRI study, subjects were scanned while they read stories in which they or someone else either intentionally or accidentally violated social norms (Berthoz et al., 2003). Amygdala activation was only seen when the story highlighted intentional actions and not mere embarrassment by virtue of actions that were accidental or involved others violating social norms. In the present study, amygdala activation occurred only when subjects judged intentional actions (deception) and not when they judged the actions to be accidental in the sense that the actor had the wrong expectation about the weight of the box. To decide whether the amygdala activation reflects an emotional response or judging social intention, one would need to record psychophysiological measures.

The rostral anterior cingulate cortex showed the same pattern of activation as the amygdala. Lesions of this area in monkeys do not affect emotional responses to threatening stimuli in the way amygdala lesions do (Hadland et al., 2003); however, the animals do show changes in their social interactions. Human subjects have been scanned while they viewed scenes in which there were moral violations (Moll et al., 2002). There was activation in the anterior cingulate cortex only when they viewed these moral violations and not when the scenes were simply unpleasant. Similarly, this area has been reported to be activated when subjects considered stories in which they transgressed social norms (Berthoz et al., 2002, 2003). The activation was much more pronounced when the transgression was deliberate. As with the amygdale, one of the critical factors appears to be the intention. Another factor may be personal involvement. The anterior cingulate was found to be activated both when subjects viewed faces that were making eye contact and when they heard their own name (Kampe et al., 2003). In both cases, there was an intention to communicate with the subject. In our experiment, the intentions were directed toward the observer. Additional experiments are needed to determine whether they are especially engaged when detecting deceit directed at oneself rather than directed at another.

Perceived changes in kinematics

In our experiment, the subjects' judgements about deception were based on the nonverbal behavior of the actors. It has been suggested that judging deception is a two-step process. The first consists of recognizing cues and signs in behavior that violate the observer's predictions. The second consists of drawing inferences about intention on this basis (Bond et al., 1992; Frank and Ekman, 1997).

Our first analysis divided the videos according to whether or not the actor was trying to deceive. These trials differ in the observed kinematics. Activity in the right STS and temporoparietal junction was found when we contrasted trials in which the actor was actually deceiving the observer and trials in which the actor was not. Several neuroimaging studies have shown that the superior temporal sulcus is involved in the perception of biological motion, such as hand, mouth, and eye movements (for review, see Allison et al., 2000). From Figure 2, it can be seen that the region of the temporoparietal junction was activated in all conditions that involved body actions compared with null events.

The activation was greater for the observation of faked compared with real actions, regardless of whether or not the subjects correctly judged them to be faked (Fig. 2). When the actor tried to deceive an observer about the weight of a box, for example by lifting it particularly slowly, postural readjustments occurred, and the amount of readjustment presumably varied as a function of the difference between the faked and actual weight. The STS region has been shown to play a role in processing behaviorally relevant stimuli (Allison et al., 2000; Corbetta and Shulman, 2002). Therefore, we suggest that this region was especially activated when subjects viewed such readjustments.

Judgements

When the subjects inferred deceptive intention from the actions of the actor, there was activity in the superior temporal sulcus, lateral orbitofrontal cortex, and cerebellum. These areas were also found in our previous experiment in which subjects viewed similar videos and had to decide whether the actor had a correct or a false belief about the weight of the box (Grèzes et al., 2004). The inference that the actor was trying to deceive was the difference between a prediction made by the observer and the action as perceived. It is known that people have a bias to judge other people's behavior as truthful (Levine et al., 1999; Vrij, 2000) and of attributing true beliefs to others (Bartsch and Wellman, 1995). These tendencies influence the predictions made by the observer. In the present study, this bias was observed, and the subjects were more likely to judge actions as truthful.

Thus, the activations listed above may relate to the violation of the subject's prediction. The first activation was in the superior temporal sulcus. A recent fMRI study on the influence of context on perceived gaze shift demonstrated that when an observer's prediction was violated, the activity in the STS was more prolonged compared with the situation in which the observer's predictions were met (Pelphrey et al., 2003). The second activation was in the cerebellum, and this has been shown to play a key role in signaling sensory discrepancy between predicted and actual consequences of movements (Blakemore et al., 2001). The third activation was in the orbital frontal cortex. Activity in this area, as well as in the temporoparietal junction, has been reported when subjects perceived a mismatch between what they expected and what actually happened (Downar et al., 2001, 2002). The activity may either reflect alerting (Downar et al., 2002) or the change in ongoing cognitive activity (Corbetta and Shulman, 2002). We conclude that activations in the superior temporal sulcus, cerebellum, and orbitofrontal cortex are best explained as reflecting the violation of the predictions made by the observer. When predictions are violated, the observer must update their representation of the mental state of the actor.

Conclusion

In our experiment, subjects judged deceptive intention over so simple a matter as the weight of a box. They did so on the basis of nonverbal behavior under conditions in which they could not rely on facial expression. When subjects inferred deceptive intention, there was activity in the amygdala. In rats (LeDoux, 2000) and monkeys (Aggleton and Passingham, 1981; Meunier et al., 1999; Baxter and Murray, 2002), the amygdala is involved in the process through which stimuli acquire emotional significance. These associations are of biological value, because they are crucial for survival. Human society particularly depends on social and economic interactions, and this depends on the ability to tell who can be trusted. We show that the role of the system for learning the emotional significance of stimuli has extended to judgements about social intentions.