Skip to main content
Frontiers in Behavioral Neuroscience logoLink to Frontiers in Behavioral Neuroscience
. 2016 Feb 24;10:25. doi: 10.3389/fnbeh.2016.00025

EMOTICOM: A Neuropsychological Test Battery to Evaluate Emotion, Motivation, Impulsivity, and Social Cognition

Amy R Bland 1, Jonathan P Roiser 2, Mitul A Mehta 3, Thea Schei 4, Heather Boland 1, Daniel K Campbell-Meiklejohn 5, Richard A Emsley 6, Marcus R Munafo 7, Ian S Penton-Voak 7, Ana Seara-Cardoso 8,9, Essi Viding 8, Valerie Voon 10,11, Barbara J Sahakian 10,11, Trevor W Robbins 4,11, Rebecca Elliott 1,*
PMCID: PMC4764711  PMID: 26941628

Abstract

In mental health practice, both pharmacological and non-pharmacological treatments are aimed at improving neuropsychological symptoms, including cognitive and emotional impairments. However, at present there is no established neuropsychological test battery that comprehensively covers multiple affective domains relevant in a range of disorders. Our objective was to generate a standardized test battery, comprised of existing, adapted and novel tasks, to assess four core domains of affective cognition (emotion processing, motivation, impulsivity and social cognition) in order to facilitate and enhance treatment development and evaluation in a broad range of neuropsychiatric disorders. The battery was administered to 200 participants aged 18–50 years (50% female), 42 of whom were retested in order to assess reliability. An exploratory factor analysis identified 11 factors with eigenvalues greater than 1, which accounted for over 70% of the variance. Tasks showed moderate to excellent test-retest reliability and were not strongly correlated with demographic factors such as age or IQ. The EMOTICOM test battery is therefore a promising tool for the assessment of affective cognitive function in a range of contexts.

Keywords: EMOTICOM, neuropsychological tests, social cognition, motivation and emotion, implusivity, neuropsychiatry, mental health

Introduction

Mental health problems represent an extremely significant health burden, with global costs estimated at $2.5 trillion, projected to increase to $6.5 trillion by 2030, more than any other form of disease (Bloom et al., 2012; Fineberg et al., 2013). Impairments of emotional, motivational and social function are increasingly thought to be fundamental to the neurobehavioral pathology of psychiatric disorders and are becoming important targets for therapeutic intervention (Roiser et al., 2012). Major advances in treatment development will therefore be facilitated by well-designed, carefully validated measures of a comprehensive range of emotional, motivational, and social functions. This is critically important in clinical trials, of both of pharmacological and psychological interventions, which specifically aim to target emotional, motivational, and social processes. Currently, the outcome measures used in trials of such interventions are typically changes in clinical symptoms, and there is a pressing need for new outcome measures that quantitatively measure the effects of these treatments. A validated affective battery would also have important implications in other research contexts; for example, investigating cognitive profiles relevant to the NIMH Research Domain Criteria (RDoC) initiative that aims to create a new framework for mental health research (Insel et al., 2010; Sanislow et al., 2010) focusing on dimensions that cut across DSM diagnostic categories, investigating endophenotypes for genetic studies' or identifying biomarkers for high-risk individuals. However, at present there is no established neuropsychological test battery that offers a comprehensive assessment of “hot” cognitive functions.

Various individual tests have been developed and validated to test specific cognitive hypotheses. However, without standardization, it is difficult to make progress, replicate results, or identify gaps that need to be addressed (Elliott et al., 2011). Multi-center studies and clinical trials would benefit from a comprehensive, validated battery probing emotional, motivational, and social functions. The success of existing, standardized cognitive batteries highlights their recognized importance for assessing cognitive function. For example, the Cambridge Automated Neuropsychological Test Battery (CANTAB; Cambridge Cognition Ltd) has become a widely used battery in both academic research and clinical trials (Robbins et al., 1994, 1998; Cambridge Cognition Ltd). However, the focus is primarily on “cold” cognitive functions (Roiser and Sahakian, 2013) such as executive function, visuospatial memory and various types of attention. Here we generate normative data for a battery of neuropsychological tasks, which assesses a comprehensive range of processes relevant to affective cognition.

Affective cognition

Affective cognition is a term used to describe aspects of cognitive function where stimuli have affective salience; the term “hot” cognition has been coined to distinguish these aspects of cognition from non-emotive “cold” cognitions (Roiser and Sahakian, 2013). Affective cognition, can be defined as reflecting an interface at which emotional and cognitive processes are integrated to generate behavior (Elliott et al., 2011).

Disrupted affective cognition is a core feature of many mental health disorders and cuts across DSM diagnostic categories. For example, biases in processing emotional stimuli have been observed in depression (Surguladze et al., 2004), anxiety (Mogg and Bradley, 2002), schizophrenia (Pomarol-Clotet et al., 2010), substance abuse (Ersche and Sahakian, 2007), eating disorders (Lovell et al., 1997), ADHD (Seymour et al., 2015), and phobic anxiety (Watts et al., 1986). Reward learning and motivation have been shown to be impaired in schizophrenia (Murray et al., 2007; Waltz et al., 2010), Parkinson's Disease (Voon et al., 2010), substance abuse (Park et al., 2010), affective disorders (Murphy et al., 2003), and ADHD (Thomas et al., 2015). Impulsivity has been in described in substance abuse (Voon et al., 2014), eating disorders (Mobbs et al., 2011), and ADHD (Malloy-Diniz et al., 2007). Finally, social cognition impairments have been demonstrated in autism (Happé and Frith, 1996), depression (Zahn et al., 2015), and schizophrenia (Fett et al., 2011). Social, emotional, and motivational aspects of cognition are thought to be key predictors of functional outcomes. Therefore, novel interventions targeting affective cognition may be effective for improving functional outcomes, as well as reducing symptoms. Examples include cognitive bias modification in depression (Baert et al., 2010; Roiser et al., 2012), social cognition training in schizophrenia (Combs et al., 2007), or pharmacological agents to promote social function such as oxytocin (Feifel et al., 2012).

A number of studies have explored the potential factor structure of affective cognition in mental health disorders. For example, the MATRICS project identified five sub-processes relevant to schizophrenia including, theory of mind, social perception, social knowledge, attributional bias, and emotional processing (Green et al., 2008). Others have identified four factors including perceiving emotions, facilitating thought, understanding emotions, and managing emotions (Mayer and Salovey, 1997; Mayer et al., 2003); or two factors including an emotional perception and understanding factor and an emotional facilitation and management factor (Eack et al., 2010).

However, despite a clear consensus that “hot” cognitive function is a multidimensional construct with many underlying sub-processes, no comprehensive battery assessing affective function is currently available. There are a number of batteries that include a limited number of affective tasks in predominately “cold” cognitive test batteries (e.g., CANTAB; www.cambridgecognition.com, MATRICS; www.matricsinc.org, CogState; www.cogstate.com, WebNeuro; www.brainresource.com). A recently developed explicit hot cognition battery, the Emotional Test Battery (ETB; www.p1vital.com), focuses on emotion processing tasks of particular relevance to depression.

In developing the test battery described here we chose to focus on four distinct domains of affective cognition: emotion processing, the ability to process and respond to affective stimuli, including emotional faces; motivation, the ability to learn, apply effort and make decisions driven by incentives; impulsivity, premature or risky responding; and social cognition, the ability to process information about situations involving interpersonal interactions. For each of these domains we piloted in 30 individuals a combination of novel, adapted and existing tasks designed to probe key underlying affective functions. We selected for inclusion in the final battery those tasks that were feasible in brief versions, readily understood and well-tolerated by participants and (for existing or adapted tasks) that elicited robust replication of previously observed effects. Further details of excluded tasks are available from the authors on request.

Emotional processing

Emotion recognition/categorization

Recognition of facial expressions is a widely-used paradigm in neuropsychiatry, particularly in studies of depressed patients who tend to rate ambiguous expressions as more negative (Bouhuys et al., 1999; Surguladze et al., 2004). Harmer et al. (2011) argues that emotional face recognition may be a sensitive biomarker for effective antidepressant treatment. We therefore aimed to develop a task that effectively probed emotional facial recognition. The Emotion Recognition Task (ERT) included in the CANTAB battery (Cambridge Cognition Ltd) has proven to be a promising task examining emotion recognition in clinical populations. However, in order to include an ERT in EMOTICOM with limited time available, we opted to focus on basic emotions; happy, sad, anger and fear and chose to exclude more complex emotions such as surprise and disgust. We also adapted the task to include two versions; one that assessed facial recognition, similarly to the original CANTAB ERT and one that more specifically assessed eye recognition. Including emotional eyes recognition was motivated by evidence supporting the “reading the mind from the eyes” test (Baron-Cohen et al., 2001) as an effective assessment of the ability to recognize the emotional state of others using just the expressions around the eyes. We further adapted the task to include control conditions, i.e., identifying the age of a face and eyes, to provide baseline measurement in neuroimaging investigations.

Attentional bias

Biased emotional attention can be effectively measured using the affective go/no-go test (Cambridge Cognition Ltd). Attentional biases have been observed in depression (Murphy et al., 1999; Erickson et al., 2005), mania (Murphy et al., 1999), anxiety disorders (Watts et al., 1986; Mogg et al., 1995), substance abuse (Ersche and Sahakian, 2007), and eating disorders (Lovell et al., 1997). Negative biases in processing emotional stimuli have been suggested as an important biomarker for antidepressant efficacy and may predict responses to both psychological and pharmacological interventions (Harmer et al., 2009; Roiser et al., 2012). We therefore adapted two versions of the Affective Go No-Go task: one similar to the CANTAB with word stimuli and one with face stimuli. The motivation for adapting the AGN to include faces was to potentially improve any cross-cultural, educational, and age influences on the word version. For example, emotionally salient word stimuli may require a minimum reading level that may not be suitable for use in children. Indeed, a facial version of the AGN has shown to be a promising tool in pediatric anxiety and depression (Ladouceur et al., 2006). Additionally, an emotionally cued Posner task (Posner, 1980) using eye gaze in emotional facial expressions was piloted as part of the development of the EMOTICOM battery but did not show significant condition effects.

Emotional memory

Biased emotional memory for personal experiences has been suggested as an important trait marker for depression (Brittlebank et al., 1993). Depressed patients also show a more general bias toward remembering negative information (Hamilton and Gotlib, 2008) and patients with schizophrenia show deficits in remembering positive stimuli (Herbener et al., 2008) suggesting a possible double dissociation between the two disorders. We therefore developed an emotional memory task that required an encoding phase presented at the start of the EMOTICOM battery and a retrieval phase presented at the end in order to assess biases in emotional memory. We also piloted an emotional working memory task using a spatial n-back (for review see Owen et al., 2005) with emotional faces, however this did not produce sufficient significant condition effects.

Motivation and reward

Reinforcement learning

Behavioral tests assessing reinforcement learning (RL) in humans are directly comparable to operant conditioning tasks used in animals (Roberts et al., 1988; Birrell and Brown, 2000). Human reinforcement learning tests typically involve learning which abstract stimuli predict winning or losing points or money (Owen et al., 1991; Pessiglione et al., 2006). Reinforcement learning, and corresponding responses in the brain's reward system, are reliably disrupted in several neuropsychiatric diseases, including schizophrenia (Murray et al., 2007; Waltz et al., 2010), Parkinson's Disease (Voon et al., 2010), alcohol dependence (Park et al., 2010), and depression (Murphy et al., 2003). One weakness of several tests is the conflation of reward and punishment learning (Cools et al., 2002). This is important, since reward and punishment may be subserved by separable, opponent processes in the brain (Daw et al., 2006). We therefore aimed to develop a novel reinforcement learning task that separated reward and punishment feedback in order to assess sensitivity to these independently.

Incentive motivation

Tests of incentive motivation measure how much effort an individual is prepared to exert to gain reward. The monetary incentive delay (MID) functional neuroimaging task features a speeded response to obtain a reward or avoid a loss (Knutson et al., 2001). However, the behavioral measure arising from this paradigm has seldom been shown to be altered by diagnosis or pharmacological manipulation (Knutson et al., 2004; Scheres et al., 2007). Indeed, the MID continually updates the threshold for success, which might reduce behavioral differences between conditions. Hence we aimed to develop an incentive motivation task that produced reliable behavioral differences that have the potential to provide important biomarkers for assessment and treatment interventions. We adapted the Salience Attribution Task (Roiser et al., 2009a) which has previously shown robust behavioral markers of adaptive motivational salience in Schizophrenia and developed a version that specifically evaluated motivation relating to reward and punishment separately.

Value-based choice

Tests of value-based choice investigate how subjects use different types of information (e.g., probability, reward, punishment) in order to guide economic decision-making. In contrast to tests of reinforcement learning, there is typically no learning component in tests of value-based choice. As such, the widely-used Iowa Gambling Task (Bechara et al., 1994) is not a specific test of value-based choice, since it also involves learning. The Cambridge Gamble Task (CGT: Rogers et al., 1999), part of the CANTAB suite of tests, asks subjects to decide on which of two options to bet, and to stake a certain percentage of their points on this bet. The CGT is sensitive to unipolar (Murphy, et al., 2001) and bipolar depression (Roiser et al., 2009b), schizophrenia (Hutton et al., 2002), and psychopharmacological manipulation (Rogers et al., 1999). However, it cannot determine whether decision-making is influenced by reward seeking or punishment avoidance. A later development (Rogers et al., 2003) can distinguish between these and is sensitive to several neuropsychiatric conditions (Roiser et al., 2006; Chandler et al., 2009) and psychopharmacological manipulations (Scarna et al., 2005), but includes a very restricted set of probabilities. We therefore adapted the CANTAB CGT (Cambridge Cognition Ltd) to investigate reward seeking and punishment avoidance separately.

Impulsivity

Waiting impulsivity

Coordination between initiation and inhibition of actions is required for successful behavior. Patients with ADHD (Aron and Poldrack, 2005), obsessive compulsive disorder (Malloy-Diniz et al., 2007), and schizophrenia (Kaladjian et al., 2007) show impairments in impulsivity. The four choice serial reaction time task (4-CSRTT) is a novel translation from the widely used 5-choice serial reaction time rodent task (5-CSRTT; Robbins, 2002). It has demonstrated clear deficits in substance abuse (Voon et al., 2014) and is sensitive to effects of dietary tryptophan depletion which is thought to reduce central 5-HT (Worbe et al., 2014). We therefore decided to incorporate the 4CSRTT (Voon et al., 2014) into the EMOTICOM battery which measures incentive motivation to rewards and premature responses elicited by anticipated reward.

Delay and probability discounting

Another aspect of impulsivity is the preference for immediate gratification, even when waiting longer might lead to higher absolute gain. Delay discounting is the progressive reduction in subjective value of a reinforcer with time. It can be assessed using two types of task—hypothetical or experiential. Hypothetical discounting tasks require choices between immediate (e.g., £1 now) and delayed (e.g., £5 in 1 month) rewards (Mazur, 1987; Green et al., 1996; Kirby, 2009). The experiential discounting tasks differs from hypothetical in that respondents directly experience the delay and receive the reward during the task (Reynolds and Schiffbauer, 2004). Patients with ADHD and substance use disorders show steeper discounting rates in such tasks, which also show good temporal stability similar to personality traits (Ohmura et al., 2006; Kirby, 2009). We therefore developed a computerized delay discounting task based on Richards et al.'s (1999) adjustment procedure.

Social cognition

Moral emotion

Moral emotions can be experimentally induced either in response to verbal descriptions or pictures of specific interpersonal behavior (Moll and de Oliveira-Souza, 2007) or behavior contravening normal social values (Zahn et al., 2009). Patients with ventromedial prefrontal (VMPFC) lesions show abnormal responses to hypothetical moral dilemmas (Ciaramelli et al., 2007; Koenigs et al., 2007) and patients with antisocial personality disorder (Blair, 1995) and Autism (Moran et al., 2011) show deficits in moral judgment. We developed a novel computerized Moral Emotions task that comprising of cartoon scenarios rather than lengthy vignettes that are more likely to be affected by reading ability, intelligence and age.

Theory of mind

Theory of Mind (TOM) refers to the ability to infer the mental states of others (Frith and Frith, 2003). A number of paradigms have been proposed to probe this function including false belief tasks (Frith and Corcoran, 1996), “faux pas” tests, visual jokes, understanding irony and the “Reading the Mind in the Eyes” test (Baron-Cohen et al., 2001; although note that this is most similar to an emotional recognition task—see above). Patients with autism typically show impaired TOM (Happé and Frith, 1996) and it is also sensitive to schizophrenia (Frith and Corcoran, 1996; Bora et al., 2009; Fett et al., 2011). While valuable in populations with overt impairment, existing TOM tasks are typically insensitive to variation in normal adult performance as most participants perform at ceiling. Therefore, we developed a complementary task that depicted ambiguous social situations with no right or wrong answer, thus allowing greater variation of responses in healthy volunteers. Rather than assessing whether participants have TOM ability, this task assesses the extent to which people choose to use TOM information.

Social economic exchange games

Economic games, such as the Ultimatum Game and Prisoners' Dilemma are popular tasks for exploring the neurobiology of social decision-making (King-Casas et al., 2005; Miller, 2005; Fehr and Camerer, 2007; Crockett, 2009). A number of patient groups have been studied using these games, including psychopathy (Koenigs et al., 2010; Rilling et al., 2015), schizophrenia (Agay et al., 2008), autism (Andari et al., 2010), depression (Pulcu et al., 2015), and borderline personality disorder (King-Casas et al., 2008; Seres et al., 2009; Unoka et al., 2009). It has been argued that these games may provide specific and sensitive biomarkers for social pathologies (Kishida et al., 2010). Traditionally, these games are long and involve a complex set-up with multiple players, which are unsuitable for neuropsychological testing. Therefore, we developed a simple one-player game of the Ultimatum Game and Prisoners' Dilemma, which probe social interaction within the context of a test battery.

Social decision making

Optimal decision-making in social contexts recruits a combination of associative and inferential computations. For example, one may have first-hand experience, one may observe choices of other people (or receive a recommendation), and one may infer the knowledge or intentions of the others to weight the influence of their decision. Therefore, we included the Inference Task which approximates the contribution of each of these processes to decision-making. Specifically, it employs the useful heuristic of confidence which can be used to infer the certainty of an agent's information and weight the influence of his/her endorsement on privately held beliefs (Thomas and McFadyen, 1995). The effects of such inferences on value computation are hypothesized to underlie the reassuring influence of another's confidence and generate distinct representations of value in the subject. Successful task performance requires cue combination, the integration of value computations, and theoretically, social inferences of other people's knowledge.

Aims and objectives

The specific aims of the project were to: (a) generate a computerized test battery assessing multiple aspects of “hot” cognition; (b) demonstrate ease of administration, feasibility, and tolerability; (c) standardize the test battery in a large cohort of healthy volunteers, including an exploratory factor analysis to identify important, independent constructs; and (d) establish measurement stability in a smaller sample of healthy volunteers.

Hypotheses

We hypothesized a factor analysis would reveal that the tasks would probe affective function best explained by a four factor model mapping onto emotion processing, motivation, impulsivity, and social cognition. We further hypothesized that tasks without a learning component would show at least moderate test-retest reliability.

Materials and methods

Participants

Two hundred healthy volunteers were assessed (see Table 1 for demographic characteristics), 42 of whom were re-tested within 5–10 days in order to assess test re-test reliability. This will furnish sufficient power to detect test-retest reliability of >0.35 (p = 0.05, 80% power). Potential participants were recruited via advertisements in the local community and on social media. Following telephone screening, participants were included if they met the following criteria: 18–50 years old; no self-reported previous or current psychiatric disorders, including depression, anxiety, eating disorders, and drug/alcohol dependence; no neurological disorders; no significant head injury resulting in unconsciousness; no current use of medication known to affect mood or cognition; no first-degree relatives suffering from any psychiatric disorders; smoked fewer than five cigarettes per day; drank less than the government guidelines for weekly alcohol intake (www.drinkaware.co.uk); and fluent in English. Participants completed the Brief Symptom Inventory (Derogatis and Melisaratos, 1983), meeting the criteria for adult non-patients across nine symptom dimensions; somatisation, obsessive compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, and psychoticism. Participants were further interviewed using the Mini International Neuropsychiatric Interview (Sheehan et al., 1998) to exclude any psychopathology.

Table 1.

Demographic characteristics of sample (N = 200), stratified by age, IQ, gender, and ethnicity for the standardization of the EMOTICOM neuropsychological test battery.

Mean SD
Age 26.66 9.81
Years in Education 14.40 2.01
WTAR IQ 112.18 6.29
Gender N %
Female 100 50
White 157 78.5
ETHNICITY
Afro Caribbean 7 3.5
Asian-Indian 10 5
East-Asian 9 4.5
Mixed 9 4.5
Other 8 4

Eligible participants were invited to attend a 3.5-h appointment at the Neuroscience and Psychiatry Unit, University of Manchester or the Behavioral and Clinical Neuroscience Institute, University of Cambridge. Participants provided written informed consent after the study procedures were explained, and their IQ was estimated using the WTAR (Wechsler, 2008). This study was approved by the University of Manchester and the University of Cambridge Research Ethics Committees.

Design

Participants completed 16 neuropsychological tests programmed in PsychoPy (Peirce, 2007) on a touchscreen laptop (Dell XT3). The tasks were administered in a quiet testing room over 3 h. Some participants chose to complete the tasks over two sessions no longer than 1 week apart. The tests were administered in a randomized sequence to eliminate systematic effects of fatigue. Participants were reimbursed for their time and travel expenses, they also received an additional bonus of up to £10, calculated on the basis of the average money won on tasks that involved a monetary incentive.

Neuropsychological tasks

Analysis

All analyses were performed with SPSS statistical software (IBM SPSS Statistics Version 20.0).

Factor analysis

The measures thought to be most reflective of the constructs investigated were standardized using z-scores (after transformation if appropriate) and entered into a factor analysis to determine the underlying latent variable structure of the data. Here, we conducted an exploratory factor analysis to identify the number of factors needed to maximize the amount of variance explained. An eigenvalue cut-off of 1 was used to determine whether a factor explained sufficient variability in the data. The method employed utilized varimax rotation with Kaiser normalization.

Reliability analysis

The reliability and stability of the tasks was assessed by comparing performance in 42 volunteers who competed the battery on two occasions, 5–10 days apart. Test–retest was assessed by calculating the average-measures intraclass correlation coefficient using a two-way mixed effects model, which controls for overall changes in performance between sessions (i.e., repetition effects). Different guidelines exist for the interpretation of the ICC. Here we take an ICC value of less than 0.40 to be poor, 0.41–0.59 as fair, 0.60–0.74 as good and values exceeding 0.75 as excellent (Fleis et al., 2003). These terms should be interpreted with caution as they do not take into account the confidence intervals of the ICC measure.

Correlation analysis

In an exploratory supplemental analysis, two tailed Pearson's correlations were used to correlate task performance with demographic measures such as age, IQ and years of education. Gender differences were examined using independent samples t-tests. The statistical significance of all correlations were corrected for multiple comparisons (0.05/n; n = number of task variables).

Task variables

For each task there are a number of possible outcome measures. For the factor analysis, test-retest analysis and correlations with demographic variables, which are the focus of the present publication, we chose the primary outcome measures outlined in Table 2.

Table 2.

Full list of neuropsychological tasks with outcome measures which are included in the EMOTICOM neuropsychological test battery.

EMOTION PROCESSING
Emotion recognition/categorization Task 1: Emotional Recognition Task (ERT)
We developed two versions of an ERT; one with full faces, and one with eyes only. In these tasks, the participant is shown a series of faces or eyes that appear on the screen briefly, and is asked to identify the emotion (happiness, sadness, anger or fear). In the control condition, participants are asked to identify the age of a face (child, young adult, middle aged, elderly).
Time to administer: 12 min
Outcome Measures: Accuracy scores were calculated for each facial emotion (happiness, sadness, anger, and fear). Average accuracy refers to average accuracy across all four emotions. Affective bias scores were calculated by subtracting accuracy for sad faces from accuracy from happy faces. This analysis was also performed for the eyes emotional recognition test.
Task 2: Emotional Intensity Morphing Task
This task assesses the point of emotional intensity at which participants can recognize a facial emotion. Participants view faces that either increase or decrease in emotional intensity and are instructed to respond when they either (a) detect the presence of emotion or (b) no longer detect the presence of emotion. The emotion that they were detecting was made explicit to participants. The task includes five different emotions: happiness, sadness, anger, fear, and disgust.
Time to administer: 5 min
Outcome Measures: The point of detection was calculated by taking the level of intensity in the facial expression needed in order to detect (increasing) or no longer detect (decreasing) each emotion. The Average point of detection refers to average point of detection across all five emotions. Affective bias scores were calculated by subtracting the point of detection for sad faces from point of detection from happy faces.
Attentional bias Task 3: Face Affective Go No-Go Task
This task assesses information processing biases for positive and negative facial expressions. The participant is told a target emotion (happy, sad, neutral), and asked to press a button only when the target emotion is present. The task consists of six blocks, each of which presents a series of faces: (1) happy target/sad distractor, (2) happy target/neutral distractor, (3) neutral target/happy distract, (4) neutral target/sad distract, (5) sad target/happy distract, and (6) sad target/neutral distract.
Time to administer: 6 min
Outcome Measures: Reaction times (RT) were calculated for all “hit” responses for each of the six conditions. Affective bias scores were calculated by subtracting the sad target/happy distract condition RT from the happy target/sad distractor condition RT.
Task 4: Word Affective Go No-Go
This task assesses information processing biases for positive, negative and neutral emotional words. Words were chosen based on their ratings in a pilot study in an independent cohort of 30 volunteers and were matched for valence, arousal, frequency and word length. Participants are given a target emotion (happy, sad, neutral), and asked to press a button only when the target emotion is present. Similarly to the faces affective go no go, the task consists of six blocks, each of which presents a series of words: (1) happy target/sad distractor, (2) happy target/neutral distractor, (3) neutral target/happy distractor, (4) neutral target/sad distractor, (5) sad target/happy distractor, and (6) sad target/neutral distractor. Note that this task is not the same as the Cambridge Cognition (www.cambridgecognition.com) word affective go no-go task.
Time to administer: 6 min
Outcome Measures: Reaction times (RT) were calculated for all “hit” responses for each of the six conditions. Affective bias scores were calculated by subtracting the sad target/happy distract condition RT from the happy target/sad distractor condition RT.
Emotional memory Task 5: The Emotional Memory Recognition Task
This task assesses biases in the recognition of emotional stimuli. During the encoding stage, participants are asked to rate images displaying positive, negative or neutral scenes, on valence and arousal intensity. Images were of scenes without people and were validated as positive, negative or neutral on the basis of pilot testing in an independent cohort of 30 volunteers. During the retrieval stage, images from the encoding phase are paired with new images. Participants are asked to indicate which image they saw previously. The encoding phase consists of 30 images (10 positive, 10 negative and 10 neutral) whilst the retrieval phase consists of 60 images (20 positive, 20 negative and 20 neutral), half of which were previously seen in the encoding phase.
Time to administer: 5 min
Outcome Measures: Valence and intensity ratings from the encoding phase were calculated for each valence condition; positive, negative and neutral. Retrieval affective bias was calculated by subtracting accuracy for negative stimuli from accuracy for positive stimuli.
MOTIVATION AND REWARD
Reinforcement learning Task 6: Reinforcement Learning Task
This task separately assesses reward and punishment learning. Participants are shown colored circles, and asked to make a choice between the two based on which one they thought was more likely to win money and not lose money. Participants receive feedback and are continually updated on their total score. There are two conditions; one condition is a no lose condition whereby participants either win (£0.50 presented as 50p) or fail to win (0p). The second condition is a no win in condition whereby they lose (50p) or avoid losing (0p). Participants must learn, through sampling the circles, which of the two is the better option, with probabilities (unknown to participants) set at 70%/30%. In the transfer phase, all possible pairs of circles are presented and participants choose their preferred option. In this phase, no feedback is given.
Time to administer: 12 min
Outcome Measures: A reinforcement learning model was applied to the data. Learning rate (alpha) refers to how fast the participant learns new information. A high learning rate indicates that the participant incorporates new information more quickly. Alpha was calculated for win and loss conditions separately.
Incentive motivation Task 7: The Monetary Incentive Reward (MIR) Task
This task assesses effort to avoid punishments and gain rewards. Participants see a pair of identical circles displayed on the screen, shortly followed by a black box. Participants are instructed to make a response as soon as the black box appears. The circles contain colored lines, which indicate that on that trial they will either gain or lose money. The distance between the lines indicates the size of loss/gain. The faster they respond the more money they win or the less money they lose, and this relationship remains constant throughout the task.
Time to administer: 10 min
Outcome Measures: Reaction times were calculated for each condition; high win, low win, low loss and high loss. These reaction times were further standardized by subtracting each of the four conditions from the neutral “baseline” reaction time. High and low win reaction times were combined to produce the average reaction time for wins. Likewise high and low loss reaction times were combined to produce the average reaction time for loss.
Task 8: The Progressive Ratio Task
Progressive ratio tasks have been widely used to examine motivation in non-human subjects (Hodos, 1961; Bradshaw and Killeen, 2012). More recently, progressive ratio tasks have been adapted for use in humans using a variety of rewards (e.g., money, stimulants, food) to assess self-control and identify participants' motivational “breakpoint,” i.e., the maximum effort that a participant will expend in order to receive a reward (Roane, 2008). In this task participants are presented with four red squares on the screen and are instructed to select the square that differs in size to the other three. Participants are paid progressively less per trial as they continue with the task. They are also told that they can stop their participation in the task at any point, but that they still have to sit facing the screen for the remaining time (20 min minus the time they performed the task).
Time to administer: 20 min
Outcome Measures: The progressive Ratio task was adapted part way through the study therefore data is only presented for the adapted task (participant n = 78). The total number of trials was calculated in order to estimate the breakpoint, i.e., the point at which participants did not wish to continue with the task. Running rate was calculated as the time taken to complete the block of trials. The post reinforcement pause was the average time taken to initiate the next trial following a reward. Approximately 57% of participants completed the task therefore only allowing us to calculate a breakpoint for the remaining participants. Consequently, the progressive ratio task was not included in the factor analysis and test-retest reliability determinations.
Value-based choice Task 9: The adapted Cambridge Gambling Task
This task was developed to assess decision-making and risk-taking behavior, with reward and loss trials administered separately. On each trial, the participant is presented with a roulette wheel; a proportion of which is colored purple and a proportion of which is orange. There are 5 different proportions ranging from very certain to very uncertain. Participants must place a bet on the outcome they expect. A spinning pointer is then displayed, which lands on one of the colors, providing feedback for the participant. There are two conditions; a loss condition and a win condition which allows the separation of reward and punishment.
Time to administer: 10 min
Outcome Measures: The average value of chips placed on each level of probability was calculated separately for the win and loss conditions. Only choices of the most likely outcome were included. This was used to compute a risk adjustment (RA) score using the formula: Risk adjustment = (2*bet at 90%) + (1*bet at 80%) + (0*bet at 70%) - (1*bet at 60%) - (2*bet at 50%)/Average bet. RA was calculated for win and loss conditions separately.
IMPULSIVITY
Waiting impulsivity Task 10: The four-choice serial reaction time task
This task (Voon et al., 2014) assesses visual attention, and ability to monitor and respond to unpredictable targets. Participants have to indicate a box, from 4 choices, in which a target has appeared.
Time to administer: 25 min
Outcome Measures: Data from 175 participants was utilized in the analyses due to initial technical problems. The motivational index was calculated by using the following formula: motivational index = (baseline reaction time—post baseline reaction time)/ baseline reaction time. The number of premature events was calculated as the combination of the number of premature releases (releasing the spacebar prematurely) and the number of premature responses (releasing the space bar prematurely and touching the screen).
Delay and probability discounting Task 11: The Discounting Task
This task assesses the rate of discounting across delays and probabilities. There are ten conditions; five levels of delay (0, 30, 90, 180, 365 days) and five levels of probability (100, 90, 75, 50, 25%). Participants must decide whether they would prefer a standard fixed amount (always £20) associated with a particular delay or probability, compared to an alternative amount definitely available immediately.
Time to administer: 7 min
Outcome Measures: Indifference points were calculated for each length of delay or degree of uncertainty. These indifference points refer to the amount of immediately available money that the participant considered to be equivalent to the delayed or uncertain reward. For delay discounting, the area under the curve was used to calculate the level of discounting using the following formula: Area under the curve = [(2-0)*((indifference point at 0 days + indifference point at 2 days)/2)]+[(30-2)*((indifference point at 2 days + indifference point at 30 days)/2)+[(180-30)*((indifference point at 3 days + indifference point at 180 days)/2)]+ [(365 -180)*((indifference point at 180 days+ indifference point at 365)/2)]. A smaller AUC, indicates more severe discounting of the delayed reward and thus greater impulsivity. A similar analysis was conducted for probability discounting, whereby smaller AUC indicates greater risk aversion.
SOCIAL COGNITION
Moral emotion Task 12: The Moral Emotions task
This task uses cartoon figures to depict moral scenarios. Half of the scenarios depicted a deliberate harm whereas the remaining half depicted an accidental harm in order to explore the effect of intention upon moral emotions. Participants were asked to imagine how they would feel in the situation as either the actor or the victim, and rated the following emotions; guilt, shame, anger and feeling “bad.”
Time to administer: 13 min
Outcome Measures: The average rating for feeling bad was calculated across all conditions: victim vs. agent and intentional vs. unintentional. Agent ratings for guilt were also calculated.
Theory of Mind Task 13: Social Information Preference Test
This task assesses information sampling in socially ambiguous situations. Participants are shown a scene, with three faces (feelings), three thoughts and three facts about the scene hidden from view. Participants are able to select only four out of nine pieces of information to help resolve ambiguity. They then choose between three possible outcomes of the situation (negative, positive or neutral), which provides a measure of interpretational bias.
Time to administer: 10 min
Outcome Measures: The proportion of thoughts selected and the valence of the chosen outcome, positive, negative or neutral was calculated. The affective bias in interpretation was calculated by subtracting the proportion of negative outcomes chosen from the proportion of positive outcomes chosen.
Social economic exchange games Task 14: Prisoners' Dilemma
This task assesses cooperation with a computerized opponent. On each trial, participants must repeatedly press the space bar as fast as they can in order to fill a jar with coins. Each trial is manipulated so that the participant wins more coins, the opponent wins more coins, or they both win equal amounts. The coin totals are then combined and participants are instructed that they may either split or steal the total sum. Participants are told that if they both choose to split, they get half the money each, and if they both steal, they each get nothing. If they split and the opponent steals they get nothing and the opponent gets everything. Alternatively, if they steal and their opponent splits, they get everything and the opponent nothing. Participants are faced with three different opponents each with a different strategy: aggressive (tit for tat, but starts with steal), tit for two tats (starts with split, then changes behavior after the player has stolen two times consecutively) and a cooperative player who always splits.
Time to administer: 10 min
Outcome Measures: The average steal proportion was calculated as the proportion of trials that participants chose to steal from their opponent from the total number of trials across each type of opponent (aggressive, tit for two tats and cooperative).
Task 15: Ultimatum Game
This task assesses sensitivity to fairness and tendency to inflict punishment. Similarly to the Prisoner's Dilemma, participants initially complete a task in which they can win money. Here they can select 3 balls from a choice of 9 and depending on what colors are revealed behind the balls, participants can win money. Each trial is manipulated so that the participant wins more money, the opponent wins more money, or they both win equal amounts. This money is then combined with the opponent's total. Next, participants are informed whether they get to decide how the money is split or whether it is up to the opponent. If the opponent divides the money, the participant gets the choice to either accept or reject their offer. These offers have seven levels ranging from fair (50:50) to increasingly unfair (10:90). If the participant accepts, they each get the allotted amount, and if they reject, they both get nothing. When the participant divides, they can choose from four divisions differing in fairness (50:50, 40:60, 30:70, 20:80, and 10:90).
Time to administer: 12 min
Outcome Measures: The proportion of offers accepted was calculated as the number of trials that participants chose to accept the offer from their opponent from the total number of trials. Risk adjustment was further calculated by using the following formula: Risk adjustment = (2*acceptance at 50% offer) + (1* acceptance at 40% offer) + (0* acceptance at 30% offer) − (1* acceptance at 20% offer) − (2* acceptance at 10% offer)/Average offer. The average offer proposed refers to proportion of times participants chose each of the four levels of offer available.
Task 16: Inference Task
Participants initially view a series of face pairs and are instructed to touch the more confident of each pair. This confirms that they are able to read confidence in faces. Participants are then asked to guess the contents of a series of buckets (mostly red or mostly blue jellybeans), based on a combination of information sources. On each trial, the subject and the honest computer (who never lies) each take a sample from the bucket. The participant is provided with: a sample of eight jellybeans, the answer of the honest computer (based on its own sample; it does not know the participant's) and the confidence of the honest computer in the answer it provided. This confidence is expressed with a human facial expression, either positive or skeptical. Each bucket is different from the rest, and independently numbered. The sample, answer of the computer and computer confidence are independently manipulated. Optimally, the subject will be able to increase the computer's influence when it expresses confidence and decrease its influence when it is apparently unsure of its decision. Information inferred from the choice and confidence of the computer must also be combined with information directly observed in one's own
Time to administer: 16 min
Outcome Measures: The proportion of red and blue buckets chosen was calculated for each level of probability (i.e., number of red jellies in participants hand 1/8, 2/8, 3/8, 4/8, 5/8, 6/8, 7/8) and for each condition of computer choice (red or blue) and confidence (confident and unconfident). Area under the curve was calculated for each. The effect of probability refers to the area under the curve collapsed across all computer choice and confidence conditions. The effect of computer choice was calculated by subtracting the AUC when the computer chose blue from AUC then the computer chose red.

Results

Standardization

A summary of the means and standard deviations can be found in Table 3.

Table 3.

Summary of the means and standard deviations.

Domain and task Test score used Mean SD
EMOTION
Facial recognition Face: affective bias 9.48 19.76
Eyes: affective bias 5.03 26.33
Emotional intensity Increasing affective bias −16.21 15.65
Decreasing affective bias 3.18 14.51
Face affective go/no-go Affective bias RT (ms) −30.27 66.93
Word affective go/no-go Affective bias RT (ms) −3.30 195.10
Emotional memory Retrieval affective bias −4.05 10.66
Average retrieval accuracy 93.67 7.83
REWARD/MOTIVATION
Reinforcement learning Win Learning rate 0.23 0.33
Loss Learning rate 0.27 0.34
Monetary incentive reward Win—neutral RT (ms) 34.50 34.42
Loss—neutral RT (ms) 28.60 33.44
Adapted Cambridge gambling Win risk adjustment 1.61 1.34
Loss risk adjustment 1.94 1.17
Progressive ratioa Breakpoint 78.12 32.35
Post reinforcement pause (seconds) 2.00 0.74
IMPULSIVITY
4CSRTTb Motivational Index 0.16 0.15
Delay discounting Delay discounting 3308.95 1928.79
Probability discounting 989.71 255.20
SOCIAL COGNITION
Moral emotions Agent guilt ratings 79.68 12.22
Feeling bad ratings 22.98 9.16
Information preference Thoughts chosen 54.10 14.86
Affective bias in outcome 10.59 20.57
Prisoners' dilemma Average steal 39.63 28.39
Ultimatum game Risk adjustment 2.06 1.80
Value of offers proposed 36.80 10.07
Inference task Effect of probability 388.00 65.84
Effect of computer choice 177.75 129.37
a

Only 78 participants were included in the analyses for the Progressive ratio task due to an update to the task part way through the study.

b

Only 175 participants were included in the correlation analyses for the 4CSRTT due to technical failure.

Factor analysis

Data from all participants were entered into the factor analysis. The results of the varimax rotation for the tasks are shown in Table 4. An eleven-factor solution was derived based on eigenvalues greater than 1, which cumulatively accounted for 70% of the variance (see Figure 1). Only factor loadings greater than 0.40 are shown. Data were assessed for the adequacy of factor analytic methods. Bartlett's test was highly significant [χ(276)2=1071.72, p < 0.001], suggesting that variable correlations did not form an identity matrix. Measures of sampling adequacy were also sufficient (KMO = 0.54).

Table 4.

Summary of the factor loadings for EMOTICOM tests on factors 1–11.

Test Factors
1 2 3 4 5 6 7 8 9 10 11
EMOTIONAL RECOGNITION
Eyes affective bias 0.63
Face affective bias 0.74
INTENSITY MORPHING
Increasing affective bias −0.66
Decreasing affective bias 0.62
WORDS AFFECTIVE GO/NO-GO
Affective bias (RT) 0.49
REINFORCEMENT LEARNING
Loss learning rate −0.77
Win learning rate 0.66
FACES AFFECTIVE GO/NO-GO
Affective bias (RT) −0.74
ULTIMATUM GAME
Risk adjustment 0.78
DELAY DISCOUNTING
Delay discounting −0.60
Probability discounting −0.49
EMOTIONAL MEMORY
Retrieval affective bias 0.75
CAMBRIDGE GAMBLING TASK
Win RA 0.79
Loss RA 0.82
MONETARY INCENTIVE REWARD
Win-neutral RT 0.87
Loss-neutral RT 0.83
MORAL EMOTIONS
Guilt rating (agent) −0.87
Feeling “bad” rating 0.89
INFORMATION PREFERENCE
Proportion thoughts −0.70
Outcome affective bias 0.65
PRISONERS DILEMMA
Steal rate (%) −0.82
ULTIMATUM GAME
Value of offers proposed 0.83
INFERENCE TASK
Effect of probability 0.96
Effect of computer choice 0.95
Figure 1.

Figure 1

Eleven-factor solution derived from an exploratory factor analysis.

Factor 1 represents affective biases in emotional recognition whereas Factors 2 and 3 capture affective biases in reaction times. Factor 4 contains tasks that have an element of value adjustment. Bias in emotional memory and probability discounting load onto Factor 5. Factor 6 represents measures of probabilistic decision making. Factor 7 represents latency measures of incentive motivation. Factor 8 represents social cognition, specifically moral emotions. Factor 9 represents social information preference and Factor 10 captures cooperation in social exchange games. Finally, Factor 11 loads onto social decision making. The 4CSRTT was omitted from the analysis in order to retain the full sample of participants; however, when running the factor analysis with the motivational index included, this variable loads onto factors 2 and 3 (affective biases in RTs).

Test-retest reliability

Test-retest reliability results are summarized in Table 5.

Table 5.

Test re-test reliability.

Domain and task (N = 200) Test score used Intraclass correlation coefficient
EMOTION
Facial recognition Face: affective bias 0.86
Eyes: affective bias 0.74
Emotional Intensity Increasing affective bias 0.80
Decreasing affective bias 0.73
Face affective go/no-go Affective bias (RT) 0.34
Word affective go/no-go Affective bias (RT) 0.44
Emotional memory Retrieval affective bias 0.41
Average retrieval accuracy 0.64
REWARD/MOTIVATION
Reinforcement learning Win learning rate 0.15
Loss learning rate −0.27
Monetary incentive reward Win—neutral RT 0.37
Loss—neutral RT 0.31
Adapted Cambridge gambling Win risk adjustment 0.75
Loss risk adjustment 0.75
IMPULSIVITY
4CSRTTa Motivational index 0.54
Delay discounting Delay discounting 0.70
Probability discounting 0.58
SOCIAL COGNITION
Moral emotions Agent guilt ratings 0.94
Feeling bad ratings 0.87
Information preference Proportion thoughts 0.62
Affective bias in outcome 0.66
Prisoners' dilemma Average steal rate 0.64
Ultimatum game Risk adjustment 0.58
Value of offers proposed 0.71
Inference task Effect of probability 0.65
Effect of computer choice 0.77
a

Only 32 participants were included into the reliability analyses for the 4CSRTT.

Associations with demographic factors

Demographic factors associated with test performance are listed in Table 6.

Table 6.

Association between tasks and demographic characteristics.

Domain and task (N = 200) Test score used Age (r) IQ (r) Years in education (r) Gender (t)
EMOTION
Facial recognition Face: affective bias 0.27* −0.16 −0.08 0.52
Eyes: affective bias 0.37* −0.09 −0.01 2.29
Emotional intensity Increasing affective bias −0.01 −0.01 −0.03 −1.30
Decreasing affective bias 0.05 −0.05 −0.03 2.43
Face affective go/no-go Affective bias (RT) −0.01 −0.06 −0.09 −0.19
Word affective go/no-go Affective bias (RT) −0.11 −0.01 −0.01 1.23
Emotional memory Retrieval affective bias −0.01 0.04 0.04 −0.33
REWARD/MOTIVATION
Reinforcement learning Win learning rate −0.02 0.02 0.13 −1.69
Loss learning rate −0.07 −0.04 0.03 −0.64
Monetary incentive reward Win-neutral RT 0.01 −0.12 −0.04 −1.83
Loss-neutral RT 0.02 −0.00 −0.04 −1.74
Cambridge gambling task Win risk adjustment −0.12 0.16 0.08 0.42
Loss risk adjustment −0.25* 0.27* 0.18 1.21
Progressive ratioa Breakpoint 0.07 −0.07 −0.09 −0.35
Post reinforcement pause 0.23 −0.32 −0.03 −0.67
IMPULSIVITY
4 CSRTTb Motivational index −0.14 0.01 0.14 0.31
Delay discounting Delay discounting −0.16 0.27* 0.24* 1.28
Probability discounting 0.03 −0.05 0.11 0.69
SOCIAL COGNITION
Moral emotions Agent guilt ratings 0.05 −0.02 −0.04 −4.02*$
Feeling bad ratings −0.12 0.11 0.14 1.96
Information preference Proportion thoughts −0.08 −0.05 −0.13 0.84
Affective bias in outcome 0.07 0.01 0.07 −0.54
Prisoners dilemma Average steal rate −0.02 0.04 0.06 0.57
Ultimatum game Risk adjustment −0.13 0.13 0.09 −0.66
Average value of offers proposed 0.19 −0.08 −0.09 0.22
Inference task Effect of jelly probability −0.16 0.15 0.18 −0.38
Effect of computer choice −0.19 0.15 0.18 −0.70

Results show Pearson correlations (r) or t-statistics (t) from independent t-test.

*

p < 0.002; N = 200.

a

Only 78 participants were included in the correlation analyses for the Progressive ratio task due to an update to the task part way through the study.

b

Only 175 participants were included in the correlation analyses for the 4CSRTT due to technical failure.

$

Females showed greater guilt ratings.

Discussion

Neuropsychological test batteries are vital tools for assessing the efficacy of treatment in neuropsychiatric disorders. In order to provide valid assessments of cognitive function, a neuropsychological test battery must possess good test retest reliability and examine a variety of cognitive functions with little redundancy. A further requirement of a test battery specifically assessing emotional and social function is that it should be (at least to some extent) independent of cognitive ability or IQ. In this paper we have presented data from 200 participants' performance to demonstrate that these requirements are met by the EMOTICOM neuropsychological test battery. This battery draws upon adaptations of pre-existing tasks as well as novel tasks in order to provide a comprehensive assessment of emotion processing, rewards and motivation, impulsivity and social cognition.

An exploratory factor analysis identified 11 factors, many of them loading onto a single task. Not all the factors are readily explicable and factors including variables with poor reliability should be viewed with considerable caution. We therefore do not attempt to draw conclusions about the meaning of individual factors. Rather we suggest that the central conclusion is simply that the tasks measure multiple constructs and therefore the battery has little redundancy. Our hypothesis of a four factor solution was categorially disproved suggesting that our prior operational concept of four domains was an over-simplification. This highlights the importance of administering multiple tests in order to assess these “hot” cognitive processes. Various reviews and meta-analyses have identified multiple domains of social cognition (Green and Leitman, 2008; Savla et al., 2012), however existing standardized batteries such as the MATRICS Consensus Cognitive Battery (MCCB; www.matricsinc.org) and CANTAB contain only one task targeting social cognition. The results presented here clearly indicate that there are different components of “hot” cognition that load onto multiple factors and therefore cannot be captured by a single test. Therefore, the EMOTICOM test battery provides a more comprehensive assessment of performance in a variety of affective processes and represents a significant advance over batteries including only a single test.

The majority of EMOTICOM tasks also showed moderate to excellent test-retest reliability. This is extremely important for assessing the efficacy of treatments and interventions, where it is important that differences in task performance can be attributed to effects of the interventions rather than methodological issues or random fluctuations. Furthermore, we demonstrate that our “hot” cognitive tasks have comparable retest reliability to traditional “cold” cognitive tasks (e.g., Lowe and Rabbitt, 1998). However, reliability of the reinforcement learning outcome variable was poor, consistent with previous observations that learning and memory tasks often do not exhibit good re-test reliability (Lowe and Rabbitt, 1998; Dikmen et al., 1999). Learning on these tasks transfers from the first session to the second. Such learning transfer results in significantly improved scores and lower variability at session 2, as we observed here. Given this poor reliability, the EMOTICOM reinforcement learning task could potentially be improved by creating parallel versions using different stimuli, although participants are still likely to be able to generalize rule-learning from the first session. Reliability of bias measures in the Affective Go No Go and Monetary Incentive Reward tasks were also poor. Bias reliability scores in reaction times are often reported to be much lower than mean RTs from each condition (Eide et al., 2002; Strauss et al., 2005; Brown et al., 2014) and our results are therefore comparable with previous studies. Poor test-retest reliability on specific tasks suggests caution in using these measures in longitudinal contexts with healthy volunteers, however it does not preclude the use of these tasks in between-group studies with patient populations.

The majority of EMOTICOM tasks were not strongly correlated with demographic factors such as age, years in education or IQ suggesting that performance of these tasks is not dependent upon general intellectual function. There are a few exceptions: the risk adjustment measure from the loss condition in the adapted Cambridge Gambling Task and the delay discounting measures were correlated with IQ, with delay discounting also being correlated with years in education. Previous studies have also suggested that gambling (Demaree et al., 2010; Webb et al., 2014) and delay discounting (Shamosh and Gray, 2008) correlate with intelligence. Therefore, it is recommended that studies using these measures take particular care to control for IQ and years of education. Interestingly we observed emotional bias measures in the face and eyes emotional recognition task to be significantly correlated with age, such that biases became more positive with increasing age. This finding supports a line of research that has recently gathered momentum, with many recent studies demonstrating that people attend to and remember positive information more as they get older (e.g., Mather and Carstensen, 2003; Reed and Carstensen, 2012). In spite of a prevailing view that hot cognitive tests are dependent on gender, we only observed a significant effect of gender in the Moral Emotions Task, whereby females show greater guilt ratings compared to males. This is in line with existing meta-analyses showing that women tend to experience negative emotions, such as guilt, more intensely than men (Else-Quest et al., 2012). This task may therefore be useful in understanding gender differences in treatment outcomes, particularly in terms of self-blame biases and their suggested link to a vulnerability to depression (Green et al., 2013).

Limitations

The ethnic characteristics of our sample of 200 participants was representative of the UK demographic (Office for National Statistics, 2011). Nevertheless, caution is recommended in generalizing these findings across cultures. Evidence suggests that cultural variations are evident in affective cognition. For example cultural variations have been observed in emotional facial recognition (Prado et al., 2014) economical games such as the Ultimatum Game and Prisoners' Dilemma (Oosterbeek et al., 2004; Wong and Hong, 2005) and arguably moral judgment (Gibbs et al., 2007). Such differences observed in performance across cultures suggest care in generalizing performance on UK validated and standardized tasks to other cultures. Another limitation is that we were not able to enter all the task variables into the factor analysis due to the reduced number of participants who completed some of the tasks. For instance, the progressive ratio parameters were improved part way through the study and so data were only available from 78 participants. Similarly, only a subset of participants completed the 4CSRTT. Therefore, in order to increase power and retain the full participant sample, the decision was made to omit these measures from factor analysis. A limitation of the test-retest reliability component was that we only assessed reliability over a short duration; in future it will important to assess longer durations to determine the potential value of the tasks in different intervention contexts.

In summary, we have demonstrated the potential power of the EMOTICOM test battery for the assessment of affective cognitive function. We have shown that affective cognition is far from a unitary construct, implying that assessment of multiple aspects of affective cognition is required. Our 16 task battery has little redundancy from the 11 factor underlying structure. We have also demonstrated that the majority of tasks have moderate to excellent test-retest reliability and are not strongly correlated with demographic factors such as IQ. We therefore conclude that the EMOTICOM test battery meets certain key criteria for a useful and valid tool with potential utility in clinical trials and studies investigating psychiatric disorders and relevant treatment interventions.

Important future directions include validation in patients and validation in intervention studies in both healthy controls and patients in order to further investigate the utility of EMOTICOM test battery, and diagnosis-appropriate subsets of tasks, as an investigative tool in mental health research. This will enable us assess which tasks are most valid, sensitive and reliable for use in particular patient populations and which can be used as outcome measures in intervention trials.

Funding

This work was supported by the MRC under Grant MR/J011894/1.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The reviewer YW and the handling Editor declared their shared affiliation, and the handling Editor states that the process nevertheless met the standards of a fair and objective review.

Acknowledgments

We thank our colleagues at the University of Manchester, University of Cambridge, University College London and Kings College London for their contribution to discussions. In particular we thank Dr Vincent Valton (UCL) and Dr Sanja Abbott (University of Cambridge) for assistance with analyses. We also thank Dr Jonathan Peirce (University of Nottingham) and Dr Martyn McFarquar (University of Manchester) for their guidance and expertise in PsychoPy.

References

  1. Agay N., Kron S., Carmel Z., Mendlovic S., Levkovitz Y. (2008). Ultimatum bargaining behavior of people affected by schizophrenia. Psychiatry Res. 157, 39–46. 10.1016/j.psychres.2006.03.026 [DOI] [PubMed] [Google Scholar]
  2. Andari E., Duhamel J.-R., Zalla T., Herbrecht E., Leboyer M., Sirigu A. (2010). Promoting social behavior with oxytocin in high-functioning autism spectrum disorders. Proc. Natl. Acad. Sci. U.S.A. 107, 4389–4394. 10.1073/pnas.0910249107 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Aron A. R., Poldrack R. A. (2005). The cognitive neuroscience of response inhibition: relevance for genetic research in attention-deficit/hyperactivity disorder. Biol. Psychiatry 57, 1285–1292. 10.1016/j.biopsych.2004.10.026 [DOI] [PubMed] [Google Scholar]
  4. Baert S., De Raedt Schacht R. R., Koster E. H. W. (2010). Attentional bias training in depression: therapeutic effects depend on depression severity. J. Behav. Ther. Exp. Psychiatry 41, 265–274. 10.1016/j.jbtep.2010.02.004 [DOI] [PubMed] [Google Scholar]
  5. Baron-Cohen S., Wheelwright S., Hill J., Raste Y., Plumb I. (2001). The ‘reading the mind in the eyes’ test revised version: a study with normal adults, and adults with asperger syndrome or high-functioning autism. J. Child Psychol. Psychiatry 42, 241–251. 10.1111/1469-7610.00715 [DOI] [PubMed] [Google Scholar]
  6. Bechara A., Damasio A. R., Damasio H., Anderson S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50, 7–15. 10.1016/0010-0277(94)90018-3 [DOI] [PubMed] [Google Scholar]
  7. Birrell J. M., Brown V. J. (2000). Medial frontal cortex mediates perceptual attentional set shifting in the rat. J. Neurosci. 20, 4320–4324. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Blair R. J. (1995). A cognitive developmental approach to mortality: investigating the psychopath. Cognition 57, 1–29. 10.1016/0010-0277(95)00676-P [DOI] [PubMed] [Google Scholar]
  9. Bloom D. E., Cafiero E., Jané-Llopis E., Abrahams-Gessel S., Bloom L. R., Fathima S., et al. (2012). The Global Economic Burden of Noncommunicable Diseases (No. 8712). Program on the Global Demography of Aging. Available online at: https://ideas.repec.org/p/gdm/wpaper/8712.html
  10. Bora E., Yucel M., Pantelis C. (2009). Theory of mind impairment in schizophrenia: meta-analysis. Schizophr. Res. 109, 1–9. 10.1016/j.schres.2008.12.020 [DOI] [PubMed] [Google Scholar]
  11. Bouhuys A. L., Geerts E., Gordijn M. (1999). Gender-specific mechanisms associated with outcome of depression: perception of emotions, coping and interpersonal functioning. Psychiatry Res. 85, 247–261. 10.1016/S0165-1781(99)00003-7 [DOI] [PubMed] [Google Scholar]
  12. Bradshaw C. M., Killeen P. R. (2012). A theory of behaviour on progressive ratio schedules, with applications in behavioural pharmacology. Psychopharmacology 222, 549–564. 10.1007/s00213-012-2771-4 [DOI] [PubMed] [Google Scholar]
  13. Brittlebank A. D., Scott J., Williams J. M., Ferrier I. N. (1993). Autobiographical memory in depression: state or trait marker? Br. J. Psychiatry 162, 118–121. 10.1192/bjp.162.1.118 [DOI] [PubMed] [Google Scholar]
  14. Brown H. M., Eley T. C., Broeren S., MacLeod C., Rinck M., Hadwin J. A., et al. (2014). Psychometric properties of reaction time based experimental paradigms measuring anxiety-related information-processing biases in children. J. Anxiety Disord. 28, 97–107. 10.1016/j.janxdis.2013.11.004 [DOI] [PubMed] [Google Scholar]
  15. Chandler R. A., Wakeley J., Goodwin G. M., Rogers R. D. (2009). Altered risk-aversion and risk-seeking behavior in bipolar disorder. Biol. Psychiatry 66, 840–846. 10.1016/j.biopsych.2009.05.011 [DOI] [PubMed] [Google Scholar]
  16. Ciaramelli E., Muccioli M., Ladavas E., di Pellegrino G. (2007). Selective deficit in personal moral judgment following damage to ventromedial prefrontal cortex. Soc. Cogn. Affect. Neurosci. 2, 84–92. 10.1093/scan/nsm001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Combs D. R., Adams S. D., Penn D. L., Roberts D., Tiegreen J., Stem P. (2007). Social cognition and interaction training (SCIT) for inpatients with schizophrenia spectrum disorders. Prelim. Find. 91, 112–116. 10.1016/j.schres.2006.12.010 [DOI] [PubMed] [Google Scholar]
  18. Cools R., Clark L., Owen A. M., Robbins T. W. (2002). Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J. Neurosci. 22, 4563–4567. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Crockett M. J. (2009). The neurochemistry of fairness. Ann. N.Y. Acad. Sci. 1167, 76–86. 10.1111/j.1749-6632.2009.04506.x [DOI] [PubMed] [Google Scholar]
  20. Daw N. D., O'Doherty J. P., Dayan P., Seymour B., Dolan R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature 441, 876–879. 10.1038/nature04766 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Demaree H. A., Burns K. J., DeDonno M. A. (2010). Intelligence, but not emotional intelligence, predicts iowa gambling task performance. Intelligence 38, 249–254. 10.1016/j.intell.2009.12.00426678400 [DOI] [Google Scholar]
  22. Derogatis L. R., Melisaratos N. (1983). The brief symptom inventory: an introductory report. Psychol. Med. 13, 595–605. 10.1017/S0033291700048017 [DOI] [PubMed] [Google Scholar]
  23. Dikmen S. S., Heaton R. K., Grant I., Temkin N. R. (1999). TestReitan neuropsychological test battery. J. Inter. Neuropsychol. Soc. 5, 346–356. [PubMed] [Google Scholar]
  24. Eack S. M., Greeno C. G., Pogue-Geile M. F., Newhill C. E., Hogarty G. E., Keshavan M. S. (2010). Assessing social-cognitive deficits in schizophrenia with the mayer-salovey-caruso emotional intelligence test. Schizophr. Bull. 36, 370–380. 10.1093/schbul/sbn091 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Eide P., Kemp A., Silberstein R. B., Nathan P. J., Stough C. (2002). Test-Retest reliability of the emotional stroop task: examining the paradox of measurement change. J. Psychol. 136, 514–520. 10.1080/00223980209605547 [DOI] [PubMed] [Google Scholar]
  26. Elliott R., Zahn R., William Deakin J. F., Anderson I. M. (2011). Affective cognition and its disruption in mood disorders. Neuropsychopharmacology 36, 153–182. 10.1038/npp.2010.77 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Else-Quest N. M., Higgins A., Allison C., Morton L. C. (2012). Gender differences in self-conscious emotional experience: a meta-analysis. Psychol. Bull. 138, 947–981. 10.1037/a0027930 [DOI] [PubMed] [Google Scholar]
  28. Erickson K., Drevets W. C., Clark L., Cannon D. M., Bain E. E., Zarate C. A., et al. (2005). Mood-congruent bias in affective go/no-go performance of unmedicated patients with major depressive disorder. Am. J. Psychiatry 162, 2171–2173. 10.1176/appi.ajp.162.11.2171 [DOI] [PubMed] [Google Scholar]
  29. Ersche K. D., Sahakian B. J. (2007). The neuropsychology of amphetamine and opiate dependence: implications for treatment. Neuropsychol. Rev. 17, 317–336. 10.1007/s11065-007-9033-y [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Fehr E., Camerer C. F. (2007). Social neuroeconomics: the neural circuitry of social preferences. Trends Cogn. Sci. 11, 419–427. 10.1016/j.tics.2007.09.002 [DOI] [PubMed] [Google Scholar]
  31. Feifel D., Macdonald K., Cobb P., Minassian A. (2012). adjunctive intranasal oxytocin improves verbal memory in people with schizophrenia. Schizophr. Res. 139, 207–210. 10.1016/j.schres.2012.05.018 [DOI] [PubMed] [Google Scholar]
  32. Fett A.-K. J., Viechtbauer W., Dominguez M. G., Penn D. L., van Os J., Krabbendam L. (2011). The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: a meta-analysis. Neurosci. Biobehav. Rev. 35, 573–588. 10.1016/j.neubiorev.2010.07.001 [DOI] [PubMed] [Google Scholar]
  33. Fineberg N. A., Haddad P. M., Carpenter L., Gannon B., Sharpe R., Young A. H., et al. (2013). The size, burden and cost of disorders of the brain in the UK. J. Psychopharmacol. (Oxford). 27, 761–770. 10.1177/0269881113495118 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Fleis J. L., Levin B., Paik M. C. (2003). Statistical Methods for Rates and Proportions. Hoboken, NJ: John Wiley & Sons. [Google Scholar]
  35. Frith C. D., Corcoran R. (1996). Exploring'theory of mind'in people with schizophrenia. Psychol. Med. 26, 521–530. 10.1017/S0033291700035601 [DOI] [PubMed] [Google Scholar]
  36. Frith U., Frith C. D. (2003). Development and neurophysiology of mentalizing. Philos. Trans. R. Soc. Lond. Ser. B 358, 459–473. 10.1098/rstb.2002.1218 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Gibbs J. C., Basinger K. S., Grime R. L., Snarey J. R. (2007). Moral judgment development across cultures: revisiting kohlberg's universality claims. Dev. Rev. 27, 443–500. 10.1016/j.dr.2007.04.001 [DOI] [Google Scholar]
  38. Green L., Myerson J., Lichtman D., Rosen S., Fry A. (1996). Temporal discounting in choice between delayed rewards: the role of age and income. Psychol. Aging 11, 79–84. 10.1037/0882-7974.11.1.79 [DOI] [PubMed] [Google Scholar]
  39. Green M. F., Leitman D. I. (2008). Social cognition in schizophrenia. Schizophr. Bull. 34, 670–672. 10.1093/schbul/sbn045 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Green M. F., Penn D. L., Bentall R., Carpenter W. T., Gaebel W., Gur R. C., et al. (2008). Social cognition in schizophrenia: an nimh workshop on definitions, assessment, and research opportunities. Schizophr. Bull. 34, 1211–1220. 10.1093/schbul/sbm145 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Green S., Lambon Ralph M. A., Moll J., Zakrzewski J., William Deakin J. F., Grafman J., et al. (2013). The neural basis of conceptual-emotional integration and its role in major depressive disorder. Soc. Neurosci. 8, 417–433. 10.1080/17470919.2013.810171 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Hamilton J. P., Gotlib I. H. (2008). Neural substrates of increased memory sensitivity for negative stimuli in major depression. Biol. Psychiatry 63, 1155–1162. 10.1016/j.biopsych.2007.12.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Happé F., Frith U. (1996). The neuropsychology of autism. Brain 119, 1377–1400. 10.1093/brain/119.4.1377 [DOI] [PubMed] [Google Scholar]
  44. Harmer C. J., Goodwin G. M., Cowen P. J. (2009). Why do antidepressants take so long to work? a cognitive neuropsychological model of antidepressant drug action. Br. J. Psychiatry 195, 102–108. 10.1192/bjp.bp.108.051193 [DOI] [PubMed] [Google Scholar]
  45. Harmer C. J., Cowen P. J., Goodwin G. M. (2011). Efficacy markers in depression. J. Psychopharmacol. 25, 1148–1158. 10.1177/0269881110367722 [DOI] [PubMed] [Google Scholar]
  46. Herbener E. S., Song W., Khine T. T., Sweeney J. A. (2008). What aspects of emotional functioning are impaired in schizophrenia? Schizophr. Res. 98, 239–246. 10.1016/j.schres.2007.06.025 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Hodos W. (1961). Progressive ratio as a measure of reward strength. Science 134, 943–944. 10.1126/science.134.3483.943 [DOI] [PubMed] [Google Scholar]
  48. Hutton S. B., Murphy F. C., Joyce E. M., Rogers R. D., Cuthbert I. T., Robbins T. W., et al. (2002). Decision making deficits in patients with first-episode and chronic schizophrenia. Schizophr. Res. 55, 249–257. 10.1016/S0920-9964(01)00216-X [DOI] [PubMed] [Google Scholar]
  49. Insel T., Cuthbert B., Garvey M., Heinssen R., Pine D. S., Quinn K., et al. (2010). Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am. J. Psychiatry 167, 748–751. 10.1176/appi.ajp.2010.09091379 [DOI] [PubMed] [Google Scholar]
  50. Kaladjian A., Jeanningros R., Azorin J.-M., Grimault S., Anton J.-L., Mazzola-Pomietto P. (2007). Blunted activation in right ventrolateral prefrontal cortex during motor response inhibition in schizophrenia. Schizophr. Res. 97, 184–193. 10.1016/j.schres.2007.07.033 [DOI] [PubMed] [Google Scholar]
  51. King-Casas B., Sharp C., Lomax-Bream L., Lohrenz T., Fonagy P., Montague P. R. (2008). The rupture and repair of cooperation in borderline personality disorder. Science (N.Y.) 321, 806–810. 10.1126/science.1156902 [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. King-Casas B., Tomlin D., Anen C., Camerer C. F., Quartz S. R., Montague P. R. (2005). Getting to know you: reputation and trust in a two-person economic exchange. Science 308, 78–83. 10.1126/science.1108062 [DOI] [PubMed] [Google Scholar]
  53. Kirby K. N. (2009). One-year temporal stability of delay-discount rates. Psychon. Bull. Rev. 16, 457–462. 10.3758/PBR.16.3.457 [DOI] [PubMed] [Google Scholar]
  54. Kishida K. T., King-Casas B., Montague P. R. (2010). Neuroeconomic approaches to mental disorders. Neuron 67, 543–554. 10.1016/j.neuron.2010.07.021 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Knutson B., Adams C. M., Fong G. W., Hommer D. (2001). Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J. Neurosci. 21, RC159. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Knutson B., Bjork J. M., Fong G. W., Hommer D., Mattay V. S., Weinberger D. R. (2004). Amphetamine modulates human incentive processing. Neuron 43, 261–269. 10.1016/j.neuron.2004.06.030 [DOI] [PubMed] [Google Scholar]
  57. Koenigs M., Kruepke M., Newman J. P. (2010). Economic decision-making in psychopathy: a comparison with ventromedial prefrontal lesion patients. Neuropsychologia 48, 2198–2204. 10.1016/j.neuropsychologia.2010.04.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Koenigs M., Young L., Adolphs R., Tranel D., Cushman F., Hauser M., et al. (2007). Damage to the prefrontal cortex increases utilitarian moral judgements. Nature 446, 908–911. 10.1038/nature05631 [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Ladouceur C. D., Dahl R. E., Williamson D. E., Birmaher B., Axelson D. A., Ryan N. D., et al. (2006). Processing emotional facial expressions influences performance on a Go/NoGo task in pediatric anxiety and depression. J. Child Psychol. Psychiatry 47, 1107–1115. 10.1111/j.1469-7610.2006.01640.x [DOI] [PubMed] [Google Scholar]
  60. Lovell D. M., Mark J., Williams G., Hill A. B. (1997). Selective processing of shape−related words in women with eating disorders, and those who have recovered. Br. J. Clin. Psychol. 36, 421–432. 10.1111/j.2044-8260.1997.tb01249.x [DOI] [PubMed] [Google Scholar]
  61. Lowe C., Rabbitt P. (1998). Test\re-Test reliability of the CANTAB and ISPOCD neuropsychological batteries: theoretical and practical issues. Neuropsychologia 36, 915–923. 10.1016/S0028-3932(98)00036-0 [DOI] [PubMed] [Google Scholar]
  62. Malloy-Diniz L., Fuentes D., Borges Leite W., Correa H., Bechara A. (2007). Impulsive behavior in adults with attention deficit/hyperactivity disorder: characterization of attentional, motor and cognitive impulsiveness. J. Inter. Neuropsychol. Soc. 13, 693–698. 10.1017/s1355617707070889 [DOI] [PubMed] [Google Scholar]
  63. Mather M., Carstensen L. L. (2003). Aging and attentional biases for emotional faces. Psychol. Sci. 14, 409–415. 10.1111/1467-9280.01455 [DOI] [PubMed] [Google Scholar]
  64. Mayer J. D., Salovey P. (1997). What is emotional intelligence?, in Emotional Development and Emotional Intelligence: Educational Implications, eds Salovey P., Sluyter D. J. (New York, NY: Harper Collins; ), 3–34. [Google Scholar]
  65. Mayer J. D., Salovey P., Caruso D. R., Sitarenios G. (2003). Measuring emotional intelligence with the MSCEIT V2.0. Emotion 3, 97–105. [DOI] [PubMed] [Google Scholar]
  66. Mazur J. E. (1987). An adjusting procedure for studying delayed reinforcement. Qt. Analys. Behav. 5, 55–73. [Google Scholar]
  67. Miller G. (2005). Economic game shows how the brain builds trust. Science 308:36. 10.1126/science.308.5718.36a [DOI] [PubMed] [Google Scholar]
  68. Mobbs O., Iglesias K., Golay A., Van der Linden M. (2011). Cognitive deficits in obese persons with and without binge eating disorder. Investigation using a mental flexibility task. Appetite 57, 263–271. 10.1016/j.appet.2011.04.023 [DOI] [PubMed] [Google Scholar]
  69. Mogg K., Bradley B. P. (2002). Selective orienting of attention to masked threat faces in social anxiety. Behav. Res. Ther. 40, 1403–1414. 10.1016/S0005-7967(02)00017-7 [DOI] [PubMed] [Google Scholar]
  70. Mogg K., Bradley B. P., Williams R. (1995). Attentional bias in anxiety and depression: the role of awareness. Br. J. Clin. Psychol. 34, 17–36. 10.1111/j.2044-8260.1995.tb01434.x [DOI] [PubMed] [Google Scholar]
  71. Moll J., de Oliveira-Souza R. (2007). Moral judgments, emotions and the utilitarian brain. Trends Cogn. Sci. 11, 319–321. 10.1016/j.tics.2007.06.001 [DOI] [PubMed] [Google Scholar]
  72. Moran J. M., Young L. L., Saxe R., Lee S. M., O'Young D., Mavros P. L., et al. (2011). Impaired theory of mind for moral judgment in high-functioning autism. Proc. Natl. Acad. Sci. U.S.A. 108, 2688–2692. 10.1073/pnas.1011734108 [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Murphy F. C., Michael A., Robbins T. W., Sahakian B. J. (2003). Neuropsychological impairment in patients with major depressive disorder: the effects of feedback on task performance. Psychol. Med. 33, 455–467. 10.1017/S0033291702007018 [DOI] [PubMed] [Google Scholar]
  74. Murphy F. C., Sahakian B. J., Rubinsztein J. S., Michael A., Rogers R. D., Robbins T. W., et al. (1999). Emotional bias and inhibitory control processes in mania and depression. Psychol. Med. 29, 1307–1321. 10.1017/S0033291799001233 [DOI] [PubMed] [Google Scholar]
  75. Murphy F. C., Rubinsztein J. S., Michael A., Rogers R. D., Robbins T. W., Paykel E. S., et al. (2001). Decision-making cognition in mania and depression. Psychol. Med. 31, 679–693. 10.1017/S0033291701003804 [DOI] [PubMed] [Google Scholar]
  76. Murray G. K., Corlett P. R., Clark L., Pessiglione M., Blackwell A. D., Honey G., et al. (2007). Substantia nigra/ventral tegmental reward prediction error disruption in psychosis. Mol. Psychiatry 13, 267–276. 10.1038/sj.mp.4002058 [DOI] [PMC free article] [PubMed] [Google Scholar]
  77. Office for National Statistics (2011). Census: Ethnic Group, Local Authorities in England and Wales.
  78. Ohmura Y., Takahashi T., Kitamura N., Wehr P. (2006). Three-month stability of delay and probability discounting measures. Exp. Clin. Psychopharmacol. 14, 318. 10.1037/1064-1297.14.3.318 [DOI] [PubMed] [Google Scholar]
  79. Oosterbeek H., Sloof R., van de Kuilen G. (2004). Cultural differences in ultimatum game experiments: evidence from a meta-analysis. Exp. Econ. 7, 171–188. 10.1023/B:EXEC.0000026978.14316.74 [DOI] [Google Scholar]
  80. Owen A. M., McMillan K. M., Laird A. R., Bullmore E. D. (2005). N-Back working memory paradigm: a meta-analysis of normative functional neuroimaging studies. Hum. Brain Mapp. 25, 46–59. 10.1002/hbm.20131 [DOI] [PMC free article] [PubMed] [Google Scholar]
  81. Owen A. M., Roberts A. C., Polkey C. E., Sahakian B. J., Robbins T. W. (1991). Extra-dimensional versus intra-dimensional set shifting performance following frontal lobe excisions, temporal lobe excisions or amygdalo-hippocampectomy in man. Neuropsychologia 29, 993–1006. 10.1016/0028-3932(91)90063-E [DOI] [PubMed] [Google Scholar]
  82. Park S. Q., Kahnt T., Beck A., Cohen M. X., Dolan R. J., Wrase J., et al. (2010). Prefrontal cortex fails to learn from reward prediction errors in alcohol dependence. J. Neurosci. 30, 7749–7753. 10.1523/JNEUROSCI.5587-09.2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  83. Peirce J. W. (2007). PsychoPy—psychophysics software in python. J. Neurosci. Methods 162, 8–13. 10.1016/j.jneumeth.2006.11.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  84. Pessiglione M., Seymour B., Flandin G., Dolan R. J., Frith C. D. (2006). Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442, 1042–1045. 10.1038/nature05051 [DOI] [PMC free article] [PubMed] [Google Scholar]
  85. Pomarol-Clotet E., Hynes F., Ashwin C., Bullmore E. T., McKenna P. J., Laws K. R. (2010). Facial emotion processing in schizophrenia: a non-specific neuropsychological deficit? Psychol. Med. 40, 911–919. 10.1017/S0033291709991309 [DOI] [PubMed] [Google Scholar]
  86. Posner M. I. (1980). Orienting of attention. Q. J. Exp. Psychol. 32, 3–25. 10.1080/00335558008248231 [DOI] [PubMed] [Google Scholar]
  87. Prado C., Mellor D., Linda Byrne K., Wilson C., Xu X., Liu H. (2014). Facial emotion recognition: a cross-cultural comparison of chinese, chinese living in Australia, and Anglo-Australians. Motiv. Emot. 38, 420–428. 10.1007/s11031-013-9383-0 [DOI] [Google Scholar]
  88. Pulcu E., Thomas E. J., Trotter P. D., McFarquhar M., Juhasz G., Sahakian B. J., et al. (2015). Social-economical decision making in current and remitted major depression. Psychol. Med. 45, 1301–1313. 10.1017/S0033291714002414 [DOI] [PubMed] [Google Scholar]
  89. Reed A. E., Carstensen L. L. (2012). The Theory behind the age-related positivity effect. Front. Psychol. 3:339. 10.3389/fpsyg.2012.00339 [DOI] [PMC free article] [PubMed] [Google Scholar]
  90. Reynolds B., Schiffbauer R. (2004). Measuring state changes in human delay discounting: an experiential discounting task. Behav. Processes 67, 343–356. 10.1016/S0376-6357(04)00140-8 [DOI] [PubMed] [Google Scholar]
  91. Richards J. B., Zhang L., Mitchell S. H., de Wit H. (1999). Delay or probability discounting in a model of impulsive behavior: effect of alcohol. J. Exp. Anal. Behav. 71, 121–143. 10.1901/jeab.1999.71-121 [DOI] [PMC free article] [PubMed] [Google Scholar]
  92. Rilling J. K., Glenn A. L., Jairam M. R., Pagnoni G., Goldsmith D. R., Elfenbein H. A., et al. (2015). Neural correlates of social cooperation and non-cooperation as a function of psychopathy. Biol. Psychiatry 61, 1260–1271. 10.1016/j.biopsych.2006.07.021 [DOI] [PubMed] [Google Scholar]
  93. Roane H. S. (2008). On the applied use of progressive-ratio schedules of reinforcement. J. Appl. Behav. Anal. 41, 155–161. 10.1901/jaba.2008.41-155 [DOI] [PMC free article] [PubMed] [Google Scholar]
  94. Robbins T. W. (2002). The 5-choice serial reaction time task: behavioural pharmacology and functional neurochemistry. Psychopharmacology (Berl). 163, 362–380. 10.1007/s00213-002-1154-7 [DOI] [PubMed] [Google Scholar]
  95. Robbins T. W., James M., Owen A. M., Sahakian B. J., Lawrence A. D., McInnes L., et al. (1998). A study of performance on tests from the CANTAB battery sensitive to frontal lobe dysfunction in a large sample of normal volunteers: implications for theories of executive functioning and cognitive aging. Cambridge neuropsychological test automated batte. J. Int. Neuropsychol. Soc. 4, 474–490. 10.1017/S1355617798455073 [DOI] [PubMed] [Google Scholar]
  96. Robbins T. W., James M., Owen A. M., Sahakian B. J., McInnes L., Rabbitt P. (1994). Cambridge neuropsychological test automated battery (CANTAB): a factor analytic study of a large sample of normal elderly volunteers. PG - 266-81. Dementia 5, 266–81. 10.1159/000106735 [DOI] [PubMed] [Google Scholar]
  97. Roberts A. C., Robbins T. W., Everitt B. J. (1988). The effects of intradimensional and extradimensional shifts on visual discrimination learning in humans and non-human primates. Q. J. Exp. Psychol. 40, 321–341. [PubMed] [Google Scholar]
  98. Rogers R. D., Blackshaw A. J., Middleton H. C., Matthews K., Hawtin K., Crowley C., et al. (1999). Tryptophan Depletion impairs stimulus-reward learning while methylphenidate disrupts attentional control in healthy young adults: implications for the monoaminergic basis of impulsive behaviour. Psychopharmacology (Berl). 146, 482–491. 10.1007/PL00005494 [DOI] [PubMed] [Google Scholar]
  99. Rogers R. D., Tunbridge E. M., Bhagwagar Z., Drevets W. C., Sahakian B. J., Carter C. S. (2003). Tryptophan depletion alters the decision-making of healthy volunteers through altered processing of reward cues. Neuropsychopharmacology 28, 153–162. 10.1038/sj.npp.1300001 [DOI] [PubMed] [Google Scholar]
  100. Roiser J. P., Elliott R., Sahakian B. J. (2012). Cognitive mechanisms of treatment in depression. Neuropsychopharmacology 37, 117–136. 10.1038/npp.2011.183 [DOI] [PMC free article] [PubMed] [Google Scholar]
  101. Roiser J. P., Stephan K. E., Den Ouden H. E. M., Barnes T. R. E., Friston K. J., Joyce E. M. (2009a). Do patients with schizophrenia exhibit aberrant salience? Psychol. Med. 39, 199. 10.1017/S0033291708003863 [DOI] [PMC free article] [PubMed] [Google Scholar]
  102. Roiser J. P., Blackwell A. D., Cools R., Clark L., Rubinsztein D. C., Robbins T. W., et al. (2006). Serotonin transporter polymorphism mediates vulnerability to loss of incentive motivation following acute tryptophan depletion. Neuropsychopharmacology 31, 2264–2272. 10.1038/sj.npp.1301084 [DOI] [PMC free article] [PubMed] [Google Scholar]
  103. Roiser J. P., Cannon D. M., Gandhi S. K., Tavares J. T., Erickson K., Wood S., et al. (2009b). Hot and cold cognition in unmedicated depressed subjects with bipolar disorder. Bipolar Disord. 11, 178–189. 10.1111/j.1399-5618.2009.00669.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  104. Roiser J. P., Sahakian B. J. (2013). Hot and cold cognition in depression. CNS Spectr. 18, 139–149. 10.1017/S1092852913000072 [DOI] [PubMed] [Google Scholar]
  105. Sanislow C. A., Pine D. S., Quinn K. J., Kozak M. J., Garvey M. A., Heinssen R. K., et al. (2010). Developing constructs for psychopathology research: research domain criteria. J. Abnorm. Psychol. 119, 631–639. 10.1037/a0020909 [DOI] [PubMed] [Google Scholar]
  106. Savla G. N., Vella L., Armstrong C. C., Penn D. L., Twamley E. W. (2012). Deficits in domains of social cognition in schizophrenia: a meta-analysis of the empirical evidence. Schizophr. Bull. 39, 979–992. 10.1093/schbul/sbs080 [DOI] [PMC free article] [PubMed] [Google Scholar]
  107. Scarna A. S., McTavish F. B., Cowen P. J., Goodwin G. M., Rogers R. D. (2005). The effects of a branched chain amino acid mixture supplemented with tryptophan on biochemical indices of neurotransmitter function and decision-making. Psychopharmacology (Berl). 179, 761–768. 10.1007/s00213-004-2105-2 [DOI] [PubMed] [Google Scholar]
  108. Scheres A., Milham M. P., Knutson B., Castellanos F. X. (2007). Ventral striatal hyporesponsiveness during reward anticipation in attention-deficit/hyperactivity disorder. Biol. Psychiatry 61, 720–724. 10.1016/j.biopsych.2006.04.042 [DOI] [PubMed] [Google Scholar]
  109. Seres I., Unoka Z., Keri S. (2009). The broken trust and cooperation in borderline personality disorder. Neuroreport 20, 388–392. 10.1097/WNR.0b013e328324eb4d [DOI] [PubMed] [Google Scholar]
  110. Seymour K. E., Reinblatt S. P., Benson L., Carnell S. (2015). Overlapping neurobehavioral circuits in adhd, obesity, and binge eating: evidence from neuroimaging research. CNS Spectr. 20, 401–411. 10.1017/S1092852915000383 [DOI] [PMC free article] [PubMed] [Google Scholar]
  111. Shamosh N. A., Gray J. R. (2008). Delay discounting and intelligence: a meta-analysis. Intelligence 36, 289–305. 10.1016/j.intell.2007.09.004 [DOI] [Google Scholar]
  112. Sheehan D. V., Lecrubier Y., Sheehan K. H., Amorim P., Janavs J., Weiller E., et al. (1998). The mini-international neuropsychiatric interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J. Clin. Psychiatry 20, 22–33. [PubMed] [Google Scholar]
  113. Strauss G. P., Allen D. N., Jorgensen M. L., Cramer S. L. (2005). Test-retest reliability of standard and emotional stroop tasks: an investigation of color-word and picture-word versions. Assessment 12, 330–337. 10.1177/1073191105276375 [DOI] [PubMed] [Google Scholar]
  114. Surguladze S. A., Young A. W., Senior C., Brébion G., Travis M. J., Phillips M. L. (2004). Recognition accuracy and response bias to happy and sad facial expressions in patients with major depression. Neuropsychology 18:212. 10.1037/0894-4105.18.2.212 [DOI] [PubMed] [Google Scholar]
  115. Thomas J. P., McFadyen R. G. (1995). The confidence heuristic: a game-theoretic analysis. J. Econ. Psychol. 16, 97–113. 10.1016/0167-4870(94)00032-6 [DOI] [Google Scholar]
  116. Thomas R., Sanders S., Doust J., Beller E., Glasziou P. (2015). Prevalence of attention-deficit/hyperactivity disorder: a systematic review and meta-analysis. Pediatrics 135, 994–1001. 10.1542/peds.2014-3482 [DOI] [PubMed] [Google Scholar]
  117. Unoka Z., Seres I., Aspan N., Bodi N., Keri S. (2009). Trust game reveals restricted interpersonal transactions in patients with borderline personality disorder. J. Pers. Disord. 23, 399–409. 10.1521/pedi.2009.23.4.399 [DOI] [PubMed] [Google Scholar]
  118. Voon V., Irvine M. A., Derbyshire K., Worbe Y., Lange I., Abbott S., et al. (2014). Measuring ‘waiting’ impulsivity in substance addictions and binge eating disorder in a novel analogue of rodent serial reaction time task. Biol. Psychiatry 75, 148–155. 10.1016/j.biopsych.2013.05.013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  119. Voon V., Pessiglione M., Brezing C., Gallea C., Fernandez H. H., Dolan R. J., et al. (2010). Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors. Neuron 65, 135–142. 10.1016/j.neuron.2009.12.027 [DOI] [PMC free article] [PubMed] [Google Scholar]
  120. Waltz J. A., Schweitzer J. B., Ross T. J., Kurup P. K., Salmeron B. J., Rose E. J., et al. (2010). Abnormal responses to monetary outcomes in cortex, but not in the basal ganglia, in schizophrenia. Neuropsychopharmacology 35, 2427–2439. 10.1038/npp.2010.126 [DOI] [PMC free article] [PubMed] [Google Scholar]
  121. Watts F. N., McKenna F. P., Sharrock R., Trezise L. (1986). Colour naming of phobia-related words. Br. J. Psychol. 77, 97–108. 10.1111/j.2044-8295.1986.tb01985.x [DOI] [PubMed] [Google Scholar]
  122. Webb C. A., DelDonno S., Killgore W. D. S. (2014). The role of cognitive versus emotional intelligence in iowa gambling task performance: what's emotion got to do with it? Intelligence 44, 112–119. 10.1016/j.intell.2014.03.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  123. Wechsler D. (2008). Wechsler Adult Intelligence scale–Fourth Edition (WAIS–IV). San Antonio, TX: NCS Pearson. [Google Scholar]
  124. Wong R. Y.-M., Hong Y. (2005). Dynamic influences of culture on cooperation in the prisoner's dilemma. Psychol. Sci. 16, 429–434. 10.1111/j.0956-7976.2005.01552.x [DOI] [PubMed] [Google Scholar]
  125. Worbe Y., Savulich G., Voon V., Fernandez-Egea E., Robbins T. W. (2014). Serotonin depletion induces ‘waiting impulsivity’ on the human four-choice serial reaction time task: cross-species translational significance. Neuropsychopharmacology 39, 1519–1526. 10.1038/npp.2013.351 [DOI] [PMC free article] [PubMed] [Google Scholar]
  126. Zahn R., Lythe K. E., Gethin J. A., Green S., Deakin J. F. W., Workman C., et al. (2015). Negative emotions towards others are diminished in remitted major depression. Eur. Psychiatry 30, 448–453. 10.1016/j.eurpsy.2015.02.005 [DOI] [PubMed] [Google Scholar]
  127. Zahn R., Moll J., Paiva M., Garrido G., Krueger F., Huey E. D., et al. (2009). The neural basis of human social values: evidence from functional MRI. Cereb. Cortex (N.Y.) 19, 276–283. 10.1093/cercor/bhn080 [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Frontiers in Behavioral Neuroscience are provided here courtesy of Frontiers Media SA

RESOURCES