Assessing Social-Cognitive Deficits in Schizophrenia With the Mayer-Salovey-Caruso Emotional Intelligence Test

Abstract

The emotion management subscale of the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) has recently been recommended by the National Institute of Mental Health Measurement and Treatment Research to Improve Cognition in Schizophrenia committee as the sole measure of social cognition for trials of cognitive enhancement in schizophrenia, yet the psychometric properties of this subscale and the larger instrument in schizophrenia patients have not been thoroughly examined. This research presents a psychometric investigation of the MSCEIT in a sample of 64 early course outpatients with schizophrenia, schizoaffective, or schizophreniform disorder. Results demonstrated that the MSCEIT possesses adequate internal consistency reliability among its branch and total scales and that patients’ branch and overall test performance was significantly below normative levels. Estimates of discriminant and concurrent validity indicated that the MSCEIT diverged from measures of neurocognitive functioning and psychopathology, but was only modestly related with objective measures of functional outcome. Convergent validity estimates suggested that, contrary to expectations, the MSCEIT did not correlate with a behavioral measure of social cognition. Finally, exploratory factor analyses suggested the possibility of a shift in the latent structure of emotional intelligence in schizophrenia, compared with studies with healthy individuals. These findings support the use of the MSCEIT as a reliable and potentially valid method of assessing the emotional components of social cognition in schizophrenia, but also point to a need for additional measurement development efforts to assess broader social-cognitive domains that may exhibit stronger relations with functional outcome. Further investigation is warranted to examine the instrument's latent factor structure and convergence with other measures of social cognition.

Schizophrenia is a chronic and debilitating disorder that places significant burden on the individuals who suffer from it, as well as their families and society. Despite marked improvement in managing positive symptoms that followed the introduction of antipsychotic medications,¹ the majority of individuals who suffer from schizophrenia continue to have a limited recovery from the illness and experience significant social disability.^2,3 Research has increasingly suggested that certain cognitive deficits may be rate-limiting factors that conspire against social recovery among individuals with schizophrenia.^4,5 Recently, deficits in social cognition, or the ability to process and interpret socioemotional information in oneself and others,⁶ have begun to show promise as important but often overlooked factors that contribute to poor social adjustment among this population.^4,7–9 Social cognition includes a broad number of cognitive abilities that may be critical for adequate social functioning and interpersonal success, such as the ability to recognize important social cues,¹⁰ infer the mental states of others,¹¹ appraise the social context,¹² and process, interpret, and manage emotions in social situations.¹³ Consequently, it is likely that, if present, deficits in these domains could substantially undermine functional recovery from schizophrenia,⁴ despite symptom stabilization. Unfortunately, while some progress has been made in understanding the nature of social-cognitive functioning in schizophrenia,^7,9,14 research in this area has been hampered by a lack of scientifically sound measures.¹⁵ The development of such measures is a key step in elucidating and ameliorating the factors that contribute to functional disability in schizophrenia.

One measure that holds significant potential as a reliable and valid assessment of some of the emotional components of social cognition in schizophrenia, and a component (emotion management) of which has recently been recommended for assessing social cognition by the National Institute of Mental Health Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) committee¹⁶ is the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT¹⁷). The MSCEIT is a unique, performance-based measure of emotional intelligence, or the ability to understand and manage emotions in oneself and others,¹⁸ that has been tested and validated in over 2000 healthy individuals. The strengths of the MSCEIT lie in its performance-based administration, which examines emotion-related social-cognitive abilities through solving problems rather than through potentially biased self-reports and observations, and extensive survey of key emotional components of social cognition, components that are particularly relevant to schizophrenia.^6,7,19 Such strengths suggest that the MSCEIT has promise in becoming a core assessment in schizophrenia research once aspects of validity and reliability are established in this population. Unfortunately, while the instrument has shown excellent psychometric properties among healthy individuals^17,20 and its emotion management subscale has been recommended by the MATRICS committee as the sole measure of social cognition to be employed in clinical trials of cognitive enhancement in schizophrenia, the measurement properties of the MSCEIT among schizophrenia patients have gone largely unexamined.

Few would argue against the need to validate existing measures among populations for whom they were not developed, particularly when instruments are cognitively demanding and to be employed among cognitively impaired groups. To date, only a single study has reported on the psychometric properties of the MSCEIT among persons with schizophrenia. Nuechterlein et al²¹ recently reported select psychometric data on 2 of the 4 subscales (emotion perception and management) of the MSCEIT and found that both subscales demonstrated adequate retest reliability (intraclass correlation coefficient > 0.70) and little detectable practice effects. In addition, the emotion management subscale was moderately related to self-reported functional outcome. While these findings provide important information about the properties of the MSCEIT when applied to persons with schizophrenia, no information was collected on the other components of the test, the internal consistency reliability of the instrument was not evaluated, and the degree to which MSCEIT assessments diverge from clinical and neurocognitive outcomes was not reported. In addition, the factor structure of the MSCEIT among persons with schizophrenia and the degree to which it converges with other social-cognitive assessments and objective measures of functional outcome is currently unknown.

Such psychometric information is particularly important for validating the use of the MSCEIT among individuals with schizophrenia. Neurocognitive and clinical limitations may pose significant impediments to successful test completion, thereby resulting in little new information about social-cognitive functioning beyond what can be gathered with established clinical and neuropsychological assessments. Misunderstanding of test questions or random answering could also lower internal consistency estimates, and associations with other social-cognitive assessments could lend support for the convergent validity of the instrument. In addition, given that self-reported and objective functioning may not necessarily be highly correlated,^22,23 the MSCEIT may show different relations with objective measures of functional outcome. Finally, elucidating the factor structure of the MSCEIT and dimensionality of emotional intelligence in schizophrenia is of critical importance for making inferences about cross-group comparisons with other populations. The current research sought to answer these critical questions regarding the reliability and validity of the MSCEIT when used with persons with schizophrenia by conducting a systematic psychometric evaluation of the instrument.

Methods

Participants consisted of 64 individuals diagnosed with schizophrenia (n = 37), schizoaffective (n = 23), or schizophreniform disorder (n = 4) recruited from Western Psychiatric Institute and Clinic, Pittsburgh, and several nearby community clinics, participating in a randomized controlled trial of cognitive enhancement therapy (CET²⁴). A subsample (N = 38) of these patients has previously been reported on in a preliminary report of the effects of CET on MSCEIT performance.²⁵ Eligibility criteria included a Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV), diagnosis of schizophrenia, schizoaffective, or schizophreniform disorder with the onset of first psychotic symptoms within the past 8 years, IQ ≥ 80, absence of a substance use diagnosis within 2 months prior to study enrollment, and the presence of significant social and cognitive disability as defined by the Cognitive Style and Social Cognition Eligibility Interview used in our previous studies.²⁶ All diagnostic evaluations were made using the Structured Clinical Interview for DSM-IV²⁷ by an expert diagnostician. Enrolled participants were young, with an average age of 25.78 (SD = 6.15) years, approximately 70% (n = 44) were male, and most were either European American (n = 43) or African American (n = 13). Participants had been ill an average of 3.20 years (SD = 2.31) since their first psychotic episode, approximately two-thirds (n = 42) had completed some college education, and most (n = 49) were not employed. All participants were maintained on antipsychotic medication approved by the Food and Drug Administration for the treatment of schizophrenia and schizoaffective disorder by a study psychiatrist, with the majority (98%) receiving atypical antipsychotic medications.

Measures

Mayer-Salovey-Caruso Emotional Intelligence Test.

The MSCEIT¹⁷ is a self-administered, performance-based measure of social cognition and emotional processing ability that assesses the domains of emotion perception, understanding, facilitation, and management. The instrument is performance based in that it requires participants to solve emotionally laden problems rather than relying on self-report,²⁸ much like tests of mathematical or verbal intelligence. The MSCEIT consists of 141 items across 8 distinct tasks that require participants to identify emotion in human faces, scenery, and artwork; match emotions to sensations; judge which emotions facilitate certain thoughts and behaviors; identify how emotions are combined to form other emotions; identify how emotions change across intensities; and identify strategies to manage one's own emotions and the emotions of others. Two tasks comprise each of the 4 “branch” scores in the Mayer and Salovey¹⁸ 4-factor (emotion perception, facilitation, understanding, and management) model of emotional intelligence. For example, the MSCEIT contains a task asking participants to identify specific emotions in faces and another task asking participants to identify emotions in different pictures of scenery and artwork, and these 2 tasks form the emotion perception branch of the test. Total, subscale, and individual items are scored through a web-based scoring program (available from Multi-Health Systems, Inc, Toronto, Ontario, Canada) using unadjusted consensus norms from a large normative sample, where participants receive a score based on the proportion of individuals in the normative sample that endorse their response to a particular question.¹⁷ Total and branch scores are automatically calculated and scaled with a mean of 100 (SD = 15), with higher scores reflecting better emotional intelligence. Individual item–scaled scores are also available through this scoring program and are scored on a common percentile metric, allowing for the aggregation of individual scaled items for item-level reliability analyses. Task and scale definitions required for scoring are available through the web-based scoring program and are also outlined by Mayer et al.¹⁷

Validation Measures.

To evaluate the discriminant, convergent, and concurrent validity^29,30 of the MSCEIT and characterize its assessments within a broader context of theoretically relevant constructs, several validation measures were employed to assess various domains of cognition, psychopathology, and functional outcome.

Neuropsychological and clinical assessments were employed to examine the degree to which the MSCEIT yielded social-cognitive assessments that diverged from general neurocognitive function and psychopathology. Clinical psychopathology was assessed using total, positive, and negative symptom subscale scores from the Brief Psychiatric Rating Scale,³¹ which demonstrated adequate internal consistency (range of α = .74 to .83). Neurocognitive function was assessed using a comprehensive neuropsychological battery reflecting the relevant cognitive domains in schizophrenia identified by the MATRICS committee.¹⁵ This battery included assessments of processing speed (simple and choice reaction time³², and visual-spatial scanning³³), working memory (digit span component of the revised Wechsler Adult Intelligence Scale³⁴), verbal memory (immediate and delayed recall components from stories A and B of the third version of the Wechsler Memory Scale,³⁵ and short- and long-term free recall from the California Verbal Learning Test³⁶), and executive functioning (number of perseverative errors from the Wisconsin Card Sorting Test³⁷). These measures were scaled to common z metric, recoded so that higher scores reflected better neurocognitive ability, and averaged to compute a neurocognitive composite index. This neurocognitive composite displayed adequate levels of internal consistency (α = .76).

A behavioral measure of social cognition, the Social Cognition Profile,²⁶ was employed to examine the degree to which the MSCEIT converged with other social-cognitive assessments. The Social Cognition Profile is a 50-item, clinician-rated measure of social-cognitive behaviors gleaned from the literature on social cognition.^12,38–40 Because this instrument was grounded not only in schizophrenia research on social cognition but also in the broader social-cognitive developmental literature, it covers a broad range of constructs not commonly assessed in schizophrenia research. These constructs are divided into 4 domains, supported by previous factor-analytic studies,²⁶ that cover tolerant (eg, accepting, respectful, cooperative), perceptive (eg, foresightful, self-aware, gistful), supportive (eg, empathic, reciprocal, friendly), and self-confident (eg, comfortable, assertive, involved) behaviors indicative of adequate social cognition. Each of the behaviors assessed are thought to be key reflections of a number of developmental social-cognitive processes, including social context appraisal, perspective-taking, interpersonal reciprocity, prospection, self-reflection on social behavior, and sensitivity to the feelings and intentions of others. Items are rated on a 5-point scale, with higher scores reflecting better social cognition. Previous research has documented that the Social Cognition Profile has adequate internal consistency and interrater reliability, converges with other behavioral measures of social cognition, and is sensitive to treatment-induced changes in social cognition.²⁶ In this research, we found the internal consistency of the Social Cognition Profile to be adequate (range of α = .79–.92).

Finally, 3 measures of functional outcome were used to examine the degree to which the social-cognitive assessments provided by the MSCEIT held any predictive utility to theoretically relevant functional outcomes. These measures included the Social Adjustment Scale-II,⁴¹ the Global Assessment Scale,⁴² and the Performance Potential Inventory.²⁶ The Social Adjustment Scale-II and Global Assessment Scale are well-established measures of functional outcome commonly employed in schizophrenia research, with high scores reflecting worse and better functioning, respectively. The total and most subscale scores of the Social Adjustment Scale-II displayed adequate levels of internal consistency (range of α = .71–.84); however, the internal consistency of the 3-item self-care subscale was low (α = .55). The Performance Potential Inventory is a 55-item, clinician-rated measure of functional disability based on work disability criteria outlined by the Social Security Administration.⁴³ Items are rated on a 5-point scale, with higher scores indicating better functioning. The instrument yields 5 subscales assessing the domains of task/work performance, social functioning, activities of daily living, work readiness, and mental status. The mental status subscale was excluded in this research in favor of the aforementioned, performance-based neuropsychological tests. Previous research has documented the reliability and convergent validity of this measure of functional outcome.²⁶ In this research, we found the Performance Potential Inventory to display adequate levels of internal consistency (range of α = .86–.95).

Procedures.

Upon recruitment, participants were assessed for eligibility in diagnostic interviews and consensus conferences. Eligible participants were then randomized to a clinical trial of CET, which has been described elsewhere,²⁵ and then assessed using the MSCEIT and aforementioned measures of cognitive, clinical, and functional outcomes after randomization and prior to treatment. Neurocognitive assessments were completed by trained neuropsychologists. The Social Cognition Profile, as well as clinical and functional assessments, were completed by masters-level clinicians who had been extensively trained in their use as part of previous studies^26,44 and were intimately involved in the treatment of the participants. This study was approved by the University of Pittsburgh Institutional Review Board, and all patients provided written, informed consent prior to participation. All data-analytic procedures were conducted using R version 2.5.0 on a Linux (2.6) 64bit operating system.

Results

Descriptive Statistics and Distributional Properties

Prior to examining the psychometric properties of the MSCEIT, the distributional properties and score ranges of the instrument were first checked. A large range of scores was observed for MSCEIT total (range = 54.09–121.70) and branch scores (range = 50.53–131.82), indicating adequate ranges and sufficient variability of emotional intelligence among the sample. Distributions for MSCEIT total and branch scores displayed little to no skewness (range of skewness = −0.30 to 0.15), and visual inspection of histograms and qq-plots, confirmed by Shapiro-Wilk⁴⁵ tests, indicated that none of the scales departed significantly from normality (all W > 0.97, all P > .30). Means and SDs for MSCEIT performance in this sample are displayed in table 1. Based on a series of 1-sample t-tests comparing sample scores with the mean (SD) test performance value of 100 (15) from a large normative sample,¹⁷ participants displayed highly significant performance deficits on MSCEIT total and all branch subscores.

Table 1.

Descriptive Statistics and Internal Consistency Estimates for the MSCEIT

Scale	Items	Cronbach α	M (SD)	Pa
MSCEIT Total	141	.94	86.10 (15.46)	<.001
Branch 1—emotion perception	50	.91	91.92 (16.26)	<.001
Faces	20	.82	99.41 (25.02)	.850
Pictures	30	.91	90.31 (12.31)	<.001
Branch 2—emotion facilitation	30	.83	93.30 (17.37)	.003
Facilitation	15	.78	99.14 (18.68)	.716
Sensations	15	.72	91.80 (13.66)	<.001
Branch 3—emotion understanding	32	.78	87.06 (12.10)	<.001
Changes	12	.70	90.27 (11.01)	<.001
Blends	20	.52	87.99 (11.42)	<.001
Branch 4—emotion management	29	.81	86.95 (11.93)	<.001
Management	20	.67	89.85 (10.56)	<.001
Relationships	9	.73	87.00 (11.94)	<.001

Note: MSCEIT = Mayer-Salovey-Caruso Emotional Intelligence Test.

^aSignificance levels reflect the results of 1-sample t-tests (2-tailed) comparing sample means to mean (SD) values of 100 (15) based on a large (N = 2112) normative research sample.¹⁷

Reliability

We examined the reliability of the MSCEIT by computing Cronbach's α coefficient of internal consistency for each of its task and branch components. Missing data in this and all subsequent analyses were assumed to be missing at random and handled using the expectation-maximization maximum likelihood approach.⁴⁶ As can be seen in table 1, the total test displayed excellent internal consistency. Similarly, all the MSCEIT branches also displayed adequate levels of internal consistency, with the emotion perception branch displaying the highest level of internal consistency and the emotion understanding branch displaying the lowest but still acceptable level of internal consistency. Reliability estimates were somewhat lower for individual tasks, with the blends and emotion management tasks displaying suboptimal levels of internal consistency. These estimates of reliability for MSCEIT total, branch, and task scores among schizophrenia patients are quite similar to those found in a large sample of healthy individuals¹⁷ and support the reliability of MSCEIT total and branch scores among persons with schizophrenia.

Validity

After examining the reliability of the MSCEIT among schizophrenia patients, we proceeded to investigate its discriminant, convergent, and concurrent validity by examining the interrelations between MSCEIT test performance and demographic, cognitive, clinical, and functional variables.

Discriminant Validity.

The discriminant construct validity of the MSCEIT was examined by estimating its relationship with demographic characteristics, as well as general cognitive function and psychopathology. Relations with continuous variables were analyzed using Pearson correlation coefficients, and associations with binary variables were examined using point-biserial correlation coefficients. In general, the MSCEIT shared little variance with age, gender, education, or illness duration (table 2), although females tended to have higher overall emotional intelligence. Individuals with more education also demonstrated greater emotional intelligence, particularly with regard to emotional understanding. In addition, those with longer illness durations demonstrated significantly poorer emotion perception abilities, which may reflect a relationship between emotion perception and either illness chronicity or age of onset, with which illness duration is confounded. As expected, all MSCEIT scale scores were significantly positively related to neurocognitive composite scores, although these relationships were at most moderate in size, suggesting that while related the MSCEIT was sufficiently independent of measures of neurocognitive function. As shown in table 2, several domains of MSCEIT performance were related to clinical psychopathology, particularly emotion perception, although these relationships were also generally small. Negative symptoms were not related to any area of MSCEIT performance. Taken together, these findings suggest that the MSCEIT possesses some discriminant construct validity, and its social-cognitive assessments are not substantially contaminated by likely demographic, clinical, and cognitive correlates.

Table 2.

Relationship Between MSCEIT Performance and Demographic Characteristics, Cognition, and Psychopathology

Variable	MSCEIT Total	Emotion Perception	Emotion Facilitation	Emotion Understanding	Emotion Management
Demographics
Age	−.10	−.11	−.11	−.03	−.09
Gender (0 = male, 1 = female)	.28*	.24†	.20	.24†	.23†
Education (0 = no college, 1 = some college)	.29*	.13	.26*	.33**	.19
Years since first psychotic episode	−.19	−.31*	−.17	−.09	.01
General cognition
Neurocognitive compositea	.38**	.31*	.35**	.38**	.27*
Clinical
BPRS totalb	−.22†	−.29*	−.18	−.24†	−.07
BPRS positive	−.17	−.25*	−.13	−.18	−.04
BPRS negative	.05	.16	.08	−.04	−.04
Social cognition
Social Cognition Profilea
Tolerant factor	.08	.03	−.02	.10	.07
Supportive factor	−.01	−.13	−.04	−.01	.12
Perceptive factor	.02	−.10	−.07	.10	.08
Confident factor	−.21	−.15	−.13	−.17	−.15

Note: MSCEIT = Mayer-Salovey-Caruso Emotional Intelligence Test; BPRS = Brief Psychiatric Rating Scale

^aHigher scores indicate better performance.

^bHigher scores indicate more symptomatology.

†P < .10, *P < .05, **P < .01, 2-tailed.

Convergent Validity.

In addition to examining the divergence of the MSCEIT from several potential confounds, we also investigated the degree to which performance on the measure converged with behavioral assessments of social cognition provided by the Social Cognition Profile. As can be seen in table 2, contrary to our expectations, the MSCEIT demonstrated little convergence with the Social Cognition Profile. No areas of MSCEIT performance were significantly related to any of the Social Cognition Profile subscales.

Concurrent Validity.

We concluded our examination of the validity of the MSCEIT by investigating its cross-sectional predictive utility with relevant functional outcomes, both before and after adjusting for demographics (age, gender, education, and illness duration), Brief Psychiatric Rating Scale³¹ total, and neurocognitive composite scores in partial correlation analyses. As shown in table 3, of the 7 measures of functional outcome provided by the Social Adjustment Scale-II, MSCEIT performance was related to 2: major role functioning and interpersonal anguish. Only overall emotional intelligence demonstrated a statistically significant relationship with better role functioning, however trends were also observed between role functioning and emotion understanding. In addition, nonsignificant trends were observed pointing to a relationship between better emotion facilitation abilities, less interpersonal anguish, and better role functioning. When adjusting for demographic, clinical, and neurocognitive characteristics, these relationship sizes were all substantially reduced, with the exception of the relationship between emotion facilitation and interpersonal anguish. A series of stepwise analyses indicated that adjusting for psychopathology alone was enough to reduce relationship sizes between MSCEIT performance and Social Adjustment Scale-II scores to nonsignificant levels.

Table 3.

Relationship Between MSCEIT Performance and Functional Outcome

Variable	MSCEIT Total	Emotion Perception	Emotion Facilitation	Emotion Understanding	Emotion Management
Social Adjustment Scale-IIa
Total	−.17 (−.07)	−.15 (.04)	−.18 (−.12)	−.08 (.05)	−.16 (−.13)
Interpersonal anguish	−.14 (−.10)	−.17 (.04)	−.22† (−.29*)	−.01 (.07)	−.04 (−.11)
Sexual relations	−.06 (−.06)	.06 (.03)	.03 (.05)	−.03 (.003)	−.20 (−.16)
Family/household relations	−.02 (.04)	.11 (.23†)	−.02 (.01)	−.01 (.02)	−.07 (−.06)
Social leisure	−.03 (.02)	−.05 (−.001)	−.09 (−.09)	.03 (.10)	−.03 (.01)
Major role functioning	−.25* (−.08)	−.19 (−.01)	−.21† (−.08)	−.24† (−.08)	−.19 (−.10)
Self-care	.05 (.09)	.16 (.21)	.07 (.10)	.08 (.13)	−.09 (−.08)
Performance Potential Inventoryb
Total	.17 (.12)	.09 (.06)	.09 (.03)	.12 (−.01)	.19 (.16)
Daily living activities	−.04 (.03)	−.15 (−.08)	−.06 (−.01)	.01 (.02)	−.004 (.01)
Social functioning	.28* (.19)	.31* (.24†)	.13 (.01)	.23† (.08)	.22† (.18)
Task/work performance	.10 (.05)	.003 (−.04)	.08 (.03)	.02 (−.10)	.18 (.16)
Global work potential	.27* (.10)	.18 (.02)	.25* (.10)	.20 (−.01)	.26* (.16)
Global Assessment Scaleb	.16 (−.02)	.09 (−.11)	.14 (.02)	.09 (−.12)	.20 (.12)

Note: Correlations adjusting for demographics (age, gender, education, and illness duration), Brief Psychiatric Rating Scale³¹ total, and neurocognitive composite scores are presented in parentheses. MSCEIT = Mayer-Salovey-Caruso Emotional Intelligence Test.

^aHigher scores indicate worse functioning.

^bHigher scores indicate better functioning.

†P < .10, *P < .05, **P < .01, 2-tailed.

With regard to the Performance Potential Inventory, relations with functional outcome were more prevalent, although still somewhat limited (see table 3). Social functioning was significantly related to overall emotional intelligence and emotion perception ability, and trends were observed between better social functioning and emotion understanding and management abilities. In addition, global work potential was also related to overall emotional intelligence and emotion facilitation and management abilities. However, relations with work potential were substantially reduced when adjusting for demographic, clinical, and neurocognitive characteristics. Relations between MSCEIT performance and social functioning were also somewhat reduced when adjusting for demographic, clinical, and neurocognitive characteristics, although emotion perception ability continued to display a nearly significant trend (P = .071) toward predicting social functioning. Subsequent stepwise analyses indicated that adjusting for neurocognitive functioning and psychopathology alone were enough to reduce associations between MSCEIT scores and global work potential and social functioning to nonsignificant levels. In addition, illness chronicity also contributed to a reduction in MSCEIT associations with global work potential.

No significant relations were found between MSCEIT performance and Global Assessment Scale scores. Further, although type I error corrections were not employed in our primary analyses due to the exploratory nature of this initial investigation of the MSCEIT, it should be noted that none of the relations with functional outcome remained statistically significant after making adjustments for multiple inference testing.

Factor Structure

Having found some evidence for the reliability and discriminant and concurrent validity of the MSCEIT, we proceeded to conduct an exploratory investigation of the instrument's factor structure to examine whether a factor solution would emerge similar to those found among healthy samples when employed with individuals with schizophrenia. This was accomplished by conducting a series of exploratory principal axis factor analyses, with oblique factor rotation, on the 8 MSCEIT tasks. Visual inspection of screenplots of component eigenvalues (not shown) indicated that only 2 eigenvalues were greater than 1; as such only 1- and 2-factor solutions were considered.

As can be seen in table 4, both 1- and 2-factor solutions explained a substantial proportion of the variance among MSCEIT tasks (45% and 47%, respectively), although the 2-factor solution did provide a statistically significant improvement in model fit over a 1-factor solution, Δχ²(7, N = 64) = 19.60, P = .007. Nonetheless, both factors from this solution were highly correlated (r = 0.61). While the resulting 1-factor solution mirrors the general emotional intelligence model of Mayer and Salovey,¹⁸ the 2-factor solution broadly represents an emotional perception and understanding factor, and an emotional facilitation and management factor, which is not consistent with either the prominent 4-factor model or the broader experiential (emotion perception and facilitation) and strategic (emotion understanding and management) area model proposed by Mayer et al.¹⁷ Consequently, it would appear that the factor structure of emotional intelligence may be different when assessed in schizophrenia.

Table 4.

Factor Structure of the MSCEIT

Variable	Standardized Factor Loadings
	1-Factor Solutiona	2-Factor Solutionb
	Factor 1	Factor 1	Factor 2
Branch 1—emotion perception
Faces	.44	−.16	.70
Pictures	.57	.06	.61
Branch 2—emotion facilitation
Facilitation	.60	.45	.20
Sensations	.74	.64	.17
Branch 3—emotion understanding
Changes	.71	.18	.64
Blends	.74	.31	.52
Branch 4—emotion management
Management	.73	.92	−.09
Relationships	.76	.70	.13

Note: Factor loadings greater than 0.30 appear in boldface. Correlation between factor 1 and factor 2 in the 2-factor solution: r = 0.61.

^aR² = 0.44, χ²(20, N = 64) = 45.49, P < .001.

^bR² = 0.47, χ²(13, N = 64) = 25.89, P = .018, ΔR² = 0.03, Δχ²(7, N = 64) = 19.60, P = .007.

Discussion

Social cognition has emerged as an important concept for further elucidating the biopsychosocial factors that limit functional recovery from schizophrenia.⁹ Unfortunately, with few exceptions, work in this area has been substantially limited by a lack of consensus regarding measurement strategies for assessing social cognition and a general absence of carefully validated measurement approaches for this population. Components from a promising measure of the emotional domains of social cognition, the MSCEIT, have recently been recommended by the MATRICS committee as the sole measure of social cognition to be included in clinical trials of cognitive enhancement in schizophrenia.¹⁶ While this instrument has shown excellent psychometric properties in a large sample of healthy individuals,^17,20 its measurement characteristics when applied to patients with schizophrenia have not been thoroughly evaluated. This research is the first detailed psychometric investigation of the use of the complete MSCEIT for assessing social cognition in schizophrenia.

Results from this investigation provide important information regarding the strengths and limitations of the MSCEIT for measuring social-cognitive dysfunction in schizophrenia. Several strengths were observed supporting the use of the MSCEIT among this population. In particular, the measure appears to possess moderate to high levels of internal consistency among its branch and total scores, signifying its reliability when employed with schizophrenia patients. In addition, although associations with individual cognitive domains may be stronger, the MSCEIT diverged significantly from an overall composite index of neurocognitive performance and clinical psychopathology, suggesting that its assessments reflect a relatively independent domain of social cognition not substantially associated by neurocognitive dysfunction or symptomatology. Further, the MSCEIT appears to be capable of reliably detecting deficits in social cognition among schizophrenia patients because individuals in this sample scored nearly 1 SD (M = −.79, SD = .33) below mean performance levels from a normative sample on total and all branch subscores. Finally, MSCEIT test performance was somewhat predictive, albeit sparsely, of select objective measures of functional outcome, particularly social and role functioning, and global work potential. These findings echo the recent results of Nuechterlein et al²¹ on the emotion perception and management branches of the instrument and point to the reliability and discriminant, construct, and concurrent validity of the measure.

Despite consistent favorable results regarding the reliability of the MSCEIT, and evidence of its discriminant and concurrent validity, the results of this investigation also highlight some of the limits of using this instrument as the sole measure of social cognition in schizophrenia trials. In particular, the domain of social cognition is known to be quite broad and includes many different cognitive abilities, of which emotional intelligence is but one.¹⁶ This point is underscored by our results showing little to no relationship between the MSCEIT and the Social Cognition Profile, a behavioral measure of social cognition. This finding, while unexpected, likely reflects the heterogeneity of social-cognitive domains assessed by these instruments because the Social Cognition Profile is a clinician-rated assessment of a wide array of social-cognitive behaviors, few of which are emotional in nature, whereas the MSCEIT exclusively covers the emotional components of social cognition. Although the 2 instruments may not be expected to show high levels of convergence due to their methodological differences in administration, as suggested by evidence indicating little overlap between ability-based and non–ability-based measures of social cognition,⁴⁷ it seems likely that the lack of association between the measures also stems from their assessment of different and possibly independent social-cognitive domains. For example, it is not clear that the ability to regulate emotions should be strongly related to the ability to abstract the gist from social situations or the capacity to gain insight into the motivations of others, as assessed by the interpersonal perceptiveness factor of the Social Cognition Profile. It will be important for future studies to examine the convergence of the MSCEIT with other performance-based measures of social cognition and emotion processing, such as Baron-Cohen's Reading the Mind in the Eyes Test⁴⁸ and various emotion recognition paradigms.

Clearly there are additional important domains of social cognition relevant to schizophrenia beyond what the MSCEIT provides, as the MATRICS committee and others have indicated,^7,16 and which is also reflected by the limited degree to which MSCEIT performance was related to functional outcome in this research. While the MSCEIT did show consistent relations with social and role functioning, as well as global work potential, there were many more measures of functional outcome assessed in this research that showed no significant relationship with MSCEIT performance. In addition, the significant relations with functional domains that did exist tended to be in the small to medium-size range,⁴⁹ consistent with the findings of Nuechterlein et al,²¹ and were attenuated, sometimes substantially, when adjusting for potential confounds, particularly neurocognitive functioning and psychopathology.

The lack of larger and more pervasive relations with functional outcome could stem from range restrictions in this domain due to the selection of patients into a clinical trial of CET. In addition, these findings may not be generalizable to more chronic patients, where relations between functioning and MSCEIT performance could be stronger. However, given the findings of this research, it also seems possible that the MSCEIT may not assess all the domains of social cognition that are relevant to functional disability in schizophrenia. Further, given that everyday functioning likely depends on a number of different factors, not all of which are under patients’ control, stronger relations might be expected between MSCEIT performance and functional capacity, rather than outcome. As such, while the results of this investigation highlight the utility of the MSCEIT for assessing the emotional intelligence domain of social cognition in schizophrenia, they also point to the need for further examinations with more proximal markers of patient functioning, such as functional capacity, as well as the critical need for broader measurement approaches. Unfortunately, at this time a unified battery that assesses the domains of social cognition relevant to the disorder does not exist. Interdisciplinary efforts with social cognition experts in social psychology, social decision sciences, and developmental psychology designed to identify and develop broader measures of social cognition relevant to schizophrenia are needed to continue to advance the social cognition research agenda spearheaded by the MATRICS committee.¹⁶

In addition to highlighting the need for further measurement development efforts to more broadly assess the domains of social cognition that are relevant to functional disability in schizophrenia, the results of this research also point to the need for further investigation of the psychometric properties of the MSCEIT itself when applied to schizophrenia patients. Although we demonstrated that patients perform substantially below normative levels on MSCEIT total and branch subscores, to date no studies have examined the degree to which MSCEIT performance differs between individuals with schizophrenia and appropriately matched healthy controls, which will be an important area for future investigations evaluating the utility of this measure in schizophrenia research. Further, the findings from this research also indicate the need for additional investigations regarding the factor structure of the MSCEIT in schizophrenia, given that our factor-analytic results found evidence of a 2-factor solution for the instrument that has yet to be identified in the several factor-analytic studies of the MSCEIT in healthy populations.^17,50 Previous studies have largely supported the 4-factor (or 4-branch) model of emotional intelligence proposed by Mayer and Salovey,^18,51 which consists of emotion perception, facilitation, understanding, and management factors. Some evidence also exists for a 2-factor area model of emotional intelligence, combining emotion perception and facilitation factors and emotion understanding and management factors, as well as a 1-factor general model.¹⁷ In this research, we found evidence for both 1- and 2-factor solutions for the MSCEIT, but this latter 2-factor solution was at variance with the aforementioned 2-factor structure documented in previous studies, as we found support for combining the domains of emotion perception and understanding and emotion facilitation and management. While such a solution is at variance with those reported in the literature examining healthy individuals, this solution does make some conceptual sense because the accurate perception and understanding of emotions both rely heavily on emotional knowledge, whereas the recruitment of emotions to facilitate thinking and the management of emotions may better reflect “hot” social-cognitive processes indicative of either the up- or down-regulation of emotion.

The evidence for an alternative factor structure for the MSCEIT among persons with schizophrenia could be due to several reasons, which require further investigation. First, the sample size employed in this research was admittedly modest, which may have precluded the identification of a larger number of factors. However, we were able to surpass the recommended 5:1 participant-to-variable ratio that is an acceptable, minimum standard for exploratory factor analysis.⁵² Second, all previous factor-analytic studies of the MSCEIT have used confirmatory approaches to test a priori theoretical models that have not considered the possibility of alternative factor solutions. Consequently, it is possible that the factor structure elucidated in this research may be a plausible solution among healthy individuals that has merely gone overlooked due to the excessive reliance on confirmatory approaches. Finally, it is also possible that the latent structure of the instrument may truly be different for persons with schizophrenia because several studies have found that measurement equivalence cannot necessarily be assumed between healthy and psychiatric populations.^53–56 Future studies will need to employ larger sample sizes and a variety of factor-analytic approaches to elucidate the latent structure of the MSCEIT among individuals with schizophrenia. Such investigations could yield important insights regarding the dimensionality of emotional intelligence deficits in schizophrenia and point to cautions regarding the interpretation of cross-group comparisons between individuals with schizophrenia and other populations.

In summary, this psychometric investigation of the use of the MSCEIT for assessing social-cognitive deficits in schizophrenia suggests that the instrument can be reliably employed in service of social-cognitive assessment among this population and that the assessments it provides overlap little with measures of neurocognitive function and psychopathology and are significantly predictive of some cross-sectional, objective measures of functional outcome. When combined with previous evidence showing the sensitivity of the instrument to social-cognitive intervention,²⁵ these findings support the MATRICS committee's decision to use the emotion management subscale of the MSCEIT as a key measure of the emotional components social cognition in clinical trials of cognitive enhancers for schizophrenia. However, a balanced view of the strengths and limitations of the MSCEIT that take into account its limited scope in assessing the range of social-cognitive constructs, along with findings from this research indicating a lack of convergence with other measures of social cognition and somewhat modest relations with functional outcome, also highlight the need for broader assessment strategies. Systematic, interdisciplinary measurement initiatives are needed to continue to advance social-cognitive measurement in schizophrenia, as are further investigations of the factor structure of the MSCEIT in schizophrenia patients and its convergence with other measures of social cognition and emotion processing.

Funding

National Institute of Mental Health (MH 79537 to S.M.E., MH 60902 to G.E.H., M.S.K.).