Abstract
Social cognition is a core limiting factor of functional recovery among persons with schizophrenia. However, there is a lack of standardized and culturally relevant assessment tools for evaluating social cognitive performance in Chinese persons with schizophrenia. The purposes of this study were to (1) develop and validate two social cognitive instruments, the Chinese Facial Emotion Identification Test (C-FEIT) and the Chinese Social Cognition and Screening Questionnaire (C-SCSQ), that assess three key domains of social cognition and (2) to evaluate preliminary psychometric properties of the two assessments. The results demonstrated that the C-FEIT and the social cognitive subscales of C-SCSQ possess satisfactory content-related validity and test–retest reliability (ICC ranging from 0.76 to 0.85). Subscales of the C-FEIT and the C-SCSQ showed low to medium correlation with two concurrent neurocognitive measures (absolute values of r ranging from 0.22 to 0.45) and concurrent measures of functional performance (absolute values of r ranging from 0.22 to 0.46). Our findings generally support the use of the C-FEIT and the C-SCSQ as reliable and valid tools for assessing emotion perception, theory of mind (intention-inferencing), and hostile attributional style, which are the key outcome indicators of social cognitive interventions for persons with schizophrenia.
Keywords: assessment, social cognition, schizophrenia, validation, Chinese
Introduction
Schizophrenia is a severe psychiatric illness characterized by marked deficits in a wide array of functional areas. A key focus of current research is to identify factors limiting function in schizophrenia. In the past decade, there has been increasing research interest in examining social cognition as a predictor of functional outcomes in rehabilitation. There is growing evidence that social cognition mediates the relationship between neurocognition and functional outcomes, and it is possible that improving social cognitive deficit or reducing social cognitive bias could make a positive impact on social and work adjustment of persons with schizophrenia (1–6).
Social cognition refers to how people think about themselves and others in the social world (7). It is conceptualized to encompass a number of cognitive processes that underlie social interactions, including perceiving, interpreting, and responding to the intentions, dispositions, and behaviors of others (8). These social cognitive processes had been further examined and confirmed in social neurosciences study, which revealed the underlying involvement of different but related brain structures (9). While, in the past, there has been controversy over what social cognitive processes should cover, social cognition is now generally defined as encompassing four key domains: emotion perception (EP)/emotion processing, theory of mind /mental state attribution, attributional style, and social perception (10, 11).
This study aims to translate, adapt, and validate a set of two instruments, the Facial Emotion Identification Test (FEIT) and the Social Cognition and Screening Questionnaire (SCSQ), for assessing social cognition in persons with schizophrenia. This study addresses several issues in social cognitive assessment with Chinese populations. First, there are uncertainties on the exact construct measured in some of the existing social cognitive assessments. For instance, the Reading the Mind in the Eyes Test (12), which intended to measure ToM ability, appeared to measure emotion recognition ability or empathy more than higher level mental state attribution (13). The Mayer–Salovey–Caruso Emotional Intelligence Test, which was originally designed to assess emotion intelligence ability (14), required participants to rate effectiveness of emotion management strategies in social scenarios. This task though requires certain level of ability to perceive emotions; it, however, did not assess facial EP directly with the use of photos. There are also questions on whether the Attributional Style Questionnaire (15) assessed the specific attributional style (16) observed among persons with schizophrenia. Based on this review, we selected the FEIT and SCSQ in this validation, as the two tests are able to cover most domains of social cognition relevant to the assessment of persons with schizophrenia. The FEIT covers the assessment of EP, while the SCSQ assesses theory of mind and attributional style, as well as a closely related construct, jump-to-conclusion that are commonly observed in persons with schizophrenia (5, 17, 18).
Second, there are very few Chinese-language version of social cognitive assessments. Many studies of social cognition with Chinese populations focused on EP (19–21), but did not cover assessment of other aspects of social cognition. As it is well documented that culture impacts on development of different social cognitive domains and possibly impacts the related social cognitive performance (22–25), this study will review the cultural relevance of common social cognitive assessments before the assessments are translated for application with Chinese populations.
Third, we found that several adaptations of the FEIT are needed to prevent bias in testing EP. A comprehensive photo set should include six basic emotions faces (26, 27) and neutral facial expression faces (28). The photo set should include an equal number of posers of both genders to display emotions as the use of a photo set with single gender poser could lead to bias in assessment result (29, 30). In addition, the posers should come from both Western and Asian cultures, in order to prevent cultural biases in assessment. In this study, we would make changes to the test photo set in FEIT to prevent these biases in measurement of EP.
In summary, we observed a number of limitations in current instrument for assessing social cognition in Chinese persons with schizophrenia. We propose the combined use of SCSQ and FEIT to form a set of social cognitive assessments that could be administered to target clients within a reasonable timeframe. The specific objectives of this study are to (1) translate and validate two social cognitive instruments, the Chinese version of the Facial Emotion Identification Test (C-FEIT) and the Chinese Social Cognition and Screening Questionnaire (C-SCSQ) to assess the three key domains of social cognition and (2) examine the preliminary psychometric properties of the two instruments, including reviewing the content validity and cultural relevance for C-SCSQ, and reviewing the reliability, concurrent validity and known-groups validity of both instruments.
Materials and Methods
Participants
Two groups of participants were recruited for the study, including patients with schizophrenia and non-psychiatric control. For the patient group (n = 62), the inclusion criteria were (1) aged 18–60 years old and (2) diagnosis of schizophrenia according to the International Classification of Diseases version 10 (ICD-10). The exclusion criteria were (1) dual diagnosis of schizophrenia, such as a neurological disorder, a developmental disability, or a substance abuse problem; (2) diagnosis of intellectual disabilities; and (3) admission to in-patient psychiatric treatment or a change in psychiatric medication within the last 30 days. The inclusion and exclusion criteria of the patient group were confirmed by retrieving information from hospital medical record. Diagnosis of schizophrenia was made by psychiatrist using ICD-10.
For the known-groups method of validation, we planned to conduct a comparison of social cognition of persons with schizophrenia and non-psychiatric controls. A total of 20 non-psychiatric control participants were planned. This sample size is calculated by PASS 12 (31) with a power of 0.80 and effect size set at 0.90. The effect size of 0.90 is set based on effect sizes of group differences between patients and controls in EP, ToM, and AS in previous studies (ranged from 0.90 to 2.47) (16, 32, 33). A group of non-psychiatric control participants (n = 19), matched on age and gender characteristics with some of the participants with schizophrenia, were recruited from a local community center. Only those with no history of psychiatric illness were recruited as non-psychiatric controls. The inclusion and exclusion criteria of the non-psychiatric control participants were determined by semi-structured interviews.
Procedures
Ethical approval to conduct the study was obtained from the Departmental Research Committee of the Department of Rehabilitation Sciences, the Hong Kong Polytechnic University and the Cluster Clinical Research Ethics Committee, New Territories West Cluster of the Hospital Authority. Written informed consent to participate in our study was obtained from participants before the start of data collection. A team member (a researcher or an assistant) briefed the potential participants about the purpose and procedures of the study, then obtained the written informed consent. We administered all social cognitive, neurocognitive, symptoms, and functional measures to the patient group. To allow us to collect data for examining the test–retest reliability, a subgroup of the patient group (n = 17) completed the two social cognitive instruments (C-FEIT and C-SCSQ) twice with a 1-week interval in between. For the non-psychiatric control group (n = 19), we administered only the social cognitive tasks (C-FEIT and C-SCSQ), to investigate the known-groups validity of the two instruments.
Instruments
We collected basic background information (age, education) of the participants through brief interviews or by extracting data from case records. The two social cognitive assessment tasks adopted in this study are the FEIT and the Social Cognition Screening Questionnaire (SCSQ).
Facial Emotion Identification Test
The FEIT is widely used to assess facial EP ability (34). The set of FEIT test photos used in current study consists of 21 photos: 12 photos showing the six basic emotions (happiness, sadness, anger, disgust, fear, and surprise) and nine photos conveying neutral emotions (used as controls). The 12 basic-emotion photos were selected from the Japanese and Caucasian Facial Expression of Emotion (JACFEE) photo set (35), which demonstrated acceptable levels of agreement in a study with Chinese subjects (36). The nine neutral-emotion photos were selected from the Japanese and Caucasian Neutral Faces (JACNeuF) collection (35, 37).
In constructing the C-FEIT used in this study, we made a few changes to the original FEIT to address some methodological issues identified in previous studies. First, we used photos displaying disgust to replace shame in the FEIT, as disgust is widely regarded as one of six basic emotions (27), and disgust is more commonly use in EP studies of persons with schizophrenia than shame (30). Second, we used an equal number of photos displaying different emotions. Third, we used photo sets from both the JACFEE and JACNeuF (photos with neutral emotion), this photo set is one of the gold standards of facial expressions (35). In fact, a similar photo set had been validated for use with Chinese population, and agreement level was acceptable (N = 120) (36). It would be best to use photos of Chinese faces to portray the six basic emotions in the test, but there is not yet any validated photo set of Chinese faces that consisted of six basic emotions (38).
A set of Chinese-language instructions was used to guide the standard administration of the test. The photo sets were presented in random order on a notebook computer or a desktop computer. Several parameters were standardized during administration. First, the time of exposure and time of rest in between two photos were each fixed at 10 s, in accordance with the standards used in similar studies. Second, the photo sets were presented on either a desktop computer or a notebook (using Microsoft PowerPoint) with a screen size of less than 18 inches, and with an arm’s length between the participant and the screen. These settings were used to approximate a face size of the subject pictured in the photo to that of a face as it would appear in day-to-day conversation. After viewing each photo, the participant was required to select which of the seven emotions (six basic emotions or the neutral option) was conveyed in the photo, and to record the answer on the answer sheet. The participants’ responses were recorded and marked as correct or incorrect.
Social Cognition and Screening Questionnaire (SCSQ)
The SCSQ is designed to assess three key aspects of social cognition: ToM, jumping-to-conclusions (JTC) bias, and paranoid attributional style. The instrument screens for neurocognitive deficits and the patient’s needs for social cognitive intervention (39). The SCSQ presents interpersonal vignettes that describe ambiguous interpersonal situations. The examiner presents the vignettes to the participant orally within 10 s, then the participant is required to answer three yes/no questions. Two of the yes/no questions test the participant’s ability to recall details in each vignette, and the total number of correct answers is summed as the “neurocognitive score” (NC). The score range is 0–20. The remaining yes/no questions involve an intention-inferencing task that tests the participant’s ability to infer a character’s intentions from information in the vignette. The total number of correct answers is summarized as the “perspective-taking score” or “theory of mind” (ToM). The score range is 0–10. The confidence-judgment question assesses the participant’s tendency to make overconfident judgments, which contributes to the “JTC score” (JTC). The “paranoid/hostile attributional bias score” (PAS) is calculated by summing the incorrect perceptions in vignettes 2, 3, 5, 6, and 9 (incorrect perceptions are those answers with a score range of 0–5).
With approval from the original author, we hired a qualified translator to translate the original English version of the SCSQ into Chinese using the idiomatic translation method. A four-member expert panel was set up to (1) appraise the quality of the translation, evaluate the semantic equivalence of the two versions, examine the fluency and clarity of the translation, and provide suggestions to improve the translated instrument and (2) evaluate the content-related validity (relevance and representativeness) of the C-SCSQ. The panel members included mental health professionals with at least 10 years of clinical experience, including psychiatrists, clinical psychologists, and occupational therapists.
Psychiatric Symptoms
The mental status of the patients was assessed using the Brief Psychiatric Rating Scale, Expanded Version (BPRS-E) (40). The BPRS-E generates both a total score and a paranoia score. The Paranoia scale (41) consists of five items assessing hostility, suspiciousness, tension, uncooperativeness, and excitement. The paranoia score is used for studying concurrent validity with the paranoid/hostile attributional style (PAS) of the SCSQ. The BPRS-E possessed satisfactory inter-rater reliability of 0.72(42).
Neurocognitive Measures
We selected several subtests of the MATRICS Consensus Cognitive Battery (43) for studying concurrent validity with the Neurocognitive (NC) scale of the SCSQ. The MATRICS Consensus Cognitive Battery measures speed of processing, verbal memory, and verbal fluency (VF). The speed of processing was assessed using the Trail Making Test Part A, and VF was measured by a categorical fluency test using animal naming (44). Verbal learning was measured by the Hopkins Verbal Learning Test, Revised Version (HVLT-R) (45).
Functional Measure
The Chinese Work Personality Profile (CWPP) is a situational assessment tool for assessing work maintenance behavior in a simulated or actual work setting. It consists of 58 items and it has five subscale scores: task orientation, social skills, self-control, attitude, and personal appearance. The CWPP is used as a concurrent measure of function and it is expected to correlate with social cognitive measures. The CWPP has demonstrated good discriminant and predictive validity in previous studies (46), and it has been found to be closely linked to social cognitive measures (6).
Statistical Analysis
The test–retest reliability of the C-FEIT and the subscales of the C-SCSQ were examined using the Intraclass Correlation Coefficient (ICC), using a two-way mixed model of ICC for evaluating consistency (47). The internal consistency of C-FEIT and C-SCSQ were examined using Cronbach’s alpha. For known-groups validity, the t-test was used to compare the differences in social cognitive performances between the patient group and the non-psychiatric control group. Correlation coefficients were used to investigate the concurrent validity with neurocognitive and clinical variables, and the concurrent validity with functional performance.
Results
Table 1 presents the demographic and clinical characteristics of the patient sample. Around half (45.2%) of the participants were males. They had an average duration of illness of 12.04 years. The majority (90.3%) of subjects received atypical medication, and we did not include further information on chlorpromazine-equivalent dosage because the effect of antipsychotic medication on cognitive and social cognitive disorders remains uncertain (48). The BRPS scores showed that the participants had few symptoms and were largely mentally stable. The social cognitive abilities, neurocognitive, and indexes of functional performance are also presented in Table 1.
Table 1.
Demographic and clinical characteristics (N = 62).
| Variables | % | M (SD) |
|---|---|---|
| Demographics | ||
| Gender (male) | 45.2 | |
| Medication (atypical) | 90.3 | |
| Age | 37.97 (11.8) | |
| Years of education | 10.67 (2.8) | |
| Age of onset | 25.93 (9.4) | |
| Duration of illness | 12.04 (9.3) | |
| Clinical Symptoms | ||
| BPRS total | 1.24 (0.2) | |
| BPRS paranoia scale | 1.16 (0.3) | |
| Social Cognitive Measures | ||
| Chinese Facial Emotion Identification Test (FEIT) | 14.50 (3.9) | |
| NC | 14.11 (3.2) | |
| Theory of mind (ToM) | 6.32 (1.8) | |
| JTC | 2.66 (0.8) | |
| Paranoid/hostile attributional style (PAS) | 1.68 (1.1) | |
| Neurocognitive Measures | ||
| TMTa | 57.97 (23.8) | |
| HVLTb | 19.92 (6.53) | |
| Verbal Fluencyb | 17.60 (5.2) | |
| Chinese Work Personality Profile | ||
| Task orientation | 74.37 (16.11) | |
| Social skills | 44.13 (9.80) | |
| Self-control | 22.03 (4.3) | |
| Acceptance of supervision | 27.27 (5.9) | |
| Personal presentation | 6.80 (1.2) | |
FEIT, emotion perception of FEIT, ToM: Theory of mind, JTC: jumping-to-conclusions, PAS: paranoid/hostile attributional style, NC, neurocognitive subscale of SCSQ, TMT, Trail Making Test A, HVLT, Hopkins Verbal Learning Test, VF, Verbal Fluency, BPRS total: total score of Brief Psychiatric Rating Scale, BPRS Paranoia scale: item 6 (tension), item 10 (hostility), item 11 (suspiciousness), item 14 (uncooperativeness), and item 17 (excitement) of BPRS.
Content Validity and Cultural Relevance of the C-SCSQ
The content validity and cultural relevance of the C-SCSQ were reviewed by an expert panel. All panel members were bilingual (Chinese and English) and had at least 10 years of experience in mental health practice. All the experts agreed that the Chinese translation of the SCSQ was semantically equivalent to the original English version, and most of them (75%) agreed that the Chinese translation was fluent. All agreed that the translated version was clear in presentation. Several modifications were suggested by the experts to address issues of cultural relevance. These included replacing “spaghetti” with “fried rice” in vignette 3; replacing “Bingo Game” with “Buy Mark Six (name of local lottery),” and “Susan/Stan” with “Mei Ling/Wai Man” (common Chinese names) in vignette 8; and adjusting the price of toothpaste mentioned in vignette 6 from US dollars to Hong Kong dollars. Most (75%) experts agreed that the translated SCSQ was culturally relevant with the above modifications.
The experts rated most (over 90%) items as having satisfactory content relevance, which exceeded the standard of 75% agreement. The only exception was item 9A, which only 50% of the panel members agreed was relevant. As for content representativeness, all the experts agreed that the theory-of-mind and JTC subscales were representative of social cognition, while 75% of the experts agreed that the neurocognitive and paranoid attributional bias subscales were representative. Based on the results of the content-related validity, three items (9A, 9C, and 2B) and three phrases in vignettes 5, 9, and 10 were modified according to the suggestions of the experts before the C-SCSQ was finalized and administered to the study samples.
Reliability
Test–retest reliability was investigated by administration of the two social cognitive tasks twice to 17 patients with a 1-week interval between tests. The test-retest reliability scores—as indicated by Intraclass Correlation Coefficients of the subscales—were all above the standard of 0.75, except the neurocognitive subscale (ICC = 0.67) (Table 2) (49). For internal consistency, the Cronbach’s alpha for C-FEIT was 0.776 and for C-SCSQ total score including NC, ToM, and AS subscales was 0.630.
Table 2.
Descriptive statistics and test–retest reliability of C-FEIT and C-SCSQ (N = 17).
| Subscales | First administration | Second administration | ICC (95% CI) |
|---|---|---|---|
| M (SD) | M (SD) | ||
| Chinese Facial Emotion Identification Test | 14.94 (2.93) | 15.53 (3.40) | 0.85 (0.57–0.94) |
| Chinese Social Cognition Screening Questionnaire (C-SCSQ) | |||
| Theory of mind | 6.71 (2.02) | 6.06 (1.56) | 0.76 (0.36–0.93) |
| Jumping-to-conclusions | 1.95 (0.92) | 2.27 (0.78) | 0.80 (0.43–0.94) |
| Paranoid/hostile attributional style | 1.47 (1.12) | 1.76 (1.03) | 0.85 (0.60–0.95) |
| Neurocognitive subscale | 13.41 (3.79) | 12.82 (3.23) | 0.67 (0.09–0.88) |
Concurrent Validity
The concurrent validity of the SCSQ was investigated by examining its relationship with clinical symptoms (BPRS), neurocognitive measures (TMT, HVLT, VF Test), and a functional measure (CWPP) (Table 3). We noted several patterns of these correlations. First, none of the correlations between clinical symptoms (BPRS) and social cognitive measures was significant. Second, all three neurocognitive test measures (TMT, HVLT, VF Test) had significant moderate correlations with the NC subscale of the C-SCSQ (absolute values of r ranging from 0.32 to 0.48, p < 0.01 to <0.05). VF was significantly correlated with all subtests of the two social cognitive tasks (absolute values of r ranging from 0.29 to 0.34, p < 0.01 to <0.05), except JTC. HVLT, a test of verbal memory, only correlated with the NC subscale of the C-SCSQ (r = 0.48, p < 0.01). TMT correlated significantly with C-FEIT, ToM, and NC (absolute values of r ranging from 0.25 to 0.45, p < 0.01 to <0.05). Third, there were only a few significant correlations between the two social cognitive tasks and functional measure (indicated by CWPP), as we examined the partial correlations controlling for age and years of education. Among the social cognitive subscales, only the PAS scale of the C-SCSQ had significant negative correlations with three out of five CWPP subscales (r = −0.40 to −0.46, p < 0.05). The other three C-SCSQ subscales (ToM, JTC, NC) had low to moderate and positive correlations with most CWPP subscales, but the correlations were not significant (probably due to the small sample size). FEIT had very low correlations with the CWPP subscales. Apart from concurrent validity, the inter-correlations between FEIT, ToM and PAS subscales of SCSQ and neurocognitive measures were in expected directions. FEIT had low correlations with ToM (r = 0.28), PAS (r = −0.27) subscales of C-SCSQ and low to medium correlations with the three neurocognitive test measures (absolute values ranging from r = 0.37 to r = 0.49). The strength of these correlations were largely comparable to previous studies which employed other social cognitive measures (2, 50).
Table 3.
Relationship between social cognitive performance (C-FEIT and C-SCSQ) and demographic characteristics, neurocognition, and clinical symptoms.
| Variables | C-SCSQ subscales |
||||
|---|---|---|---|---|---|
| C-FEIT | ToM | JTC | PAS | NC | |
| Demographic (n = 62) | |||||
| Age | −0.32* | 0.04 | −1.8 | −0.04 | −0.36** |
| Gender | 0.04 | 0.16 | 0.10 | −0.15 | 0.24 |
| Years of education | 0.47** | 0.14 | 0.19 | −0.05 | 0.42** |
| Illness duration | −0.15 | 0.17 | −0.15 | −0.20 | −0.14 |
| Neurocognition (n = 62) | |||||
| Trail Making Test Aa | −0.45** | −0.25* | −0.07 | 0.22 | −0.40** |
| Hopkins Verbal Learning Testb | 0.22 | 0.10 | 0.02 | −0.12 | 0.48** |
| Verbal Fluencyb | 0.29* | 0.33** | −0.04 | −0.34** | 0.32* |
| Clinical symptoms (n = 62) | |||||
| BPRS totalc | −0.10 | −0.13 | −0.18 | 0.09 | −0.03 |
| BPRS paranoia scalec | −0.07 | −0.11 | 0.08 | 0.05 | −0.01 |
| Functional measure (Chinese Work Personality Profile; n = 30)d | |||||
| Task orientation | −0.11 | 0.38 | 0.22 | −0.46* | 0.22 |
| Social skills | −0.00 | 0.22 | 0.28 | −0.28 | 0.14 |
| Self-control | 0.11 | 0.33 | 0.35 | −0.43* | 0.10 |
| Attitude toward supervision | −0.02 | 0.33 | 0.10 | −0.40* | 0.13 |
| Personal presentation | 0.20 | 0.29 | −0.13 | −0.38 | 0.09 |
*p < 0.05.
**p < 0.01.
aHigher scores indicate worse performance.
bHigher scores indicate better performance.
cHigher scores indicate more clinical symptoms.
dPartial correlations are presented, adjusting for effects of age, years of education.
C-FEIT, Chinese Facial Identification Test; ToM, Theory of mind; JTC, Jumping-to-conclusions; PAS, Paranoid/hostile attributional style; NC, Neurocognitive subscale of SCSQ.
Known-Groups Validity
To compare the social cognitive performance between the patient and non-psychiatric control groups, a matched sample of patient and control groups was recruited (19 pairs). We found that there were significant differences in social cognitive performance between persons with schizophrenia and non-psychiatric controls for the C-SCSQ scores, but not for the C-FEIT scores (Table 4). As with our expectation, the non-psychiatric controls had significantly higher scores for ToM (t = −3.89, p < 0.001, d = 1.26), NC (t = −5.61, p < 0.001, d = 1.82), and lower PAS (t = 10.07, p < 0.001, d = −3.27) scores than the patient group. It was noteworthy that the patient group had much higher PAS scores than the controls. An unexpected finding was that the non-psychiatric control group had higher JTC tendency scores than the patient group. As the non-psychiatric control group had significantly more years of education than the patient group (t = −5.47, p < 0.0001), we conducted analysis to control this potential confounding variable. When education was added as covariate, significant results maintained that the non-psychiatric controls had significant higher scores for ToM (F = 13.12, p = 0.001), NC (F = 11.04. p = 0.002), and lower PAS (F = 59.13, p < 0.0001). We included the t-test result in the final table as education did not correlate with performance in the five social cognitive domains.
Table 4.
Comparison of social cognitive performance between matched pairs of persons with schizophrenia and non-psychiatric controls.
| Persons with schizophrenia (n = 19) | Non-psychiatric controls (n = 19) | ||||||
|---|---|---|---|---|---|---|---|
| M | SD | M | SD | t | p | d | |
| EP | 13.63 | 3.9 | 15.47 | 2.5 | −1.72 | 0.094 | 0.56 |
| ToM | 6.32 | 1.2 | 7.84 | 1.2 | −3.89 | <0.001 | 1.26 |
| JTC | 1.68 | 1.2 | 2.42 | 1.0 | −2.05 | 0.047 | 0.67 |
| PAS | 2.93 | 0.6 | 0.68 | 0.7 | 10.07 | <0.001 | −3.27 |
| NC | 13.05 | 2.1 | 16.68 | 1.9 | −5.61 | <0.001 | 1.82 |
EP, emotion perception, ToM, theory of mind, JTC, jumping-to-conclusions, PAS, paranoid/hostile attributional style, NC, neurocognitive subscale of SCSQ.
Discussion
The study results show that there are strengths and limitations in using C-FEIT and C-SCSQ as a set of social cognitive assessment for assessing patients with schizophrenia. First, the assessment set has good content validity and cultural relevance. After some minor modifications to the test items of the C-SCSQ, the assessment set is ready for further testing. Second, the two social cognitive tasks have fair to good test–retest reliability. FEIT and two SCSQ subscales (JTC, PAS) have ICC over 0.80, and the ToM and NC subscales of the SCSQ have fair reliability—0.76 and 0.67, respectively. Third, the neurocognitive (NC) subscale of the SCSQ correlates significantly with three standardized neurocognitive tests (TMT, HVLT, VF Test). This result supports the concurrent validity of the NC subscale. The strength and direction of correlations between FEIT, ToM, PAS, and neurocognitive tests were largely comparable to that of previous studies which employed other social cognitive assessment tasks (2, 50). Fourth, the four SCSQ subscales are able to differentiate persons with schizophrenia and non-psychiatric controls, although the FEIT scores are marginally insignificant between the two groups. In particular, the non-psychiatric controls have significantly higher ToM, and NC subscales and lower PAS scores than the persons with schizophrenia. The effect sizes are large for the ToM, PAS, and NC subscales, which support the known-groups validity of the SCSQ. However, the difference in FEIT score is small (d = 0.56) between the two groups. This effect size is a bit lower in magnitude than in previous studies (30). In general, these results provide support for the known-groups validity of the assessment set.
On the other hand, there are also several aspects of the results that are not completely consistent with our expectations or with the results of previous studies. First, we note that the two social cognitive assessment tasks (C-FEIT and C-SCSQ) have low and insignificant correlations with clinical symptoms (BPRS) in schizophrenia. This is not completely in line with the results of previous studies (51). This result may have arisen because of the complex interaction among psychiatric symptoms, neurocognition, and social cognition (1). Another possible explanation for this result is that the patient group had relatively few psychiatric symptoms.
Second, the ToM, PAS, and JTC subscales of the SCSQ have low-to-medium correlations with measures of functional performance (CWPP) after controlling for age and education. These results echo previous findings on the predictive significance of social cognition in functional outcomes (4, 6, 52). However, some of these correlations appear to be insignificant because of the small sample size.
Third, unlike the internal consistency of C-FEIT being acceptable (53), the internal consistency of C-SCSQ total score falls into questionable range (53). This is likely related to the inclusion of different C-SCSQ subscales that measure different but related social cognitive domains. It would be best to examine the internal consistency of C-SCSQ subscales individually. This was not conducted as some of the subscales had very limited number of items, ToM (10 items) and AS (5 items), which could significantly impact on the internal consistency (54).
There are also some limitations in the methodology that may affect the interpretation and generalizability of the study results. First, there are very few Chinese-language versions of social cognitive assessment tools available for studying the concurrent validity of the assessment set. For instance, the Faux Pas tasks and Eyes Tasks have been commonly used to measure ToM in previous studies, but these may not be the best concurrent measures as these tests measure other facets of ToM ability but not the intention-inferencing ability of the C-SCSQ. Second, more comprehensive functional measures based on performance assessment could be adopted in future studies. The CWPP is an observational assessment of patients’ functional performance in simulated work situations. One option to further improve the validity of functional assessment is to adopt performance-based tests; this is now generally preferred for more direct and more valid estimates of functional disability (55). Third, the small sample size of the study could lead to some of the unexpected findings, such as non-psychiatric controls had higher JTC score than patient group. While the effect sizes for between-group differences in other social cognitive domains could be assumed at 0.90 from previous studies, the combined effect size for JTC is 0.60 as confirmed in a recent meta-analysis (56). The finding of higher JTC score in non-psychiatric control is possibly related to the inadequate sample size. Fourth, our study used static faces in C-FEIT to measure EP ability, but some studies recommended the use of dynamic faces over static faces for better ecological validity (57, 58). These findings, however, do not have strong support from behavioral study (59). Nevertheless, there is a need to further discuss if dynamic or static faces are more valid for future studies of EP in future. Fourth, we acknowledged that an optimal test–retest interval would be 2–4 weeks (60). The use of retest interval of 1 week as it would greatly increase the probability that participants would come for retest. The sample size for evaluating test–retest reliability also needs to around 60 in order to obtain a powerful estimate of the ICC for this test (31). Last, the C-FEIT and C-SCSQ cover most aspects of social cognitive assessment, but social perception (or social knowledge) is not covered by these two tests. There is a need to include a sub-test of social perception to form a battery that provides a comprehensive evaluation of social cognition.
In summary, the C-FEIT and C-SCSQ generally possess acceptable to good psychometric properties for assessing the key social cognitive domains, including EP, intention-inferencing ability (ToM), and paranoid attributional style in persons with schizophrenia. The assessment set demonstrates satisfactory test–retest reliability and known-groups validity. Subscales of the instruments have low to moderate relationships with neurocognitive function and are predictive of some domains of functional performance. The social cognitive assessment set, consisting of the C-FEIT and the C-SCSQ, addresses the need for social cognitive assessment with a Chinese population. Nevertheless, applications of the two instruments should be carried out with caution in certain populations. The C-FEIT may not differentiate EM performance between patients with few psychotic symptoms and non-psychiatric controls, and the JTC scores of non-psychiatric controls need to be interpreted carefully. Future studies can further investigate the psychometric properties of the two assessment tasks through collecting data on discriminant validity using larger healthy and patient samples and exploring the predictive validity of the assessment set with a wider range of performance-based functional measures.
Ethics Statement
This study was carried out in accordance with the recommendations of Hospital Authority Guide on Research Ethics with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Clinical & Research Ethics Committee of the New Territories West Cluster, Hospital Authority.
Author Contributions
PL initiates the research, conducts data collection and analysis, and drafts the manuscript. AS designs on research methodology and writes the manuscript. Both authors approve the final version to be published and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
The authors acknowledge the support and assistance from staff of the New Life Psychiatric Association and staff of the Providence Garden of Rehabilitation on subject recruitment and data collection.
References
- 1.Addington J, Saeedi H, Addington D. Facial affect recognition: a mediator between cognitive and social functioning in psychosis? Schizophr Res (2006) 85:142–50. 10.1016/j.schres.2006.03.028 [DOI] [PubMed] [Google Scholar]
- 2.Bell M, Tsang HW, Greig TC, Bryson GJ. Neurocognition, social cognition, perceived social discomfort, and vocational outcomes in schizophrenia. Schizophr Bull (2009) 35:738–47. 10.1093/schbul/sbm169 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Fujii DE, Wylie AM. Neurocognition and community outcome in schizophrenia: long-term predictive validity. Schizophr Res (2003) 59:219–23. 10.1016/S0920-9964(01)00328-0 [DOI] [PubMed] [Google Scholar]
- 4.Gard DE, Fisher M, Garrett C, Genevsky A, Vinogradov S. Motivation and its relationship to neurocognition, social cognition, and functional outcome in schizophrenia. Schizophr Res (2009) 115:74–81. 10.1016/j.schres.2009.08.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Horan WP, Kern RS, Shokat-Fadai K, Sergi MJ, Wynn JK, Green MF. Social cognitive skills training in schizophrenia: an initial efficacy study of stabilized outpatients. Schizophr Res (2009) 107:47–54. 10.1016/j.schres.2008.09.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lo P, Siu AM. Social cognition and work performance of persons with schizophrenia in a Chinese population. Work (2015) 50:629–36. 10.3233/WOR-141828 [DOI] [PubMed] [Google Scholar]
- 7.Marcopulos BA, Kurtz MM. Clinical Neuropsychological Foundation of Schizophrenia. New York: Routledge; (2012). [Google Scholar]
- 8.Fiske ST, Taylor SE. Social Cognition. 2nd ed New York: McGraw-Hill; (1991). [Google Scholar]
- 9.Green MF, Horan WP, Lee J. Social cognition in schizophrenia. Nat Rev Neurosci (2015) 16:620–31. 10.1038/nrn4005 [DOI] [PubMed] [Google Scholar]
- 10.Green MF, Penn DL, Bentall R, Carpenter WT, Gaebel W, Gur RC, et al. Social cognition in schizophrenia: an NIMH Workshop on definitions, assessment, and research opportunities. Schizophr Bull (2008) 34:1211–20. 10.1093/schbul/sbm145 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Pinkham AE, Penn DL, Green MF, Buck B, Healey K, Harvey PD. The social cognition psychometric evaluation study: results of the expert survey and RAND panel. Schizophr Bull (2013) 40:813–23. 10.1093/schbul/sbt081 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Baron-Cohen S, Weelwright S, Hill J, Raste Y, Plumb I. The “Reading the Mind in the Eyes” Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. J Child Psychol Psychiatry (2001) 42:241–51. 10.1111/1469-7610.00715 [DOI] [PubMed] [Google Scholar]
- 13.Sprong M, Schothorst P, Vos E, Hox J, Engeland HV. Theory of mind in schizophrenia: meta-analysis. Br J Psychiatry (2007) 191:5–13. 10.1192/bjp.bp.107.035899 [DOI] [PubMed] [Google Scholar]
- 14.Mayer JD, Salovey P, Caruso DR, Sitarenios G. Measuring emotional intelligence with the MSCEIT V2. 0. Emotion (2003) 3:97–105. 10.1037/1528-3542.3.1.97 [DOI] [PubMed] [Google Scholar]
- 15.Peterson C, Semmel A, Baeyer CV, Abramson LY, Metalsky GI, Seligman MEP. The attributional style questionnaire. Cogn Ther Res (1982) 6:287–99. 10.1007/BF01173577 [DOI] [Google Scholar]
- 16.Combs DR, Penn DL, Melanie W, Waldheter E. The ambiguous intentions hostility questionnaire (AIHQ): a new measure for evaluating hostile social-cognitive biases in paranoia. Cogn Neuropsychiatry (2007) 12:128–43. 10.1080/13546800600787854 [DOI] [PubMed] [Google Scholar]
- 17.Freeman D. Suspicious minds: the psychology of persecutory delusions. Clin Psychol Rev (2007) 27:425–57. 10.1016/j.cpr.2006.10.004 [DOI] [PubMed] [Google Scholar]
- 18.Roberts DL, Penn DL. Social cognition and interaction training (SCIT) for outpatients with schizophrenia: a preliminary study. Psychiatry Res (2009) 166:141–7. 10.1016/j.psychres.2008.02.007 [DOI] [PubMed] [Google Scholar]
- 19.Huang CLC, Hsiao S, Hwu HG, Howng SL. The Chinese facial emotion recognition database (CFERD): a computer-generated 3-D paradigm to measure the recognition of facial emotion expressions at different intensities. Psychiatr Res (2012) 200:928–32. 10.1016/j.psychres.2012.03.038 [DOI] [PubMed] [Google Scholar]
- 20.Leung JS, Lee TM, Lee CC. Facial emotion recognition in Chinese with schizophrenia at early and chronic stages of illness. Psychiatry Res (2011) 190:172–6. 10.1016/j.psychres.2011.07.001 [DOI] [PubMed] [Google Scholar]
- 21.Pan YJ, Chen SH, Chen WJ, Liu SK. Affect recognition as an independent social function determinant in schizophrenia. Compr Psychiatry (2009) 50:443–52. 10.1016/j.comppsych.2008.11.003 [DOI] [PubMed] [Google Scholar]
- 22.Biehl M, Matsumoto D, Ekman P, Hearn V, Heider K, Kudoh T, et al. Matsumoto and Ekman’s Japanese and Caucasian facial expressions of emotion (JACFEE): reliability data and cross-national differences. J Nonverbal Behav (1997) 21:3–21. 10.1023/A:1024902500935 [DOI] [Google Scholar]
- 23.Elfenbein HA, Ambady N. Is there an in-group advantage in emotion recognition? Psychol Bull (2002) 128:243–9. 10.1037/0033-2909.128.2.243 [DOI] [PubMed] [Google Scholar]
- 24.Matsumoto D, Ekman P. American-Japanese cultural differences in intensity ratings of facial expressions of emotion. Motiv Emotion (1989) 13:143–57. 10.1007/BF00992959 [DOI] [Google Scholar]
- 25.Triandis HC. Individualism-collectivism and personality. J Pers (2001) 69:907–24. 10.1111/1467-6494.696169 [DOI] [PubMed] [Google Scholar]
- 26.Ekman P. Are there basic emotions? Psychol Rev (1992) 99:550–3. 10.1037/0033-295X.99.3.550 [DOI] [PubMed] [Google Scholar]
- 27.Ekman P, Friesen WV, Ellsworth P. Emotion in the Human Face: Guidelines for Research and an Integration of Findings. New York: Pergamon Press; (1972). [Google Scholar]
- 28.Kohler CG, Turner TH, Bilker WB, Brensinger CM, Siegel SJ, Kanes SJ, et al. Facial emotion recognition in schizophrenia: intensity effects and error pattern. Am J Psychiatry (2003) 160:1768–74. 10.1176/appi.ajp.160.10.1768 [DOI] [PubMed] [Google Scholar]
- 29.Going M, Read JD. Effects of uniqueness, sex of subject, and sex of photograph on facial recognition. Percept Mot Skills (1974) 39:109–10. 10.2466/pms.1974.39.1.109 [DOI] [Google Scholar]
- 30.Kohler CG, Walker JB, Martin EA, Healey KM, Moberg PJ. Facial emotion perception in schizophrenia: a meta-analytic review. Schizophr Bull (2010) 36:1009–19. 10.1093/schbul/sbn192 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Hintze J. PASS 12. Kaysville, Utah, USA: NCSS, LLC; (2013). Available from: www.ncss.com [Google Scholar]
- 32.Bora M, Yucel M, Panetils C. Theory of mind impairment in schizophrenia: meta-analysis. Schizophr Res (2009) 109:1–9. 10.1016/j.schres.2008.12.020 [DOI] [PubMed] [Google Scholar]
- 33.Horan WP, Green MF, DeGroot M, Fiske A, Hellemann G, Kee K, et al. Social cognition in schizophrenia: part 2: 12-month stability and prediction of functional outcome in first-episode patients. Schizophr Bull (2011) 38:865–72. 10.1093/schbul/sbr001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Kerr SL, Neale JM. Emotion perception in schizophrenia: specific deficit or further evidence of generalized poor performance? J Abnorm Psychol (1993) 102:312–8. 10.1037/0021-843X.102.2.312 [DOI] [PubMed] [Google Scholar]
- 35.Matsumoto D, Ekman P. Japanese and Caucasian Facial Expression of Emotion (JACFEE) and Neutral Faces (JACNeuF). [Assessment Manual]. San Francisco: University of California; (1988). [Google Scholar]
- 36.Yip JTH, Lee TMC. Cultural differences in facial emotion recognition. Brain Cogn (2003) 51:169–70. 10.1016/S0278-2626(02)00541-9 [DOI] [Google Scholar]
- 37.Marsh AA, Elfenbein HA, Ambady N. Nonverbal “accents”: cultural differences in facial expressions of emotion. Psychol Sci (2003) 14:373–6. 10.1111/1467-9280.24461 [DOI] [PubMed] [Google Scholar]
- 38.Gao W, Cao B, Shan S, Chen X, Zhou D, Zhang X, et al. The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Trans Syst Man Cybern B Syst Man Hum (2008) 38:149–61. 10.1109/TSMCA.2007.909557 [DOI] [Google Scholar]
- 39.Roberts DL, Kleinlein P, Stevens B. An alternative to generating alternative interpretations in social cognitive therapy for psychosis. Behav Cogn Psychother (2012) 40:491–5. 10.1017/S1352465812000082 [DOI] [PubMed] [Google Scholar]
- 40.Lukoff D, Nuechterlein KH, Ventura J. Manual for the expanded brief psychiatric rating scale. Schizophr Bull (1986) 12:594–602. [Google Scholar]
- 41.Peer JE, Rothmann TL, Penrod RD, Penn DL, Spaulding WD. Social cognitive bias and neurocognitive deficit in paranoid symptoms: evidence for an interaction effect and changes during treatment. Schizophr Res (2004) 71:463–71. 10.1016/j.schres.2004.03.016 [DOI] [PubMed] [Google Scholar]
- 42.Hafkenscheid A. Reliability of a standardized and expanded brief psychiatric rating scale: a replication study. Acta Psychiatr Scand (1993) 88:305–10. 10.1111/j.1600-0447.1993.tb03464.x [DOI] [PubMed] [Google Scholar]
- 43.Nuechterlein KH, Green MF, Kern RS, Baade LE, Barch DM, Cohen JD, et al. The MATRICS consensus cognitive battery, part 1: test selection, reliability and validity. Am J Psychiatry (2008) 165:203–13. 10.1176/appi.ajp.2007.07010042 [DOI] [PubMed] [Google Scholar]
- 44.Luteijn F, Van der Ploeg FAE. Groninger Intelligentie Test: Handleiding [Groninger Intelligence Test: Manual]. Lisse: Swets, Zeitlinger BV; (1983). [Google Scholar]
- 45.Brandt J. The Hopkins verbal learning test: development of a new memory test with six equivalent forms. Clin Neuropsychol (1991) 5:125–42. 10.1080/13854049108403297 [DOI] [Google Scholar]
- 46.Law CK, Siu AM, Lee JL, Lee SW. Prediction of work rehabilitation placements using the Chinese work personality profile. Psychiatr Rehabil J (2006) 30:120–8. 10.2975/30.2006.120.128 [DOI] [PubMed] [Google Scholar]
- 47.Weir JP. Quantifying test-retest reliability using the intraclass classification coefficient and the SEM. J Strength Cond Res (2005) 19:231–40. 10.1519/00124278-200502000-00038 [DOI] [PubMed] [Google Scholar]
- 48.Sergi MJ, Green MF, Widmark C, Resit C, Erhart S, Braff DL, et al. Social cognition and neurocognition: effects of risperidone, olanzapine and haloperidol. Am J Psychiatry (2007) 164:1585–92. 10.1176/appi.ajp.2007.06091515 [DOI] [PubMed] [Google Scholar]
- 49.Portney LG, Watkins MP. Foundations of Clinical Research: Application to Practice. New Jersey: Prentice-Hall Health; (2000). [Google Scholar]
- 50.Mancuso F, Horan WP, Kern RS, Green MF. Social cognition in psychosis: multidimensional structure, clinical correlates, and relationship with functional outcome. Schizophr Res (2011) 125:143–51. 10.1016/j.schres.2010.11.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Bowie CR, Reichenberg A, Patterson TL, Heaton RK, Harvey PD. Determinants of real-world functional performance in schizophrenia subjects: correlations with cognition, functional capacity, and symptoms. Am J Psychiatry (2006) 163:418–25. 10.1176/appi.ajp.163.3.418 [DOI] [PubMed] [Google Scholar]
- 52.Fett AK, Viechtbauer W, Dominguez M, Penn DL, Os JV, Krabbendam L. The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: a meta-analysis. Neurosci Biobehav Rev (2011) 35:573–88. 10.1016/j.neubiorev.2010.07.001 [DOI] [PubMed] [Google Scholar]
- 53.George D, Mallery P. SPSS for Windows Step by Step: A Simple Guide and Reference. 4th ed Boston: Allyn & Bacon; (2003). [Google Scholar]
- 54.Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika (1951) 16:297–334. 10.1007/BF02310555 [DOI] [Google Scholar]
- 55.Harvey PD, Velligan DI, Bellack AS. Performance-based measures of functional skills: usefulness in clinical treatment studies. Schizophr Bull (2007) 33:1138–48. 10.1093/schbul/sbm040 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.So SHW, Siu NYF, Wong HL, Chan W, Garety PA. Jumping to conclusions’ data-gathering bias in psychosis and other psychiatric disorders—two meta-analyses of comparisons between patients and healthy individuals. Clin Psychol Rev (2016) 46:151–67. 10.1016/j.cpr.2016.05.001 [DOI] [PubMed] [Google Scholar]
- 57.Kilts CD, Egan G, Gideon DA, Ely TD, Hoffman JM. Dissociable neural pathways are involved in the recognition of emotion in static and dynamic facial expressions. Neuroimage (2003) 18:156–68. 10.1006/nimg.2002.1323 [DOI] [PubMed] [Google Scholar]
- 58.Sato W, Sakiko Y. Enhanced experience of emotional arousal in response to dynamic facial expressions. J Nonver Behav (2007) 31:119–35. 10.1007/s10919-007-0025-7 [DOI] [Google Scholar]
- 59.Johnston PJ, Enticott PG, Mayes AK, Hoy KE, Herring SE, Fitzgerald PB. Symptom correlates of static and dynamic facial affect processing in schizophrenia: evidence of a double dissociation? Schizophr Bull (2010) 36:680–7. 10.1093/schbul/sbn136 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Falleti MG, Maruff P, Collie A, Darby DG. Practice effects associated with the repeated assessment of cognitive function using the CogState battery at 10-minute, one week and one month test-retest intervals. J Clin Exp Neuropsychol (2006) 28:1095–112. 10.1080/13803390500205718 [DOI] [PubMed] [Google Scholar]