The use of narrative sampling in the assessment of social cognition: The Narrative of Emotions Task (NET)

Abstract

Social cognitive deficits in schizophrenia are well documented and related to functional outcome. Current social cognition measures are often not psychometrically validated, too heterogeneous for standardization, and focus principally on one domain of social cognition rather than the simultaneous activation of multiple domains. Also, few if any allow for personalization of stimuli and interpretation of personally evocative events. An alternative methodology that addresses these limitations is the analysis of samples of personal narratives. The present study evaluates the psychometric properties of a measure called the Narrative of Emotions Task (NET). The NET was used to assess the performance of participants with a diagnosis of schizophrenia or schizoaffective disorder and nonclinical controls. Use of the NET revealed significant impairments in the emotional narratives of participants with schizophrenia. Various NET indices were significantly related to current measures of theory of mind and emotion perception, as well as a social skills role-play, but were not related to measures of attributional style or clinician-rated functioning scales. Overall, the NET's psychometric properties justify further use of the narrative sampling method of social cognition assessment in this population.

1. Introduction

Individuals with schizophrenia show pronounced deficits in social cognition, a construct broadly understood as the ability of persons to think about themselves and others in the social world (Penn, Sanna, & Roberts, 2008). These deficits are not redundant with neurocognitive impairments among individuals with schizophrenia (Sergi et al., 2007). Relative to non-clinical controls, individuals with schizophrenia are impaired in its subdomains, including theory of mind (Bora, Yucel, & Pantelis, 2009; Brune, 2005), emotion perception (Kohler et al., 2009), emotion processing (Cohen et al., 2008), and attributional style (Aakre et al., 2009). Social cognition is associated with functional outcomes (Couture et al., 2006; Fett et al., 2011; Kee et al., 2003) and responsive to psychosocial intervention (Kurtz et al., 2011).

Although the literature on social cognition in schizophrenia has grown considerably in recent years, many measures in the field have not been examined for their psychometric properties (Green et al., 2008). Because of this, meta-analyses of this literature face methodological challenges as tasks representing a given construct (e.g., Theory of Mind) are sometimes too heterogeneous (Hoekert et al., 2007) or have poor or unknown psychometric properties (Yager & Ehrman, 2006). These issues have prompted the ongoing Social Cognition Psychometric Evaluation (SCOPE) Study (Pinkham et al., in press) in which a combination of expert consultation and empirical study is currently seeking to validate consensus sub-domains and measures of the construct.

In addition to concerns about psychometrics, social cognition tends to engage cognitive operations that are closely related if not largely overlapping (e.g., the simultaneous co-operation of interpreting another's emotion while representing his or her mental state), while most current measures attempt to focus specifically on only one domain of social cognition. In addition, many social cognition tasks involve non-personal stimuli (photos of unknown individuals’ faces expressing a group of emotions), or otherwise contrived hypothetical scenarios that may be particularly detached from an individual's experience of the world and social interaction (McDonald et al., 2003). This is a critical limitation in that it detracts from the measures’ ecological validity, as social interactions and their related processes are often fraught with personal significance.

Analysis of narratives is a potential candidate for a personalized analysis of social cognition, as narratives provide a glimpse of how people idiosyncratically respond to their environment. Compared to use of standard social cognition measures, they provide insight into the way in which an individual makes sense of events, important others, and emotions across a range of situations. Evaluating the richness and clarity of these modes of thinking could provide insight into individuals’ quality of social cognition in a personalized and ecologically valid manner. In addition, narrative speech samples could allow for simultaneous evaluation of the overlapping processes associated with social cognition.

Narrative measures have been used with individuals with emotional and social deficits. Losh and Capps (2006) conducted interviews with children with high-functioning autism (HFA) eliciting emotional narratives (e.g., “Tell me a time when you felt...”) covering a range of emotions including simple (happy, sad), complex (curious, disappointed), self-conscious (proud, embarrassed), and non-emotions (tired and sick). Compared to typically developing children, the most pronounced deficits for the HFA children were for narratives pertaining to self-conscious emotions, particularly the ability to organize narratives according to causal-explanatory frameworks. Similar methodology was used by Gruber and Kring (2008) with a schizophrenia sample, finding that individuals with schizophrenia told emotional narratives that were less appropriate to context and less linear, as well as narratives (emotional or not) that were less tellable and more detached compared to controls.

For the present study, we adapted methods used by Losh and Capps (2006) and Gruber and Kring (2008) into a measure of social cognition based on personal narrative samples of people with schizophrenia, called the Narrative of Emotions Task (NET). Thus, the primary aim was to examine the psychometric properties of the NET as a new measure of social cognition using narrative sampling.

2. Methods

2.1. Participants

Participants meeting DSM-IV criteria for either schizophrenia or schizoaffective disorder were recruited from the UNC Hospitals Schizophrenia Treatment and Evaluation Program (STEP), the Outreach and Support Intervention Services (OASIS) program, and community mental health facilities in the Raleigh-Durham region. Interviewers reviewed participants’ medical charts, confirming diagnosis by administering the Structured Clinical Interview for DSM-IV Patient Edition (SCID-P; First et al., 1996). In order to participate, individuals had to report difficulties interacting with others per the Social Functioning Scale (Birchwood et al., 1990), as they were participating in a study evaluating the efficacy of social cognition and interaction training (SCIT), a 20-24 week psychosocial intervention targeting deficits in social cognition (Roberts et al., in press). Of 137 referred participants, 66 met screening criteria and were randomized to the SCIT study. All data examined in the current study were collected at the baseline visit of the study. Because the NET was added to the study protocol as the SCIT study was in progress, the present study includes the subset of participants that completed the NET interview (n = 45). Individuals were excluded if they currently met DSM-IV criteria for substance dependence on the SCIDP, or scored an IQ of 80 or lower on the Wechsler Abbreviated Scales of Intelligence (WASI; Whitmyre & Pishkin, 1958).

A control group consisting of fifty English-speaking non-psychiatric controls from the Raleigh-Durham area was recruited with flyers and Internet postings. All non-psychiatric controls were between the ages of 20 and 65 years old and reported no first-degree relatives with a psychotic disorder, bipolar disorder, or autism. As summarized in Table 1, the groups significantly differed in marital status, education level, and on WASI total, as well as on the vocabulary subscale. The groups approached a significant difference in WASI matrix reasoning.

Table 1.

Demographics of study participants.

	Group
	Controls (n = 50)	SCZ (n = 45)	Test for differences
Age	39.86 (9.85)	38.54 (12.14)	t = .58 p = .56
Education (years)	13.40 (1.18)	12.22 (1.30)	t = 4.64**p = <.001
Gender			X2 = .01 p = .95
Male	33 (66.0%)	30 (66.7%)
Female	17 (34.0%)	15 (33.3%)
Race			X2 = .13 p = .71
White	34 (68.0%)	29 (64.4%)
Black	16 (32.0%)	16 (35.6%)
Marital Status			X2 = 24.66**p = <.001
Married	23 (46.0%)	1 (2.2%)
Single/Never Married	20 (40.0%)	31 (68.9%)
Divorced	7 (14.0%)	12 (26.7%)
Widowed	0 (0.0%)	1 (2.2%)
Age of first hospitalization	-	23.09 (9.16)	- -
Number of previous hospitalizations	-	6.43 (7.13)	- -
Symptom severity ratings
PANSS Positive	-	16.13 (4.80)
PANSS Negative	-	14.96 (4.38)
PANSS General	-	33.44 (7.42)
PANSS Total	-	64.53 (13.00)
WASI Total#	110.80 (15.00)	100.33 (15.12)	t = 3.38**p < .01
WASI Vocabulary	56.26 (9.71)	48.18 (11.21)	t = 3.77**p < .01
WASI Matrix Reasoning	55.50 (10.14)	51.36 (10.17)	t = 1.99 p = .05

^P<.10

*P< 0.05

^**P < 0.01

#FSIQ totaled from Vocabulary and Matrix Reasoning subscales

2.2. Development of the NET

2.2.3. The NET interview

The Narrative of Emotions Task consists of an interview prompting participants to define a range of simple (happy, sad, angry, and afraid), complex (suspicious and surprised), and self-conscious (guilty and ashamed) emotions (Losh & Capps, 2006; Stipek, Recchia & McClintic, 1992). They were asked to define the emotion (“What does happy mean?”), provide a narrative account involving the emotion (“Tell me about a time when you felt happy.”), and explain why the described event elicited the target emotion (“Why did that make you feel happy?”). This interview structure is similar to the method adapted by Losh and Capps (2006) from earlier work in the developmental literature (Seidner et al., 1988) with a few changes. For example, in addition to prompting individuals to share narratives of experiences during which they felt the target emotion, the third interview question of the NET specifically asks individuals to offer causal circumstances for the event. The NET differs from other narrative methods in that it focuses on brief samples of narrated speech rather than spontaneously generated longer narratives, like, for example, the Indiana Psychiatric Illness Interview (Lysaker et al., 2005) which has been used to assess metacognition in schizophrenia. NET interviews generally require between 8 and 12 minutes to administer.

2.2.3. Narrative of Emotions Task; item generation

The NET consists of eight individual scales that are rated on a zero-to-three scale for each emotional narrative, with higher scores indicating better performance. Items assess the correctness of definition provided for each emotion, whether an account is given in narrative form (“two or more temporally conjoined clauses that represent a sequence of events”; Labov & Waletsky, 1968; Losh & Capps, 2006), the contextual appropriateness of the narrative account, plausibility of causal inferences (why the given event elicited the target emotion), clarity of meaning, clarity of grammar, sociality (the extent to which others were involved), and elaboration (the richness of narrative account, as evaluated by the amount of information provided about time, place, or other relevant details). Because the scale aims to assess how the individual narrates events and engages in social cognition across events of differing emotional valence, these “narrative domains” are totaled across all emotions.

Narrative domains on the NET were, in turn, summed to create a variety of social cognition indices. Specifically, the definition of emotion and contextual appropriateness scales assess identification and understanding of emotional states. Therefore, they are summed to create an emotion perception index. Social interaction draws on one's theory of mind abilities in that individuals give adequate amounts of information when aware of the listener's relative lack of knowledge on a subject or story (Corcoran & Frith, 1996; Frith, 2004; Grice, 1975). Therefore, the level of specificity and richness one offers should be indicative of theory of mind abilities. In addition, one's ability to give a rich narrative of social or emotional events is dependent on one's ability to represent mental states in general. We define theory of mind here in a broader sense, as indicated in representations of both one's own and others’ mental states, particularly mindful of the representation involved in inferring one's own past (rather than current) mental states. As such, elaboration and presence of narrative scales assess the richness and coherence with which one explains emotional and social events. Therefore, they are summed together to create a theory of mind index. The causal inferences scale explores one's explanation for why certain emotions were elicited during given events. Given the tendency of people with schizophrenia to personalize and externalize attributions for negative events (Kinderman & Bentall, 1997; Combs et al., 2007), this could be a relevant domain in determining how rich and nuanced a person's attributions for emotional events are. Therefore, this causal inferences domain represents an attributional index. Importantly, the NET assesses how accurate and thorough participants are in providing attributions for others’ behavior. According to this logic, an inability to generate accurate and thorough causal attributions for others results in either bias (Aakre et al., 2009) or confusion.

Intraclass correlations were calculated for the social cognition indices between two trained raters and a gold standard rater who co-developed the scale (BB) on a randomly-selected subset of NETs including both participants with schizophrenia (n = 17) and controls (n = 6). The intraclass coefficients were acceptable to very good for the emotion perception (ICC = .63, .69), and theory of mind indices (ICC = .83, .89), and NET total score (ICC = .84, .91). Both the sociality score (.55, .85) and attribution index (ICC = .45, .79) had one rater that did not reach acceptable reliability. The unreliable NET transcripts were not included in the current sample; all NETs were rated either by the gold standard rater or the adequately reliable rater. The NET generally requires between 10 and 15 minutes per interview to score (not including transcription time), given the number of items and the scale's attention to specific words in each utterance.

2.3. Social cognition measures

2.3.1. Emotion perception

Emotion perception was assessed using two related measures. The Face Emotion Identification Test (FEIT; Kerr & Neale, 1993) asks participants to identify the emotions expressed by 19 faces depicting six basic emotions (happy, sad, afraid, angry, surprised, and ashamed), and scores are totaled as number correct out of 19. The Face Emotion Discrimination Task (FEDT; Kerr & Neale, 1993) asks participants to determine whether two paired faces are expressing the same or different emotions out of a total of 30 pairs, with performance indexed as number correct out of 30. These two measures are significantly correlated with one another, r = .43, p < .01. In the interest of parsimony, each individual's score on these measures were converted to z-scores, and summed to create a composite index of emotion perception for convergent validity analyses.

2.3.2. Theory of mind

Theory of Mind was assessed with two measures. The Hinting Task (Corcoran et al., 1995) involves participants interpreting ten brief written stories that require them to identify and make inferences involving others’ mental states. Scores range from 0 to 20 on the Hinting Task, with higher scores indicating better performance. The Awareness of Social Inference Test – Part II (TASIT; McDonald et al., 2003) consists of Yes/No questions related to four video-taped social vignettes requiring individuals to infer individual motives which may contradict verbal communication (e.g., sarcasm). The TASIT is scored based on number of correct responses out of 60 possible. The TASIT and Hinting Tasks are significantly related to one another, r = .39, p < .01. Again, in the convergent validity analyses, these two scales were converted into z-scores and summed to create a comprehensive measure of ToM (i.e. one that considers both verbal and visual ToM cues).

2.3.3. Attributional style

The Ambiguous Intentions Hostility Questionnaire, Ambiguous Items (AIHQ-A; Combs et al., 2007) consists of five second-person vignettes of negative social situations with ambiguous causal circumstances (e.g., “you are walking by a group of young people who laugh as you pass by”). Participants rate the following on a Likert scale: the level of intention on the other's part (definitely not on purpose to definitely on purpose) how angry it would make them feel (not at all angry to very angry) and how much they would blame the other (not at all to very much). These are standardized and totaled for an overall “blame index.” Following the interview, two independent raters compute a hostility bias related to interpretation of the other's action (a five point Likert scale from not at all hostile to very hostile) and an aggression bias related to the individual's response to the action. Intraclass correlations were computed and were > .70 for both bias measures.

2.4. Functional Measures

2.4.1. Social functioning

The Social Skills Performance Assessment (SSPA; Patterson et al., 2001) is an observer-rated assessment of social skill performance in two three-minute role-play conversations with a confederate. Scores range from 1 to 5 on each subscale, with higher scores indicating better performance. Outcomes of interest for the present study included a paralinguistics total (performance on speech fluency and clarity summed across both role-plays), participation total (performance on interest and focus summed across both role-plays), as well as total score for affect and social appropriateness (individually-rated scales summed across role-plays). In addition to these subscores, total scores were calculated for each role-play individually as the sum of these subscales. Intraclass correlations were computed and all were greater than .80 for all subscales.

2.4.2. Role functioning

The Role Functioning Scale (RFS; McPheeters, 1984) is an interviewer-rated assessment of functioning based on a semi-structured interview covering four domains: independent living, work performance, as well as immediate and extended work social relationships. Scores on this scale range from 1 to 7, with higher scores indicating better functioning.

2.5. Intellectual quotient

The Wechsler Abbreviated Scales for Intelligence (WASI) is a brief version of a full assessment of intelligence quotient, comprised of our subtests of the full Wechsler Adult Intelligence Scales (WAIS): block design, similarities, vocabulary, and matrix reasoning. To minimize the length of long study visits, only select subscales of the WASI were administered. Participants were administered the vocabulary subscale as representative of Verbal IQ, and the matrix reasoning subscale as representative of Performance IQ. Total WASI scores were generated from these two subscales.

2.6. Psychiatric symptoms

The Positive and Negative Syndrome Scale (PANSS; Kay et al., 1987) is an interview-based measure comprised of 30 items assessing for positive and negative symptoms of schizophrenia, as well as general psychopathology symptoms. Interviews were completed by graduate students or trained staff with experience working with a schizophrenia-spectrum population.

2.7. Procedure

Advanced graduate students and staff with experience working with this population conducted all interviews comprising NET, social cognition, and functioning measures, which were completed at the baseline visit of the SCIT study (Roberts et al., in press).

2.8. Data analytic plan

Construct validity was explored by examining group differences between participants with schizophrenia and non-clinical controls. We conducted t-tests comparing group performance on each domain of the NET, as well as on the NET total score, hypothesizing significant differences in each domain. We repeated these analyses covarying for verbal IQ (ANCOVA) to acknowledge the likely influence of general verbal intelligence on NET performance, as it is also a verbal task. We also examined the inter-correlations among the social cognition indices for each group separately.

Convergent validity was evaluated via the relationship between the NET and the social cognition measures. Specifically, the NET emotion perception index was correlated with the standardized emotion perception measure. The NET theory of mind index was correlated with the standardized composite score from the two measures of theory of mind. The NET attribution index was correlated with each scale of attributional bias: hostility bias, aggression bias, blame index, and AIHQ total. All correlations were hypothesized to be positive, except for the measures of attributional bias, which were hypothesized to be negative. The relationships of NET Total scores were also examined with all social cognition comparison measures.

Ecological validity was explored through a correlational analysis of the relationship between the NET total and total scores on the SSPA and RFS, as well as a specific correlational analysis of the NET domain of Sociality with the immediate social network and extended social network scales of the RFS. It was hypothesized that all of these correlations would be positive.

3. Results

3.1. Distributions

We defined outliers as scores two standard deviations above or below the mean. We found nine total outlier data points across all subscales and total scores on the NET, from five individual participants: two from the control group, and three from the schizophrenia group. The influence of outliers was reduced by winsorizing them, so that they were within the two standard deviation range. We then examined the normality of NET total and subscales using the Shapiro-Wilk's test of normality. Two subscales deviated from normality: the attributions index (.94, p = .04) in the schizophrenia sample and the sociality score among controls (.95, p = .03). The attributions index among participants with schizophrenia was slightly negatively skewed (skewness = -.28), while the sociality score among controls was highly leptokurtic (kurtosis = 3.28).

3.2. Internal consistency

Internal consistency was relatively high for the Theory of Mind index (α = .73), NET total score (α = .80), but was lower for the emotion perception (α = .52) and attribution indices (α = .47), and sociality scores (α = .52).

3.3. Construct validity

To examine construct validity, we examined differences between controls and participants with schizophrenia. Groups significantly differed in total score, Emotion Perception Index, Theory of Mind Index and Attributions Index. The means for each variable in the control and patient groups can be found in Table 2. These analyses were repeated using an ANCOVA with Verbal IQ as a covariate. The results were generally unchanged with the exception of the theory of mind index, which now approached statistical significance, F(1,91) = 2.83, p = .10.

Table 2.

Means and standard deviations for schizophrenia and control groups.

	NET Scores				Cohen's d
	Controls (n = 50)	SCZ (n = 45)	t	p
Emotion Perception Index	42.26 (3.11)	39.43 (3.68)	4.04**	.000	.83
Theory of Mind Index	33.57 (7.67)	28.96 (7.78)	2.90**	.005	.60
Attributions Index	17.19 (2.20)	15.09 (2.07)	4.68**	.000	.98
NET Total	135.06 (12.62)	123.86 (13.38)	4.09**	.000	.86
NET Sociality Score	19.04 (3.06)	18.58 (3.21)	.72	.476	.14

^P<.10

*P< 0.05

^**P < 0.01

Inter-correlations between the social cognition indices can be found in Table 3. All of the subscales were inter-correlated in the schizophrenia group, with particularly high correlations between Theory of Mind Index and Attributions Index. A similar pattern was found in the control group; however, no subscales were inter-correlated with the Sociality score.

Table 3.

Inter-correlations between the social cognition indices and sociality score.

	NET Scores
	EP Index	ToM Index	Attr. Index	Soc. Score
Emotion Perception Index	-	.46**	.58**	.19
Theory of Mind Index	.42**	-	.67**	.18
Attributions Index	.51**	.57**	-	.21
Sociality Score	.22	.60**	.50**	-

^P<.10

*P< 0.05

^**P < 0.01

Participants from the schizophrenia group are highlighted and appear below the diagonal. Controls appear above the diagonal.

3.4. Convergent and divergent validity

As summarized in Table 4, the Theory of Mind Index was significantly associated with the theory of mind composite measure (Hinting + TASIT), and the Emotion Perception Index was significantly associated with the emotion perception composite measure (FEIT + FEDT). The attribution index was not associated with the AIHQ blame score, aggression, hostility bias, or total scores. As the NET is hypothesized to be an integrated measure of social cognition, we also examined the relationships between NET total scores and each domain. There were significant relationships between NET total scores, and the composite measures of emotion perception and theory of mind.

Table 4.

Convergent validity analyses showing correlations between the social cognition indices and measures of social cognition in the schizophrenia sample.

	NET Scores
	EP Index	ToM Index	Attr. Index	NET Total
Emotion Perception Composite	.46**	.34*	.38*	.49**
Theory of Mind Composite	.48**	.43**	.33*	.55**
AIHQ – Hostility bias	.13	−.13	.13	.01
AIHQ – Aggression bias	−.05	.36*	.24	.26
AIHQ – Blame score	−.01	−.09	.08	−.02
AIHQ – Total	.01	−.06	.12	.02

^P<.10

^*P< 0.05

^**P < 0.01

Highlighted correlations are those hypothesized to be statistically significant.

To examine divergent validity, NET total scores were correlated with performance on both the verbal (r = .41, p = .01) and performance (r = .45, p < .01) subscales of the WASI. Of note, given the specific verbal nature of the NET, we planned to further examine the influence of Verbal IQ by repeating convergent and ecological validity analyses controlling for Verbal WASI.

3.5. Ecological validity

As summarized in Table 5, Total NET scores were significantly correlated with the paralinguistic, participation, appropriateness and affect scores, as well as overall performance on the SSPA (role-play task). When controlling for Verbal IQ, the relationships with the total, appropriateness and affect subscales remained statistically significant, while the correlations with the paralinguistics scale (r = .31, p = .06) and participation scale (r = .29, p = .09) approached statistical significance.

Table 5.

Ecological validity analyses showing correlations between the social cognition indices and social functioning in the schizophrenia sample.

	NET Scores
	Sociality Score	NET Total
Role Functioning Scales – Total	.04	.23
Immediate Social Network	−.12	−.08
Extended Social Network	−.09	−.01
Work Functioning	.09	.19
Independent Living	.15	.44**
Global Social Functioning	.07	.05
Social Skills Performance Assessment – Total	.27	.58***
Paralinguistics Total	.13	.38*
Participation Total	.18	.38*
Appropriateness Total	.32*	.64**
Affect Total	.29	.54**

^P<.10

^*P< 0.05

^**P < 0.01

P<.001

Highlighted correlations are those hypothesized to be significant.

Total NET scores were also significantly correlated with independent living skills as measured by the RFS, but not by any of the other subscales. When controlling for Verbal IQ, the relationship with independent living skills remained statistically significant. NET Sociality scores were correlated with the appropriateness subscale of the SSPA, and this relationship remained statistically significant after controlling for Verbal WASI.

3.6. Relationships to symptoms

We examined correlations between the NET total score and symptoms as measured on the PANSS, finding significant relationships between NET total and positive symptoms, r = -.43, p < .01, negative symptoms, r = -.41, p < .01, and PANSS total, p = -.39, p = .01, but not between NET total and PANSS general psychopathology symptoms, r = -.16, p = .30.

4. Discussion

The present study evaluated the psychometric properties of the Narrative of Emotions Task as an integrated and personalized measure of social cognition. Overall, the NET has good inter-rater reliability, is successful in differentiating between those with schizophrenia and non-clinical controls, and has evidence of convergent, divergent, and ecological validity. While the NET is not without limitations, this study supports its continued use. Further questions raised by the present study include the latent factor structure of the NET, as well as its use as an indicator of social cognition, social functioning, or a combination of the two. Nonetheless, the present study provides initial evidence for the feasibility and psychometric properties for measuring social cognition in an integrated and personalized manner via eliciting emotional narrative samples.

The NET demonstrated adequate reliability, with a few areas for development. First, the inter-rater reliability of the NET was acceptable to very good for all of the social cognition indices and total score, with the exception of the Attributions Index. And second, items showed strong internal consistency for some indices (e.g., theory of mind index and NET total score), and only moderate levels of it for others (e.g., emotion perception index, attributions index, and sociality score). It is important to note, however, that the scales with moderate levels of internal consistency have fewer items (attribution index and sociality score are one item per emotion narrative), and may be conceptually heterogeneous. For instance, the emotion perception index draws on aspects of both one's definition of the emotion, and their narrative, which may reduce reliability.

Convergent validity analyses indicated that the NET taps into aspects of emotion perception and theory of mind. These relationships were unchanged after controlling for verbal IQ. However, there is no evidence that it is associated with attributional bias. The lack of a relationship with attributional bias in the NET is consistent with early research on the factor structure of social cognition in individuals with schizophrenia, according to which measures of attributional style load on their own separate factor, distinct from (and only weakly correlated with) measures of emotion perception and theory of mind (Mancuso et al., 2011). This could be a result of a separable attributional style factor, or could be a product of the methods of measuring attributional style (Combs et al., 2007), in that it is identified as a cognitive style, rather than a skill-based trait domain like theory of mind and emotion perception.

There were several significant intercorrelations between the subscales of the NET. This underscores the point that while narrative performance is related to social cognition more generally, it remains unclear how the NET's specific indices are isolating sub-domains of each component of social cognition. The issue of conceptual and empirical overlap within social cognition measures is not unique to the NET. First, many social cognitive measures used in schizophrenia research have not been thoroughly reviewed for their factor structure or relationship to functional outcome. Second, without more factor analytic work, it is unclear at this point to what extent these two domains represent wholly distinct abilities. These concerns – along with other more general issues with reliability and validity – have prompted the Social Cognition Psychometric Evaluation (SCOPE) project (Pinkham et al., 2013), which has tasked an expert panel with identifying social cognition domains, selecting tests to measure those domains, and a large-scale evaluation of the psychometric properties of those tests.

The ecological validity analyses of the NET showed mixed results. The overall pattern of findings reflected a significant relationship between the NET and a social skills role-play, but only one significant association with more global measures of social and community functioning. While the NET requires individuals to engage in processes specific to social cognition, it also an interview-based measure which could draw on an individual's past social interaction and conversation skills, resulting in a close link with role-play performance. It is important to note the complex nature of measurement of functional outcome in schizophrenia while interpreting this pattern of results. First, objective indicators of real-world functioning such as job performance and social relationships are products not only of skills competence, but also negative symptoms and other intervening factors like motivation, social environment and depressive symptoms (Harvey, 2010). Second, many have noted the small overlap between self-report of cognitive impairment and actual performance on a battery of cognitive assessment (as low as Pearson's r of .04; Harvey, 2010; Keefe, Poe, Walker, Kang & Harvey, 2006). Considering the complexity of functioning assessment in schizophrenia, it could be the case that the NET is an effective cognitive measure in predicting skill competence (as evidenced by the SSPA), but does not account for intervening factors between skill competence and measures of objective functioning.

The findings point to a number of limitations of the present study that need to be considered in future research. First, the size of the sample prevented the use of factor analysis to identify an empirically defined factor structure of the NET. Second, there was no evidence that the NET measures attributional style. With the reliability challenges and non-normality of the attribution index, the present study does not support its further use as a measure of attributional style. Importantly, as attributional skill is conceptualized slightly differently in the current study than in those that examine attributional biases, it could be the case that the Attribution Index measures a social cognitive ability, but not the attributional bias that has been previously widely studied (Aakre et al., 2009). Third, there appeared to be little incremental validity provided by the NET sociality score, which lacked significant relationships with measures of outcome and was not normally distributed. Lastly, there remain additional questions that should be asked about the NET and other narrative measures of social cognition in further research, including whether or not the NET exhibits short-term test-retest reliability, responsiveness to treatment, and prediction of treatment and functional outcomes prospectively.

Ultimately, the NET provides a useful first step moving toward the use of narrative sampling as an additional means of assessing social cognition in people with schizophrenia. The NET shows several potential benefits and areas of promise above and beyond extant measures in the field of social cognition. First, it assesses the domains of emotion perception and theory of mind in co-operation. This could be both a practical and ecological benefit. Second, it requires engagement in an active social activity of making sense of and retelling one's own emotional experiences. Finally, it provides personalization that has not yet been provided by current measures. While much work can be done to further examine the NET, it appears to provide a promising new methodology of measuring social cognition in individuals with schizophrenia.