Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Feb 9.
Published in final edited form as: Schizophr Res. 2021 Aug 20;236:80–86. doi: 10.1016/j.schres.2021.08.012

Performance-Based Assessment of Social Skills in a large Sample of Participants with Schizophrenia, Bipolar Disorder and Healthy Controls: Correlates of Social Competence and Social Appropriateness

Michelle L Miller 1, Martin T Strassnig 1, Evelin Bromet 2, Colin A Depp 3,4, Katherine Jonas 2, Wenxuan Lin 2, Raeanne A Moore 3, Thomas L Patterson 3, David L Penn 5,6, Amy E Pinkham 7,8, Roman A Kotov 2, Philip D Harvey 1,9
PMCID: PMC10857848  NIHMSID: NIHMS1951410  PMID: 34425381

Abstract

Introduction.

Performance-based assessments of social skills have detected impairments in people with severe mental illness and are correlated with functional outcomes in people with schizophrenia and bipolar disorder. The most common of these assessments, the Social Skills Performance Assessment (SSPA), has two communication scenarios and items measuring both social competence and appropriateness. As real-world competence and appropriateness appear to have different correlates, we hypothesized that SSPA Items measuring competence and appropriateness would be distinct and have different correlations with other outcomes.

Methods.

We aggregated data from 557 people with schizophrenia, 106 with bipolar disorder, and 378 well controls from 4 separate research studies. All participants were assessed with both SSPA scenarios and other performance based and clinician-rated measures. A single expert rated the SSPA interactions for competence and appropriateness while blind to participant diagnoses.

Results.

Participants with bipolar disorder and schizophrenia performed more poorly on every item of the SSPA than healthy controls. Items measuring social competence and appropriateness in communication were intercorrelated across scenarios, as were elements of socially competent communication, although the items measuring competence did not correlate substantially with appropriateness. Items assessing social competence, but not social appropriateness, correlated with better cognitive and functional performance and residential and financial independence.

Discussion.

Social competence and social appropriateness were distinct elements of performance-based social skills with potential differences in their functional correlates. As both social competence and appropriateness impact functional outcomes, improvement in the measurement and treatment of appropriate communication seems to be an important goal.

1.0. Introduction

Impaired social functioning is a common feature of severe mental illness that poses significant real-world consequences, such as unemployment (Depp et al., 2010; Morgan & Gopalaswamy), difficulty building and maintaining stable relationships (Fett et al., 2011), reduced community functioning (Couture et al., 2006), aggression and violence (Jones & Harvey, 2020), and poor quality of life (Penn et al., 1997). Individuals with schizophrenia and bipolar disorder demonstrate social deficits (Depp et al., 2010; Patterson et al., 2001; Sitzer et al., 2008), although those with schizophrenia typically demonstrate more impaired social functioning (van Liempt et al., 2017).

Social dysfunction is also a potential indicator of vulnerability to developing severe mental illness, as evidenced by a positive association between worse premorbid function and an increased risk of developing psychosis in individuals with schizophrenia and bipolar disorder (Cannon et al., 1997). Additionally, social impairment persists during periods of affective and psychotic symptom remission, framing social impairment as a stable feature of severe mental illness (Bae et al., 2006; Bauwens et al., 1991). Thus, like neurocognition and performance of nonsocial everyday skills, social functioning deficits are not limited to period of symptomatic exacerbation in either schizophrenia or bipolar disorder.

Social functioning is a product of multiple factors, including skills such as social cognition (Bae et al., 2010; Kalin et al., 2014) and social competence (Orpinas, 2010), the presence of certain negative symptoms (Robertson et al., 2014), psychotic symptoms (Harvey et al., 2019) and social appropriateness (Wykes & Stuart, 1986). Social competence is “knowledge and skills for functioning n his or her community or social environment” (Orpinas & Horne, 2006, p. 108) and is necessary for individuals to perform daily life tasks, acquire and maintain employment, build effective interpersonal relationships (Bustillo et al., 2001), and achieve overall success (Owens et al., 2010). Social appropriateness or acting in accordance with social norms (Wykes & Stuart, 1996) requires knowledge of social norms in combination with sufficient social cognitive skills to predict the impact of one’s behavior on others and the self-control to conform behavior accordingly (Frith & Corcoran, 1996). Failure to conform to social norms can manifest as disruptive, aggressive, or intrusive behavior (Wykes & Stuart, 1996) and is likely to be exacerbated during periods of psychotic or mood symptoms. Individuals with schizophrenia have been reported to have greater challenges in understanding the viewpoint of those affected by a social norm violation than in their specific knowledge of social norms (Mazza et al., 2007).

Social competence and social appropriateness have previously been combined in the performance-based measurement of social skills. For example, the Social Skills Performance Assessment (SSPA) is a commonly used measure of social behavior that includes items measuring both social competence and social appropriateness (Patterson et al., 2001). Adults with schizophrenia and bipolar disorder consistently demonstrate impairment on the SSPA (Depp et al., 2010; Patterson et al., 2001; Sitzer et al., 2008), with participants with schizophrenia typically performing worse (Mahmood et al., 2018). The contributions of social competence versus social appropriateness to performance and diagnostic differences in SSPA performance are unclear. Patterson and colleagues (2001) reported that individuals with schizophrenia and schizoaffective disorder demonstrated impairments compared to well controls in all items in the SSPA, other than social appropriateness. On the other hand, Sitzer and colleagues (2008) report that middle-aged and older individuals with schizophrenia performed significantly worse than controls on every item of the SSPA, including social appropriateness. Adolescents with bipolar disorder rated their social skills performance as significantly worse than healthy controls on all scales of the Maston Evaluation of Social Skills with Youngsters assessment except for social appropriateness (Goldstein et al., 2006), although their parents rated them as impaired on all subscales, including social appropriateness. Thus, the diagnostic and functional significance of appropriateness measured with performance-based strategies is an unsettled question.

Performance on the SSPA, previously examined as a single global score, has been reported to have several correlates. SSPA performance has been reported to be associated with working memory, executive functioning, and letter fluency, as well as everyday functional skills measured by the UPSA (Bowie et al., 2008; Mahmood et al., 2018; McClure et al., 2007). Total scores on the SSPA also correlate with real-world milestones reflecting social function, such as economic and residential independence in individuals with schizophrenia (Bowie et al., 2008; 2010; Silberstein et al., 2018) and bipolar disorder (Bowie et al., 2010; Depp et al., 2010). However, previous research has suggested that the two scenarios of the SSPA, an instrumental task (dealing with a landlord) and a social task (introducing oneself to a new neighbor) may themselves have different cognitive correlates (Moore et al., 2016). Thus, an item-level examination of the SSPA across the two different scenarios offer promise for potentially delineating diagnostic differences and correlates of social competence and appropriateness across cognitive, functional capacity, and everyday outcomes domains.

In this large sample (N>1,000) of participants with schizophrenia, bipolar illness, and healthy controls, we performed item-level analyses of social competence and social appropriateness. We hypothesized that there would be differences in both social competence and social appropriateness across diagnostic groups, with the greatest impairments in participants with schizophrenia. We further hypothesized that items measuring social competence would be correlated with performance-based measures of cognition and functional capacity and functional outcomes, such as being financially independent and living independently. We finally hypothesized that social competence and social appropriateness would be distinct, as other studies have suggested that hostility and aggression co-aggregate with appropriate behavior and paranoid symptoms (Jones & Harvey, 2020; Pinkham et al., 2011), possibly due to alterations in threat perception (Green and Phillips, 2004). In addition to the large sample size, another unique feature of this study is the aggregated database of SSPA scores’ basis on the ratings of a single, blinded rater, who rated all the participants in this study remotely while unaware of diagnosis, clinical state, or any other information about the participants.

2.0. Methods

2.1. Participants

For these analyses, data from four studies conducted in multiple geographic areas were aggregated. All studies contributed data from participants with schizophrenia; two contributed information from participants with bipolar disorder; and two studies included healthy controls. Diagnoses for all studies were based on structured diagnostic interviews and consensus procedures, and healthy controls were confirmed to not meet criteria for an axis- I psychotic disorder or major depression.

2.11. Suffolk County Mental Health Project.

Participants were recruited from the 12 inpatient facilities in Suffolk County, NY (pop 1.3 million). All participants provided written informed consent at the initial (1990–1995) and subsequent assessments. Approval was obtained annually from the Stony Brook University IRB. Inclusion criteria at study entry consisted of individuals aged 15–60 years, residing in Suffolk County NY, and psychosis not due to a medical condition; exclusion criteria consisted of a psychiatric hospitalization more than six months before the index admission, more than borderline intellectual disability (IQ < 70), incapacity to provide informed consent, and being a non-English speaker. The Structured Clinical Interview for DSM-IV (SCID; Spitzer et al., 1990) was administered at all in-person assessments (Bromet et al., 2011). Based on the SCIDs, medical record information, and interviews with significant others, longitudinal DSM-IV consensus diagnoses were reached by study psychiatrists for each participant (Bromet et al., 2011). This report used the final study diagnosis to categorize participants with schizophrenia/schizoaffective disorder and bipolar disorder.

Never-psychotic participants were recruited in 2012–2014 using random digit dialing and matched to cases on zip code, age, and gender. Exclusion criteria were lifetime history of psychosis or psychiatric hospitalization (Velthorst et al. 2017). Cognitive performance, psychiatric symptoms, and functioning were collected corresponding to the 20-year follow-up of the clinical participants and 5-year follow-up of well controls.

2.12. Social Cognition Psychometric Evaluation (SCOPE).

The data come from Phases 3 and 5 of the SCOPE study (Pinkham et al., 2018), a multisite study performed at the University of Miami Miller School of Medicine (UM phases 3 and 5), Southern Methodist University (SMU: phase 3), the University of Texas at Dallas (Phase 5), and the University of North Carolina at Chapel Hill (Phase 5). At UM, patients were recruited from both the Miami VA medical Center and Jackson Memorial Hospital-University of Miami Medical Center. At SMU and later at UT Dallas, patients were recruited from Metrocare Services, a non-profit mental health services provider organization located in Dallas County, Texas. UNC patients were recruited from the Schizophrenia Treatment and Evaluation Program (STEP) in Carrboro, NC, and the Clinical Research Unit (CRU) in Raleigh, NC. Participants were stable outpatients diagnosed with schizophrenia or schizoaffective disorder (n’s=179 and 218) and healthy controls (n’s=104 and 154). At all sites, healthy controls were recruited via community advertisements. All participants provided written informed consent, and IRBs at each site approved this study.

Patients were required to have a DSM-IV diagnosis of schizophrenia or schizoaffective disorder, with this diagnosis confirmed by clinical interview utilizing the SCID Psychosis Module (First et al., 2015) and the MINI International Neuropsychiatric Inventory (MINI; Sheehan et al., 1998) In addition to a DSM-IV diagnosis of schizophrenia or schizoaffective disorder, patients had to be on a regular medication schedule for at least six weeks with no dose changes for at least two weeks. Patients also could not have been hospitalized in the past two months. A local consensus procedure was used to generate final diagnoses.

To ensure that the healthy controls did not have a history of significant psychopathology, they were also interviewed for the presence of major Axis I or II disorders. Exclusion criteria for both groups included: 1) presence or history of pervasive developmental disorder or mental retardation (defined as IQ<70) by DSM-IV criteria, 2) presence or history of medical or neurological disorders that may affect brain function (e.g., seizures, CNS tumors, or loss of consciousness for 15 minutes or more), 3) the presence of sensory limitation including visual (e.g., blindness, glaucoma, vision uncorrectable to 20/40) or hearing impairments that interfere with assessment, 4) no proficiency in English, 5) presence of substance abuse in the past month, and 6) the presence of substance dependence not in remission for the past six months.

2.13. Introspective Accuracy in Schizophrenia and Bipolar disorder.

This study was conducted at UT Dallas, UM, and the University of California San Diego. Diagnostic methods and procedures were the same as for the SCOPE study, except for using DSM-5 criteria for the mental health diagnoses.

2.2. Assessments

2.21. Social Skills Performance Assessment (SSPA).

The SSPA is a widely used performance-based assessment. It includes two scenes, one that involves the participants introducing themselves to a new neighbor and one that involves the participants requesting their landlord to fix a leak after the landlord had previously agreed to fix the leak but had never completed the task. There are six items common to both scales: interest, fluency, clarity, focus, overall conversation, and social appropriateness. As the second interaction is instrumental, there are two additional items: negotiation ability and persistence. All items are rated on a five-point (1–5) scale, with one as the lowest and five as the highest score. Audio-recordings of SSPA interviews conducted at all sites were sent to a central rater who rated all the audio-recordings without having any other information about the participants. There is an additional item, appearance, which could not be rated with this strategy.

2.22. Cognition and Functional Capacity.

Although participants in these studies were rated with various measures, there was an overlapping subset of cognitive tests that are presented in the current paper. These include the Maryland Letter-Number Sequencing Test, Trail-making part A, Animal fluency, and the USCD Performance-Based skills assessment (UPSA-B). As reported previously, processing speed and working memory measures commonly account for most of the variance in composite scores of neuropsychological performance in participants with schizophrenia and bipolar disorder (Keefe et al., 2006; Harvey et al., 2016).

2.23. Everyday Functioning Ratings.

All participants were rated on two everyday functional outcomes, including whether they lived independently versus a supported residence and whether they were personally responsible for their living expenses through some productive activity or were completely supported by disability compensation, relatives, or other public assistance. These ratings were based on a comprehensive interview at the time of the assessment, supplemented by reports of informants for all the SCZ and BPI participants. In order increase the validity of the information, participants were asked to provide specifics of their residential and employment status, including current and prior employers as well as the residential arrangements (who owns the residence? how much do you pay to live there?).

2.3. Data Analyses

All data analyses were conducted via SPSS (IBM Corp. Released 2019. IBM SPSS Statistics for Windows, Version 26.0. Armonk, NY: IBM Corp). To determine whether SSPA performance in the samples derived from the different studies should be examined separately or combined, we compared the four performance-based variables (3 cognitive and the UPSA-B) and the 14 SSPA variables across the different sources of the participants in the three diagnostic groups using t-tests or one-way Analyses of variance (ANOVAs). Subsequent analyses addressed the relative magnitude of diagnostic differences in items indexing social competence and social appropriateness and the intercorrelations of these SSPA items, as well as their correlations with the 4 performance-based assessments. One-way analysis of variance compared the SSPA variables and Pearson correlations were used to examine the associations of the SSPA variables with the other outcome variables. We compared the prevalence of the two everyday functioning outcomes (financial responsibility and residential independence) between groups with Chi-square tests. For analyses regarding the everyday functioning correlates of social competence and social appropriateness, we combined with items measuring social competence into a single scale by summing the item scores and used t-tests to compare the two functioning groups across the social competence and social appropriateness scores. In a final analysis, we related age and sex to the 14 SSPA items using t-tests to examine sex effects and Pearson correlations for age effects.

3.0. Results

Table 1 presents descriptive information for the research participants. As shown in the table, the Suffolk County participants were older (because they were seen at a 25-year follow-up) and included fewer minorities.

Table 1.

Descriptive Information on the participants

Suffolk County Scope 3 and Scope 5 Introspective Accuracy X2 df p
N % N % N % 20.95 2 <.001
Female 95 45 250 38 98 57
African American 16 8 298 45 72 43 134.7 4 <.001
Caucasian 177 84 315 48 70 42
Other/multiple 16 8 4 1 3 15
Unknown/declined to answer  0 26 27
Ethnic Status
 Latino 20 10 118 1 45 26 36.76 3 <.001
Schizophrenia 57 397 87
Bipolar 32 58
HC 120 258
M SD M SD M SD F df p
Age 54.82 8.79 41.45 12.38 40.83 11.05 114.75 2,1037 <.001
Open in a new tab

3.1. Group Differences in Performance-Based Measures

In the comparison of the healthy controls across the SCOPE and SCMHP samples, only one of the 14 SSPA variables differed significantly between the groups, scene 1 social appropriateness, t (377) =2.08, p=.038. There was also a significant difference in animal naming performance across the two samples, with SCOPE HC performing better, t (377) =2.96, p=.003. For the bipolar participants, in SCMHP and IA studies, not one of the 18 t-tests was significant, all t<1.19, all p>.24. For the schizophrenia patients in SCOPE, IA, and SCMHP, there were two SSPA variables that differed significantly between the groups, persistence and overall argument, F (2,519) =5.22 and 5.25, p=.006. The other 12 variables did not differ. Trail making test part A performance was also different between the groups, F (2,519) =10.76, p<.001. The combined SSPA competence items for scene 1 did not differ across the sites for the participants with schizophrenia, F (2,519) =0.96, p=.39 and the scene 2 total score for the competence items was significant, but only at a nominal level: F (2,519) =2.03, p=.046. Given that 3/42 differences in SSPA performance were significant, we decided to analyze the diagnostic groups on a combined basis across sites.

Figure 1 presents group differences in SSPA ratings, as standard scores with the heathy controls set to 0. For all ratings, participants with schizophrenia scored the lowest. Supplementary table 1 presents means and standard deviations and more detailed descriptions of statistical tests. Group differences were significant, based on the one-way ANOVA, for all items other than social appropriateness for scene 2. Tukey HSD follow-up tests indicated that schizophrenia participants performed more poorly than healthy controls on all items in scene 1 and all items in scene 2 other than social appropriateness. Bipolar patients performed equivalently to healthy controls and better than schizophrenia participants on all scene 1 items other than focus, where their performance did not differ from that of the participants with schizophrenia, and social appropriateness, where they scored worse than the healthy controls and equivalently to participants with schizophrenia. For scene 2, the results were similar in that the bipolar participants scored better than the schizophrenia participants on all items other than focus.

Fig. 1.

Fig. 1.

Open in a new tab

Performance on SSPA items for both scenes separated by diagnosis.

T-tests were used to compare males and females on the different SSPA items, using the Bonferroni correction and setting the p value at .05/14=.003. Even with the conservative criterion we found that female participants performed received higher scores on interest, fluency, and clarity for scenes 1 and 2, as well as persistence and overall argument for scene 2. In contrast to the sex effects, there were no significant correlations between age and any of the 14 SSPA variables, all r<.06, all p>.08.

Performance and group differences in cognitive and functional capacity measures are presented in Table 2. The participants with schizophrenia scored worse than the healthy controls on all variables. Participants with bipolar disorder performed worse than healthy controls on letter-number span and UPSA-B but showed no differences on the trail making test part A and animal naming. Participants with schizophrenia were least likely to live independently or to be financially responsible, with bipolar participants being more likely to live independently and to be financially responsible, but still less likely than healthy controls to meet both milestones.

Table 2.

Performance on SSPA Measures by Diagnosis

SCZ (N=521) BPI (N=90) HC (N=378) F p d
M SD M SD M SD
Scene 1
Interest 3.83 1.00 4.43 .73 4.49 .67 68.81 .000 .985
Fluency 4.20 .73 4.65 .55 4.67 .53 63.84 .000 .887
Clarity 4.30 .78 4.71 .56 4.83 .43 78.19 .000 1.233
Focus 4.24 .93 4.36 .68 4.58 .62 19.24 .000 .548
Overall conversation 3.82 .94 4.38 .77 4.44 .66 64.74 .000 .939
Social appropriateness 4.80 .55 4.84 .54 4.93 .32 7.50 .001 .30
Scene 2
Interest 4.05 .61 4.43 .55 4.33 .51 34.41 .000 .745
Fluency 4.46 .69 4.88 .40 4.87 .37 63.86 .000 1.135
Clarity 4.61 .64 4.91 .28 4.92 .32 43.12 .000 .969
Focus 4.73 .67 4.76 .56 4.92 .32 13.25 .000 .594
Negotiation 3.08 1.28 3.74 1.21 3.81 1.15 40.66 .000 .635
Persistence 3.22 1.12 3.78 1.19 3.94 1.07 47.62 .000 .673
Overall conversation 3.04 1.16 3.60 1.25 3.75 1.09 44.95 .000 .651
Social appropriateness 4.83 .54 4.74 .73 4.87 .38 2.37 .094 .342
Open in a new tab

3.2. Correlations

We calculated intercorrelations of the competence measures for the two SSPA scenes across the entire sample. We created an average for the competence items within each scene because the intercorrelations for the competence items in scene 1 ranged from r=.54 to r=.83, all p<.001 and the corresponding intercorrelations for scene 2 ranged from r=.27 to r=.57, all p<.001. We correlated the two appropriateness items, the two competence subscales and the other performance-based items and presented the results in Table 3. Correlations between the two social competence average scores and all the performance-based measures were significant at p<.001. In contrast, the correlations between social appropriateness scores and all performance-based measures were non-significant. The two social competence average scores were significantly correlated across the two scenes, as were the two items for social appropriateness.

Table 3.

Functional and Other Performance-Based Outcome by Diagnosis

SCZ (N=521) BPI (N=90) HC (N=378) F p d
M SD M SD M SD
Trail Making Part A 39.85 17.59 29.53 11.32 31.06 12.04 45.29 .000 .857
Letter-Number Sequencing 11.77 4.17 13.03 3.28 14.09 3.51 39.74 .000 .661
Animal Naming 18.65 6.96 23.08 9.68 23.71 9.50 44.64 .000 .532
UCSD Performance-Based Skills Assessment-Brief1 69.73 14.84 78.93 12.07 85.93 9.64 73.47 .000 1.680
N % N % N % X1(2) p
Living Independently 456 82.0 94 89.5 363 97.3 50.78 .000
Financially Responsible 320 57.7 68 64.8 340 90.9 120.35 .000
Open in a new tab
1.

Note HC in SCOPE did not do the UPSA-B, so the n of HC=114

Our final analysis compared the residential and financial milestones across the social competence and social appropriateness variables (table 4). There were statistically significant differences for social competence in both scenes 1 and 2 for both residential and financial independence. In contrast, socially appropriate behavior did not differ as a function of milestone achievements.

Table 4.

Intercorrelations of Variables

LNS AN UPSA SC1 SC2 IN1 IN2
Trail Making Part A (TMT) −.38 −.34 −.33 −.28 −.23 .02 .09
Letter-Number Sequencing (LNS) - - .41 .47 .36 .31 .07 .01
Animal Naming (AN) - - - - .31 .32 .28 .02 .04
UCSD Performance-Based Skills Assessment-Brief (UPSA) - - - - - - .42 .34 .05 .00
Social Competence Scene 1 (SC1) - - - - - - - - .56 .31 .09
Social Competence Scene 2 (SC2) - - - - - - - - - - .19 .10
Social Inappropriate Scene 1 (IN1) - - - - - - - - - - - - .40
Open in a new tab

4.0. Discussion

We hypothesized that performance-based items measuring social competence would differ between participants with schizophrenia, bipolar disorder and healthy0controls and these predictions were confirmed. Participants with schizophrenia had the lowest scores across all these items and bipolar participants manifested several impairments compared to healthy controls. Differences in social appropriateness were considerably less evident, but there are potential biases that are involved in these analyses. Performance on items measuring social competence was correlated consistently with performance on cognitive and functional capacity measures, as well as predicting real-world functioning, while items measuring appropriateness were not. This is consistent with previous research highlighting correlations between SSPA total scores and neurocognitive performance (Bowie et al., 2008; Depp et al., 2010, Mahmood et al., 2018; McClure et al., 2007) and functional status (Patterson et al., 2001) in individuals with serious mental illness. The findings that items measuring competence and appropriateness were generally not correlated with each other, while being intercorrelated within domains, also suggests that these two dimensions of social abilities may be distinct, as we had hypothesized. There were sex differences in many of the social competence items, with this finding replicating research on social skills in the general population (e.g., Joseph, 2000) and psychiatric populations (Tamminga, 1997).

Consistent with previous studies, we found diagnostic differences on the other performance-based assessments and real-world outcomes. Impairments in social appropriateness are a common feature of schizophrenia, particularly during a period of psychotic exacerbation, and may be associated with other real-world outcomes, such as legal repercussions of violent or indecent behavior. The differential correlations between real-world outcomes and performance-based outcomes and social competence vs. appropriateness are consistent with previous research that suggests disruptive and unskilled social behaviors may have different determinants. As socially appropriate behavior is an important target for clinical attention, it seems important to develop performance-indices of social appropriateness that have reduced ceiling effects. The original role play assessment from which the SSPA was adapted, the Maryland Assessment of Social Competence (MASC; Bellack et al., 2006), had a scale for “social norm violation” which was replaced in the SSPA with the single item social appropriateness assessment. For the worst performing group, participants with schizophrenia, the mean scores on the appropriateness items across the scenes differed from the HC by .04 and .13 points on a five-point scale. This is a likely an underestimate of true differences in social appropriateness between these samples.

The current study has several limitations. As this is a pooled database, there were clear demographic differences among the samples that likely originate from differences in source populations and from the cross-sectional versus longitudinal differences in study design. Only overlapping performance-based assessments could be employed and other measures might have better correlations with appropriateness, including measures of social cognition which were not administered in all samples. As the appropriateness index is only a single item, this alone could also be adversely impacting the psychometric characteristics in addition to potential ceiling effects. The lack of correlation of social appropriateness with other variables is likely impacted by ceiling effects, although the two social appropriateness items across scenes correlated at a level that shared 16% variance. Cost of living differences can impact independence in residential status, but Long Island, Miami-Dade County, Dallas, and San Diego do not have marked variance in the cost of living compared to more rural areas. All ratings were done from audio recordings. The single rater method can be viewed as both a strength and a weakness, in that inter-rater reliability is not possible to examine. As video technology has improved markedly in ease of use over the time period that these studies were conducted, later studies should consider this approach, even though ethical considerations may be more substantial in terms of transmission of video information. Finally, generally clinically stable populations would be expected to have increased appropriate behavior and it is known that clinical symptoms impact social outcomes, perhaps through influences on socially appropriate behavior (e.g., Galderisi et al., 2014; Harvey et al., 2019). Thus, the ceiling effects in patient populations might be less in more symptomatic groups.

Conclusions.

Performance based assessments of social competence showed diagnostic differences and correlations with performance and real-world outcomes. Current assessments of social appropriateness in the SSPA appear to have ceiling effects and may require some modification to increase sensitivity. As performance-based measures of social competence were broadly correlated with functional outcomes, the importance of treatment of social competence deficits is underscored. Given the correlations with cognitive performance and functional capacity of the social competence measures, additional interventions aimed at cognitive enhancement, such as computerized cognitive training, may assist in generating more rapid gains in social competence and, eventually, improvement in social outcomes.

Supplementary Material

Supplemental Table 1

Table 5.

Functional Milestones and Social Competence and Inappropriateness

Residentially Independent Financially Responsible
Yes No Yes No
M SD M SD t p M SD M SD t p
Social Competence Scene 1 4.36 0.64 3.90 0.80 7.11 <.001 4.42 0.60 4.03 0.75 8.70 <.001
Social Competence Scene 2 4.15 0.62 3.69 0.78 7.00 <.001 4.20 −.61 3.85 0.69 7.48 <.001
Social Inappropriate Scene 1 4.85 0.48 4.86 0.48 0.20 0.84 4.86 0.46 4.83 0.52 0.89 0.37
Social Inappropriate Scene 1 4.84 0.49 4.83 0.61 0.14 0.87 4.85 0.45 4.80 0.62 1.55 0.12
Open in a new tab

Acknowledgments

This research was supported by NIMH Grants R01MH93432 to Drs. Harvey, Penn, and Pinkham, R01MH110434 to Dr. Kotov, and RO1MH112620 to Dr. Pinkham

References

  1. Bae SM, Lee SH, Park YM, Hyun MH, Yoon H. 2010. Predictive factors of social functioning in patients with schizophrenia: exploration for the best combination of variables using data mining. Psychiatry Investig. 7(2), 93–101. 10.4306/pi.2010.7.2.93. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bauwens F, Tracy A, Pardoen D, Elst MV, Mendlewicz J. 1991. Social Adjustment of Remitted Bipolar and Unipolar Out-patients: A Comparison with Age- and Sex-Matched Controls. Br J Psychiatry. 159(2), 239–244. 10.1192/bjp.159.2.239. [DOI] [PubMed] [Google Scholar]
  3. Bellack AS, Brown CH, Thomas-Lohrman S. 2006. Psychometric characteristics of role-play assessments of social skill in schizophrenia. Behav. Ther. 37(4), 339–52. 10.1016/j.beth.2006.01.005. [DOI] [PubMed] [Google Scholar]
  4. Bowie CR, Leung WW, Reichenberg A, McClure MM, Patterson TL, Heaton RK, & Harvey PD. 2008. Predicting schizophrenia patients’ real-world behavior with specific neuropsychological and functional capacity measures. Biol Psychiatry, 63(5), 505–511. 10.1016/j.biopsych.2007.05.022 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bowie CR, Depp C, McGrath JA, Wolyniec P, Mausbach BT, Thornquist MH, Luke J, Patterson TL, Harvey PD, Pulver AE. 2010. Prediction of real-world functional disability in chronic mental disorders: a comparison of schizophrenia and bipolar disorder. Am J Psychiatry. 167(9), 1116–1124. 10.1176/appi.ajp.2010.09101406. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bustillo JR, Lauriello J, Horan WP, Keith SJ. 2001. The psychosocial treatment of schizophrenia: An update. Am. J. Psychiatry. 158, 163–175. 10.1176/appi.ajp.158.2.163. [DOI] [PubMed] [Google Scholar]
  7. Couture SM, Penn DL, Roberts DL. 2006. The Functional Significance of Social Cognition in Schizophrenia: A Review. Schizophr Bull. 32(1), S44–S63. 10.1093/schbul/sbl029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Depp CA, Mausbach BT, Harvey PD, Bowie CR, Wolyniec PS, Thornquist MH, Luke JR, McGrath JA, Pulver AE, Patterson TL. 2010. Social competence and observer-rated social functioning in bipolar disorder. Bipolar Disord. 12(8), 843–850. 10.1111/j.1399-5618.2010.00880.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Fett AK, Viechtbauer W, Dominguez MD. 2011. The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: a meta-analysis. Neurosci Biobehav Rev. 35:573–588. 10.1016/j.neubiorev.2010.07.001. [DOI] [PubMed] [Google Scholar]
  10. First MB, Williams JBW, Karg RS, Spitzer RL. 2015. Structured Clinical Interview for DSM-5—Research Version (SCID-5 for DSM-5, Research Version; SCID-5-RV). Arlington, VA, American Psychiatric Association. [Google Scholar]
  11. Frith CD, Corcoran R. 1996. Exploring’ Theory of mind’ in people with schizophrenia. Psychol Med. 26(3):521–530. 10.1017/s0033291700035601. [DOI] [PubMed] [Google Scholar]
  12. Galderisi S, Rossi A, Rocca P, Bertolino A, Mucci A, et al. (2014). The influence of illness-related variables, personal resources and context-related factors on real-life functioning of people with schizophrenia World Psychiatry, 13, 275–87. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Goldstein TR, Miklowitz DJ, Mullen KL. 2006. Social skills knowledge and performance among adolescents with bipolar disorder. Bipol. Disord. 8, 350–361. 10.1111/j.1399-5618.2006.00321.x. [DOI] [PubMed] [Google Scholar]
  14. Green MJ, Phillips ML. 2004. Social threat perception and the evolution of paranoia. Neurosci Biobehav Rev. 28(3), 333–342. 10.1016/j.neubiorev.2004.03.006. [DOI] [PubMed] [Google Scholar]
  15. Harvey PD, Aslan M, Du M, et al. 2016. Factor structure of cognition and functional capacity in two studies of schizophrenia and bipolar disorder: Implications for genomic studies. Neuropsychology;30(1):28–39. doi: 10.1037/neu0000245 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Harvey PD, Strassnig MT, Silberstein J. 2019. Prediction of disability in schizophrenia: Symptoms, cognition, and self-assessment. J. Exper. Psychopathol. 1–20. [Google Scholar]
  17. Jones MT, Harvey PD. 2020. Neurocognition and social cognition training as treatments for violence and aggression in people with severe mental illness. CNS Spectrums. 25(2), 145–153. 10.1017/S1092852919001214. [DOI] [PubMed] [Google Scholar]
  18. Joseph R. The evolution of sex differences in language, sexuality, and visual-spatial skills. Arch Sex Behav. 2000;29(1):35–66. doi: 10.1023/a:1001834404611 [DOI] [PubMed] [Google Scholar]
  19. Kalin M, Kaplan S, Gould F, Pinkham AE, Penn DL, Harvey PD. 2015. Social cognition, social competence, negative symptoms and social outcomes: Inter-relationships in people with schizophrenia. J Psychiatr Res. 68, 254–260. 10.1016/j.jpsychires.2015.07.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Keefe RS, Bilder RM, Harvey PD, et al. 2006, Baseline neurocognitive deficits in the CATIE schizophrenia trial. Neuropsychopharmacology;31(9):2033–2046. doi: 10.1038/sj.npp.1301072 [DOI] [PubMed] [Google Scholar]
  21. Mahmood Z, Burton CZ, Vella L, Twamley EW. 2018. Neuropsychological predictors of performance-based measures of functional capacity and social skills in individuals with severe mental illness. J Psychiatr Res. 102, 201–206. 10.1016/j.jpsychires.2018.04.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Mazza M, Costagliola C, Di Michele V, Magliani V, Pollice R, Ricci A, Di Giovanbattista E, Roncone R, Casacchia M, Galzio RJ. 2007. Deficit of social cognition in subjects with surgically treated frontal lobe lesions and in subjects affected by schizophrenia. Eur Arch Psychiatry Clin Neurosci. 257, 12–22. 10.1007/s00406-006-0676-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. McClure MM, Bowie CR, Patterson TL, Heaton RK, Weaver C, Anderson H, Harvey PD. 2007. Correlations of functional capacity and neuropsychological performance in older patients with schizophrenia: evidence for specificity of relationships?. Schizophr Res. 89(1–3), 330–338. 10.1016/j.schres.2006.07.024. [DOI] [PubMed] [Google Scholar]
  24. Morgan R, Gopalaswamy AK. 1983. Employability of the mentally disabled in the 1970s. Br J Psychiatry. 142, 572–4. 10.1192/bjp.142.6.572. [DOI] [PubMed] [Google Scholar]
  25. Orpinas P. 2010. Social competence. In Weiner IB & Craighead WE (Eds.), The Corsini Encyclopedia of Psychology. Wiley, Hoboken, New Jersey, pp. 1623–1625. [Google Scholar]
  26. Orpinas P, Horne AM. 2006. Bullying prevention: Creating a positive school climate and developing social competence. American Psychological Association, Washington DC. [Google Scholar]
  27. Owens LA, Johnston-Rodriguez S. 2010. Social Competence, in: Peterson P, Baker E, McGaw B, (Eds.), International Encyclopedia of Education, third edition. Academic Press, Oxford, pp. 865–869. [Google Scholar]
  28. Patterson TL, Moscona S, McKibbin CL, Davidson K, Jeste DV. 2001. Social skills performance assessment among older patients with schizophrenia. Schizophr Res. 48(2–3), 351–360. 10.1016/s0920-9964(00)00109-2. [DOI] [PubMed] [Google Scholar]
  29. Penn DL, Corrigan PW, Bentall RP, Racenstein JM, Newman L. 1997. Social cognition in schizophrenia. Psychol Bull. 121(1), 114–132. 10.1037/0033-2909.121.1.114. [DOI] [PubMed] [Google Scholar]
  30. Pinkham AE, Brensinger C, Kohler C, Gur RE, Gur RC. 2011. Actively paranoid patients with schizophrenia over attribute anger to neutral faces. Schizophr Res. 125(2–3), 174–178. 10.1016/j.schres.2010.11.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Pinkham AE, Harvey PD, Penn DL. 2018. Social Cognition Psychometric Evaluation: Results of the Final Validation Study. Schizophr Bull. 44(4), 737–748. 10.1093/schbul/sbx117. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Robertson BR, Prestia D, Twamley EW, Patterson TL, Bowie CR, Harvey PD. 2014. Social competence versus negative symptoms as predictors of real-world social functioning in schizophrenia. Schizophr Res. 160(1–3), 136–141. 10.1016/j.schres.2014.10.037. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Sheehan DV, Lecrubier Y, Harnett-Sheehan K, et al. , 1998. The mini international neuropsychiatric interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview. J. Clin. Psychiatry; 59, 22–33 [PubMed] [Google Scholar]
  34. Silberstein JM, Pinkham AE, Penn DL, Harvey PD. 2018. Self-assessment of social cognitive ability in schizophrenia: Association with social cognitive test performance, informant assessments of social cognitive ability, and everyday outcomes. Schizophr Res. 199, 75–82. 10.1016/j.schres.2018.04.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Sitzer DI, Twamley EW, Patterson TL, Jeste DV. 2008. Multivariate predictors of social skills performance in middle-aged and older outpatients with schizophrenia spectrum disorders. Psychol Med. 38(5), 755–763. 10.1017/S0033291707001304. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Tamminga CA. 1997. Gender and schizophrenia. J Clin Psychiatry. 1997;58 Suppl 15:33–37. [PubMed] [Google Scholar]
  37. van Liempt S, Dols A, Schouws S, Stek ML, Meesters PD. 2017. Comparison of social functioning in community-living older individuals with schizophrenia and bipolar disorder: a catchment area-based study. Int J Geriatr Psychiatry. 32(5), 532–538. 10.1002/gps.4490. [DOI] [PubMed] [Google Scholar]
  38. Velthorst E, Fett AKJ, Reichenberg A, Perlman G, van Os J, Bromet EJ, & Kotov R. 2017. The 20-year longitudinal trajectories of social functioning in individuals with psychotic disorders. Am J Psychiatry. 174(11), 1075–1085. 10.1176/appi.ajp.2016.15111419. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Wykes T, Stuart E. 1986. The measurement of social behaviour in psychiatric patients: An assessment of the reliability and validity of the SBS schedule. Br J Psychiatry. 148, 1–11. 10.1192/bjp.148.1.1. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Table 1
NIHMS1951410-supplement-Supplemental_Table_1.docx (15.2KB, docx)

RESOURCES