Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Apr 1.
Published in final edited form as: Schizophr Res. 2021 Mar 2;230:17–23. doi: 10.1016/j.schres.2021.02.011

Self-reported Social Functioning and Social Cognition in Schizophrenia and Bipolar Disorder: Using Ecological Momentary Assessment to Identify the Origin of Bias

Dante Durand 1, Martin T Strassnig 1, Raeanne C Moore 2, Colin A Depp 2,3, Robert A Ackerman 4, Amy E Pinkham 4,5, Philip D Harvey 1,6
PMCID: PMC8222067  NIHMSID: NIHMS1677939  PMID: 33667854

Abstract

Objectives.

People with schizophrenia (SCZ) and bipolar illness (BPI) generate self-reports of their functioning that diverge from objective information. It has been suggested that these participants do not base such reports on daily experiences, relying on other information. We used ecological momentary assessment (EMA) to sample socially relevant daily activities in SCZ and BPI and related them to self-reported and observer-rated social functioning and social cognitive ability.

Methods.

71 people with (BPI) were compared to 102 people with SCZ. Participants were sampled 3 times per day for 30 days with a smartphone-based survey. Each survey asked where they were, with whom they were, what they were doing, and if they were sad. Participants and observers were asked to provide ratings on social functioning and social cognitive abilities at the end of the EMA period.

Results.

There was no association between being home or alone and self-reports of everyday social functioning. In contrast observer ratings were highly correlated with the momentary survey results. Reports of very low levels of sadness were associated with overestimated functioning and participants who were commonly home and alone rated their social functioning as better than participants who were commonly away in the presence of others.

Implications.

Both SCZ and BPI were marked by a disconnect between momentary experiences and self-reports. The largest effect was overestimation of functioning by participants who reported no sadness. Experience appears important, as participants who were routinely home and alone reported better social functioning than participants who spent more time others.

Keywords: Schizophrenia, Bipolar disorder, sadness, disability, activity, Ecological Momentary Assessment

1.0. Introduction

Despite multiple advances in the treatment of schizophrenia (SCZ), it remains the leading cause of disability worldwide (Charlson et al., 2018). There is consensus in the field that current functional assessment methods do not adequately capture the daily functioning behaviors of participants with schizophrenia (Ben-Zeev et al., 2012). Part of the limitation is that many functioning measures that rely on self-reports can be influenced by challenges in self-assessment, often referred to as introspective accuracy (IA; Harvey and Pinkham, 2015). These challenges extend to self-assessment of cognitive abilities, functional abilities, social cognition, and social outcomes (Durand et al., 2015) and are shared across psychiatric conditions (Strassnig et al., 2018).

These difficulties with introspective accuracy are commonly accompanied by introspective bias, wherein individuals either overestimate or underestimate their functioning (Silberstein and Harvey, 2019). It has been consistently reported in SCZ that overestimation biases are associated with poorer performance on social cognitive and neurocognitive tests (Jones et al., 2020; Gould et al., 2015; Perez et al., 2020); presence of certain types of paranoid ideation (Moritz et al., 2015; Savla et al., 2013), poorer social outcomes (Silberstein et al, 2018), and poorer functional outcomes (Gould et al., 2015; Durand et al., 2015). Both healthy controls (HC) and participants with SCZ who overestimated their social cognitive performance on a momentary basis had poorer performance on social cognitive measures and several neurocognitive tasks as well (Perez et al., 2020), congruent with earlier research on healthy people suggesting that poor performers often overestimate their abilities (Kruger and Dunning, 1999).

Depressed mood has a complex relationship with self-assessment, in both HC and in people with SCZ. For example, higher scores on self-reported depression scales were correlated with reports of worse social functioning in both HC and people with SCZ (Oliveri, et al., 2020). The self-reported absence of sadness in the context of objective adversity has been reported to be associated with overestimation of a range of abilities (Harvey et al., 2017; Harvey et a., 2019; Siu et al., 2015). Strassnig et al. (2018) found that in participants with either SCZ or bipolar illness (BPI), the severity of clinically rated depressed mood was associated with self-reported disability measured with a self-report everyday functioning scale, the World Health Organization Disability Assessment Scale (WHODAS; Ustun et al., 2010). In that same sample, across diagnoses, participants who were unemployed or living in supported residences or with their parents did not rate themselves as more disabled on the WHODAS than participants who had full-time jobs and were living independently. Thus, mood state, as defined as higher clinical ratings of depressed mood on the day of the self-reported disability assessment, was more strongly correlated with self-reported disability than with on-going life experiences with daily living implications (unemployment, supported residential status).

Social cognition is another domain where self-assessment is important (Penn et al., 2008). Participants with SCZ show deficits in performance on tests of social cognition and also in the self-assessment of their social cognitive abilities and their everyday social outcomes (Silberstein et al., 2018). Therefore, they may misinterpret interactions in either overly negative or overly positive directions (Kother et al., 2012; Moritz et al., 2012; Silberstein et al., 2018). Additionally, confidence when performing social cognitive tests has been reported to be unassociated with actual performance in participants with SCZ and HC (Cornacchio et al., 2018) and high confidence in social cognitive performance predicted poorer social outcomes in people with SCZ (Pinkham et al., 2018). In order to efficiently assess social cognition, the 8-item Observable Social Cognition Rating Scale (OSCARS; Healey et al., 2015) was developed for use as a self- or informant-report measure. Over-estimation of social cognitive abilities, indexed by self-reported OSCARS scores that were greater than those generated by informants, was found to be associated with poorer everyday social outcomes rated by observers (Silberstein et al., 2018).

An alternative strategy for assessment of everyday functioning is that of capturing participant’s daily activities and moods on a momentary basis. The use of ecological momentary assessment (EMA) has held promise in assessing functional outcomes in SCZ and BPI. Recent studies demonstrated the feasibility, sensitivity, reliability, and validity of EMA to assess functioning in SCZ (Granholm et al., 2020; Raugh et al., 2020; Strauss et al., 2020) and BPI (Merikangas et al., 2019; Raugh et al., 2020). EMA reports have provided an index of what individuals with SCZ are actually doing in real-world scenarios (Granholm et al., 2020; Strassnig et al., 2020), in comparison to HC. Validation of EMA responses with mobile phone global positional satellite (GPS) sensors, as well as augmentation with multi-channel passive measurements including actigraphy, accelerometry, and ambient speech has been reported recently. These studies find that people with schizophrenia left home less frequently, returned home after shorter periods away, traveled shorter distances overall as well as shorter distances away from home. These studies reported substantial convergence between EMA responses and GPS data, with improved prediction of negative symptom severity with combined approaches (Depp et al., 2019; Parrish et al., 2020; Raugh et al., 2020; Strauss et al., 2020).

One of the major challenges of understanding impairments in introspective accuracy and introspective bias is identification of the origin of these deficits. The hypotheses include reliance on momentary states for global judgments (Oliveri et al., 2020), impairments in cognitive monitoring processes (Gaweda et al.,2013), challenges in momentary encoding of episodic information regarding performance (Orfei et al., 2017), and the inability to utilize feedback on task performance to guide future behavior (Gould et al., 2015; Koren et al.,2005). Strassnig et al. (2018) suggested that objective information on everyday functional milestones was not being considered by participants asked to generate global estimates of their disability. Although assessments of mood symptoms with depression rating scales is common and the retrospective reporting period for these clinical assessments is designed to be longer than the current day, momentary assessments of mood are an efficient way to determine see if clinical ratings collected at an office are related to the daily experience of moods.

In this study, we used EMA strategies to examine the nature of self-reported social cognitive abilities and social functioning outcomes in a sample of participants with schizophrenia and bipolar disorder. We collected momentary objective data on socially relevant behaviors, including being alone or with other people, and in particular, being home and alone. Also, because momentary mood was concurrently sampled, we were able to examine the momentary experience of sad mood and concurrent social interactions over a 30-day period as well as the association between end of study self-reports and observer assessments of functioning. We used momentary daily activities to see how they informed self-reports of social cognitive abilities and everyday social functioning collected at the end of the period.

We had several hypotheses. Our primary hypothesis was that observer ratings of social functioning would be more strongly related to EMA -derived data regarding socially relevant daily functioning than self-reports of social functioning. Our secondary hypothesis was that momentary reports of sad mood would predict self-reports of social functioning, with very low sadness scores predicting over-estimation and higher sadness scores predicting self-reports of poorer social functioning. Given our previous findings of essentially equivalent IA deficits in people with SCZ and BPD, we also anticipated no diagnostic differences in the relationship between objective social contact and self-reported social functioning.

2.0. Methods

2.1. Participants.

Participants who met DSM-V criteria for Schizophrenia, any subtype, Schizoaffective Disorder, Bipolar Disorder (I or II), with or without current or previous psychotic symptoms participated in this study. They were recruited at three different sites: The University of Miami Miller School of Medicine (UM), the University of California San Diego (UCSD), and The University of Texas at Dallas (UTD). UM participants were recruited from the Jackson Memorial Hospital-University of Miami Medical Center and the Miami VA Medical Center. UCSD participants were recruited from the UCSD Outpatient Psychiatric Services clinic, a large public mental health clinic, the San Diego VA Medical Center, and other local community clinics and by word of mouth. UTD participants were recruited primarily from Metrocare Services, a non-profit mental health services organization in Dallas County, TX, and from other local clinics. The study was approved by each University’s respective Institutional Review Board, and all participants provided written informed consent. Diagnostic information was collected by trained interviewers using the Mini International Neuropsychiatric Interview (MINI; Sheehan et al., 1998) and the psychosis module of the Structured Clinical Interview for DSM Disorders-5 (SCID-5; First et al., 2015) and a local consensus procedure was used to generate final diagnoses.

2.1.1. Inclusion/Exclusion Criteria.

To be eligible, participants had to meet criteria for one of the disorders mentioned above. Individuals with bipolar disorder also had to meet a staging model severity of 3 or higher, indicating at least one mood episode recurrence or incomplete remission from a first-episode (Frank et al., 2014). Participants were also required to be have no hospitalizations and to be on a stable medication regimen for a minimum of six weeks with no medication dose changes >20% for a minimum of two weeks. All antipsychotics or antipsychotic combinations were accepted.

For participants in both diagnostic groups exclusion criteria included: (1) history of or current medical or neurological disorders that may affect brain functioning (e.g., CNS tumors, seizures, or loss of consciousness for over 15 minutes), (2) history of or current intellectual disability (IQ<70) or pervasive developmental disorder according to the DSM-5 criteria, (3) presence of substance use disorder other than tobacco not in remission for at least six months, (4) visual or hearing impairments that interfere with assessment, and (5) lack of proficiency in English. Cases with a Wide Range Achievement Test-3rd edition (WRAT-3; Jastak, 1993) grade Equivalent score of less than 8th grade was also not enrolled because of concerns that they would not be able to read the consent form and might have other challenges in reading the instructions for the EMA queries and self-assessment scales. We also excluded all potential participants who did not complete the clinical assessment at endpoint, regardless of the amount of EMA data collected prior to that assessment.

2.2. Overview of the study methods.

Participants had a brief office visit at the beginning of the study. They received diagnostic and clinical assessments and then began the 30-day EMA period described below. At the end of the EMA period, a follow-up visit took place, with a repeat of the baseline clinical assessments and a detailed performance-based assessment of neurocognition, social cognition, and functional capacity. Participants also self-reported on their everyday functioning and their social and neurocognitive abilities. Observers who had not performed the baseline assessments performed these follow-up assessments. The assessment time frame for all self-report and observe ratings of functioning were “the last month”, which is convergent with the 30-day EMA period. These analyses report on a subset of this very detailed assessment.

2.3. Assessments.

2.3.1. Social cognitive ability

was assessed with the Observable Social Cognition Rating Scale (OSCARS). Both observers and participants completed the OSCARS (Healey et al., 2015), an 8-item interview-based assessment of social cognition. Each question on the OSCARS is made up of a question probing a domain of social cognition (attribution style, cognitive rigidity, theory of mind, emotional perception, and jumping to conclusions). For each item, study participants ranked their abilities using a 7-point scale, with higher rankings translating to greater impairment. For this study we used the total score, which was the sum of all 8 items for both observers and participants. The internal consistency of the OSCARS was found to be .80 in participants in the development study. In a recent study, the OSCARS demonstrated adequate psychometric properties to detect functioning deficits in participants with schizophrenia and efficiently identify individuals in need of additional assessment or psychosocial interventions (Halverson et al., 2020).

2.3.2. Real-World Functioning

was rated with the 31-item version of the Specific Levels of Functioning (SLOF; Schneider and Struening, 1983). The SLOF is an informant- or self-rated assessment of functioning. In this study we focused on the interpersonal functioning scale because of our EMA data on social activities. A trained rater administered the measure to participants to obtain a self-report of functioning. In line with past use of this scale in several studies, observers completed the scale as if it were a questionnaire (Harvey et al., 2011; Pinkham et al., 2018). Each item was rated from 1–5, with higher scores reflecting better functioning.

2.4. Procedures

2.4.1. All-sources Observer SLOF and OSCARS Ratings.

In this study, we generated ratings on the basis of all sources of information. In our previous studies that recruited at these sites (Harvey et al., 2011; Pinkham et al., 2016; 2018), we discovered that we were only able to obtain a high-quality informant (high contact clinician; caregiver who lived with the participant) for about 75% of potential participants. Rather than reject cases without such an informant, we used an all sources observer rating procedure as we described before (Harvey et al., 2019). Raters who interviewed the participants on the SLOF and the OSCARS also had access to informant reports if available for these two rating scales as well as clinical and performance-based assessment data (including social cognitive test performance). Raters were instructed to make their ratings on the basis of what they thought was the correct rating, regardless of the source of information, and to consider information from their own observations of the participant when generating their ratings. These raters were unaware of any EMA data on the participants.

2.4.2. EMA Procedures

A Samsung smartphone with Android OS was used to deliver EMA surveys. The device was provided by the investigators to participants. Participants received text messages with weblinks to EMA surveys 3 times daily for 30 days, with data instantly uploaded to a cloud-based data capture system. The signals occurred at stratified random intervals that varied from day to day within, on average, 2.0-hour windows starting at approximately 9:00AM and ending at 9:00PM each day. The first and last daily assessment times were adjusted to accommodate each participant’s typical sleep and wake schedules. All responses were time-stamped and were only allowed within a 1-hour period following the signal, although participants had the option of silencing alarms for 30-minute intervals (e.g., driving, naps, classes). We selected this one-hour window in contrast to other intervals (e.g., 15 minutes) because of our findings in previous studies that participants commonly engaged in only one activity in the past hour (Strassnig et al., 2020) and were also either home or away for the entire past hour in 85% of the surveys returned (Granholm et al., 2020; Parrish et al., 2020). Thus, the longer window was aimed at augmentation of adherence.

An in-person training session (typically <20 min) was provided on how to operate and charge the device and respond to surveys, including the meaning of all questions and response choices. EMA surveys were check-box questions asking about behaviors performed since the previous survey. The first question asked about the participant’s location, with the following options: at my home, at home of family, at home of friends, at work, at outpatient medical visit, in hospital, at community center, in public business/store, in vehicle, outside walking, in class/educational setting, inside other, and outside other. Then participants were queried about who they were with. Options for this question included alone, spouse or partner, friends, other family members, pets, healthcare providers, other known people, and unknown people. The subsequent screens then used branching logic to delivered customized surveys that asked what the participant was doing based on where they were and with whom they were at the present time. Being with a pet, but not a human, was considered “alone”.

Mood questions included sadness, happiness, anxiety, and relaxation. All mood items were scaled with a 1–7 range, with higher values indicating greater severity. We examined only sadness, for two reasons. First is because other mood states have not been reported to be associated with systematic biases in IA. The second is because of potential redundancy. In the entire sample, the MMRM association (adjusting for random intercept, day, and survey) between the up to 90 happy ratings and time-linked linked sad ratings was: X2(6) =3771, p=3.2*10−6, B=−.48 and in previous studies of smaller samples, less sadness and more happiness were essentially redundant with each other (Depp et al., 2016).

2.5. Data Analyses.

Data analyses were performed using the SPSS edition 26 Generalized Linear Models (GLM) module (IBM, 2020). In order to perform a sophisticated examination of all of the survey data, we used a mixed model repeated measures analysis of variance (MMRM) where we predicted self and observer reports of everyday interpersonal functioning and social cognitive abilities with information about where the participant was, who they were with, and their current mood. We entered day (1–30) and survey (1–3) as the repeated factors and diagnosis as the between-subjects factor, using subject as a random intercept. We combined all variants of BPI into a single group, as well as combining schizophrenia and schizoaffective diagnoses into a single group. For each model, we used the omnibus test to determine that the fitted model improved on the intercept only model.

We predicted the four Global Outcomes variables (Self-report vs. Rater Judgment; Social Functioning vs. Social Cognitive Ability) in separate analyses. In these analyses, we used the repeated-measures from the EMA surveys of home vs. away and alone vs. with someone and their two-way interaction as the primary independent variables, with day and time of day as additional repeated-measures factors. We then sequentially added diagnosis (schizophrenias spectrum vs. bipolar disorder) and momentary assessments of sad mood to the models.

In the final analyses, we examined the difference between self-reports and observer reports on Social functioning and social cognitive abilities, expressed as the extent of overestimation of self-reports compared to observer ratings. We used the same analysis strategy for these models as well.

3.0. Results.

Descriptive information on the participants with bipolar disorder (n=71) and schizophrenia (n=102) is presented in Table 1. Participants with bipolar disorder had more education and higher WRAT-3 reading scores than the participants with schizophrenia. In terms of functional milestones, participants with bipolar disorder were more likely to have ever been married or equivalent, with no significant differences in current residential or employment status. There was a total of 11,907 EMA surveys with non-missing data for being home, being alone, and the sadness item for cases with both SLOF and OSCARS self-report and observer ratings with 7110 from SCZ and 4797 from BPI. Adherence to surveys was 78% for participants with SCZ and 75% for participants with BPI.

Table 1.

Descriptive and Demographic Information on Participants

Schizophrenia Bipolar Illness
n=102 n=71
M SD M SD t p
Age 41.98 10.44 39.22 11.75 1.63 .11
Years of Education 12.53 2.32 14.22 2.64 4.42 <.001
Mothers Education 13.05 3.54 13.67 3.67 1.81 .069
WRAT-3- Standard Score 95.42 11.85 102.13 11.70 3.67 <.001
X2 p
Sex (% Female) 48 69 8.22 .004
Racial Status (%)
   Caucasian 32 53 15.27 .009
   African American 54 25
   Asian 2 3
   Native American, Hawaiian, Alaskan 1 1
   Other, Multiple, Unknown 11 12
Ethnic Status
   Hispanic 24 29 0.64 .42
   Non-Hispanic 76
Ever Married or Equivalent (%) 49 70 7.14 .007
Financially Responsible (%) 71 70 0.02 .88
Unemployed for More than one Year(%) 60 45 2.74 .10
M SD M SD t p
Mean Momentary Depression Rating 2.58 1.40 2.88 1.42 1.57 .13
Open in a new tab

Table 2 presents the means and standard deviations for the observer reported and self-reported social functioning and social cognition variables. As can be seen in the table, there were no significant diagnostic group differences in self-reported social functioning, while the observer ratings suggested that the participants with BPI had better social functioning. Participants with SCZ reported they that had significantly worse social cognitive ability than the participants with BPI.; observer ratings were consistent with this difference.

Table 2.

Self-reported and Observer-rated Social Cognition and Social Functioning

Schizophrenia N=102 Bipolar Illness N=71
M SD M SD t p
SLOF Interpersonal Functioning
Self-reported 24.23 6.77 24.35 6.38 0.11 .91
Observer Rated 22.56 5.03 24.95 5.53 2.75 .007
Difference (Overestimation) 1.67 5.87 −.60 5.54 1.85 .06
OSCARS Total Score
Self-reported 23.81 11.61 17.41 9.27 3.32 <.001
Observer Rated 24.10 7.10 20.19 7.04 3.28 <.001
Difference (Overestimation) −.29 10.82 2.78 7.69 0.75 .45
Open in a new tab

Note. For SLOF: Higher scores are Better; For OSCARS, Higher scores reflect more impairment. All difference scores represent over-estimation with positive values

Paired t-tests indicated that participants with schizophrenia rated their social functioning significantly better than the observers, t (101) =2.45, p=.012, while there was no difference on average between their self-reports on the OSCARS and the observer ratings, t (101) =.08, p=.91. Participants with BPI generated reports of their social functioning that did not differ from reports of the observers, t (70) =0.41, p=.68. However, they reported that their social cognitive functioning was significantly better than the observer ratings, t (70) =3.05, p=.005.

3.1. MMRM Analyses of Self-Reported and Observer Rated Social Functioning and Social Cognition.

The analyses of self-reported and observer rated social functioning and social cognition ability are presented in Table 3. When predicting self-reported social functioning with home, alone, and their interaction, the model did not improve on the null model. When diagnosis was added to the model, the model also was not improved. When momentary sadness was added into the model, the model achieved significance compared to the null model. There were statistically significant effects of momentary sadness, diagnosis, and answering more surveys while away from home, but no effects of being alone or interactions of being home and alone on self-reported social functioning. Participants who answered more surveys while at home reported better social functioning, EM Mean=24.48, SE=.13, than those who answered surveys from away: EM Mean 24.01, SE=.09.

Table 3.

Results from Hierarchical Modeling of Self Reports and Observer Ratings of Social Functioning and Social Cognitive Abilities

Social Functioning
Self-Reported Observer Rated
X2 df P X2 df p
Omnibus Test 38 34 .29 85.56 34 <.001
Home - - 66.18 1 <.001
Alone - - 2.90 1 .09
Home × Alone - - 1.07 1 .30
Omnibus Test 34 35 .51 544.75 35 <.001
Home - - 38.56 1 <.001
Alone - - 22,22 1 <.001
Home × Alone - - 11.26 1 <.001
Diagnosis - - 488.47 1 <.001
Omnibus Test 1186.08 41 <.001 944.07 41 <.001
Home 9.11 1 .003 23.27 1 <.001
Alone 0.14 1 .71 23.84 1 <.001
Home × Alone 1.40 1 .34 8.92 1 .003
Diagnosis 6.10 1 .014 526.92 1 <.001
Sadness 1215 6 <.001 401.87 6 <.001
Social Cognitive Ability
Self-Reported Observer Rated
X2 df P X2 df P
Omnibus Test 63.57 34 <.001 103.74 34 <.001
Home 50.79 1 <.001 64.81 1 <.001
Alone 17.79 1 <.001 9.66 1 .002
Home × Alone 2.43 1 .12 .01 1 .99
Omnibus Test 944.62 35 <.001 606.15 35 <.001
Home 18.69 1 <.001 36.70 1 <.001
Alone 47.14 1 <.001 .04 1 .82
Home × Alone 1.75 1 .19 5.35 1 .021
Diagnosis 922.93 1 <.001 361.84 1 <.001
Omnibus Test 939.04 41 <.001 986.16 41 <.001
Home 13.36 1 <.001 29.29 1 <.001
Alone 53.31 1 <.001 0.03 1 .86
Home × Alone 2.10 1 .15 5.21 1 .02
Diagnosis 916.31 1 <.001 576.18 1 <.001
Sadness 337.22 1 <.001 3898.76 1 <.001
Open in a new tab

When observer-reported social functioning was examined with the same strategy, observer rated social functioning was statistically higher in participants who answered more surveys while away from home. When diagnosis was added to the model, there were statistically significantly higher observer rated scores for participants with BPI and participants who answered more surveys while home alone got the lowest ratings from observers. When momentary sadness was added there were statistically lower observer-rated functioning scores associated with greater momentary sadness.

When we modeled self-reported social cognitive ability, all of the models improved on the null model. Participants who answered more surveys while alone and participants who answered more surveys away from home rated themselves as having better social cognitive ability. Bipolar patients rated themselves as having better ability and higher momentary sadness predicted self-reports of poorer social cognitive abilities.

Observer-reported social cognitive ability was examined with the same modeling` strategy and all of the models improved on the null model. Participants who answered more surveys away from home were rated as having better social cognitive abilities, as were participants who answered more surveys while with another person. The participants who answered more surveys while home alone got the lowest observer ratings. Bipolar participants were received higher ratings. Greater momentary sadness was associated with observer ratings of poorer social cognitive abilities.

3.2. MMRM Of Over Estimation

In the final two MMRM analyses, we modeled the overestimation of social functioning and social cognitive ability. Table 4 presents the results of these analyses, as well as the EM Means from the final step in the model. For social functioning, the model became significant when diagnosis was added. Participants with schizophrenia significantly overestimated their functioning compared to participants with BPI. Participants who answered more surveys while home overestimated their functioning the most and participants who answered more surveys away from home underestimated their functioning. Participants who endorsed the complete absence of sadness overestimated their functioning compared to observer ratings, while increasing severity of momentary sadness was associated with a U-shaped curve, with congruence between self-reports and observer ratings at moderate levels of sadness and under-estimation at the highest levels.

Table 4.

Prediction of Overestimation of Social Functioning and Social Cognitive Ability

Social Functioning Social Cognitive Ability
X2 df p X2 df p
Omnibus Test 210 35 <.001 39.26 35 .006
Home 5.18 1 .023 1.62 1 .20
Alone 2.89 1 .43 2.34 1 .12
Home × Alone 4.84 1 .028 21.90 1 <.001
Diagnosis 197.19 1 <.001 5.23 1 .02
Omnibus Test 1186.08 41 <.001 168.30 41 <.001
Home 9.11 1 .003 2.28 1 .13
Alone 0.14 1 .71 1.48 1 .22
Home × Alone 1.40 1 .34 24.16 1 <.001
Diagnosis 6.10 1 .014 3.79 1 .051
Sadness 1215.45 6 <.001 108.89 1 <.001
EM Means From Final Model Social Functioning Social Cognitive Ability
Overestimation M SE M SE
SCZ: 1.43 0.8 −.36 .04
BPI: −.22 .10 .27 .04
Home Alone 0.70 .10 .43 .04
Home with Someone 0.20 .10 −.12 .04
Away Alone 0.58 .18 .25 .08
Away with Someone −.20 .10 −.43 .05
Sadness Rating
1 1.85 .10 1.70 .18
2 0.09 .13 1.24 .26
3 −.47 .15 0.45 .29
4 −.55 .16 −.51 .29
5 −.66 .14 −.96 .36
6 −1.55 .27 −1.49 .36
7 −1.54 .31 −1.54 .36
Open in a new tab

For overestimation of social cognitive ability, the model became significant when diagnosis was added. Participants with BPI significantly overestimated their functioning compared to participants with schizophrenia. However, across both groups overestimation of social cognitive abilities was associated with answering more surveys while being home and alone. Finally, reduced levels of momentary sadness were associated overestimation of social cognitive ability, which manifested the same U-shaped curve across moderate and t higher levels of momentary sadness.

4.0. Discussion

Participants with schizophrenia and bipolar disorder generated self-ratings of their everyday social functioning that were uncorrelated with 30 days of real-world sampling of socially relevant information, including if they were home and if they were alone. In contrast to the disjunction between 30 days of EMA sampling and self-reports of functioning, observer ratings on the same scales were associated with the results of EMA survey sampling. These findings, suggest that being more often home and alone was associated with a clinical impression of poorer social functioning and poorer social cognition. Thus, our hypothesis that participants with SCZ and BPI would be challenged in terms of the ability to utilize their daily activities to generate self-reports of functioning convergent with observer ratings was confirmed. The rates of being home and alone in this sample of participants with BPI and SCZ was very similar to our previous studies of participants with SCZ compared to HC (Granholm et al., 2020; Parrish et al., 2020), suggesting that the EMA data are representative of this population. We also confirmed our hypothesis that momentary sadness ratings would systematically associate with self-reports of both social functioning and social cognitive ability. There were opposite introspective biases associated with higher and lower momentary sadness ratings. Our exploratory hypothesis that there would not be major differences in IA and IB between the groups was confirmed. Participants with BPI were rated as higher functioning and more capable in social cognitive abilities, but diagnosis did not contribute toward the use of daily experience to generate self-assessments of social cognition

Overestimation of social cognitive abilities and social functioning was greatest in participants who answered more surveys while alone, and, for self-reported social cognition, home alone. Participants who answered more surveys while away with someone underestimated their social functioning and social cognition compared to observer ratings. We have previously found participants with SCZ who had never been employed reported that their work skills were equivalent to and their ability to perform everyday activities was superior to SCZ participants who were currently employed full time (Gould et al., 2013), despite performing notably more poorly on measures of neurocognition (effect size = 1.0 SD). The current results suggest that variation in ongoing social experiences may predict self-assessments of abilities as well and that experience may lead to judgments of functioning that are accurate to underestimated.

These data regarding momentary sadness are congruent with previous findings that suggested that extremely, and probably unrealistically, low scores on self-report and clinician depression rating scales are associated with overly positive self- assessments of real-world functioning (Harvey et al., 2017; 2019). Responding to EMA surveys with momentary reports of moderately sad moods was associated with generation of self-assessments were congruent with observer ratings. Finally, the lack of experience hypothesis seems supported by findings that participants who respond to fewer surveys in the company of other people rate both their social functioning and social cognitive abilities as better than participants who are responding to surveys while away from home in the company of others.

As in any study, there are limitations in this dataset. The sample sizes are not equal in terms of diagnosis, but the current study is still one of the largest dense-sampling studies of EMA in severe mental illness to date. Demographic differences in racial/ethnic status, sex distributions, history of marriage, and educational attainment were detected between participants. These same differences are common even in very large-population-based studies (Harvey et al., 2016). Participants were not fully adherent with sampling and missing data could be based being un able to respond to survey invitations while performing certain activities, some of which could have been performed away from home in the company of others (e.g., working). Obviating this concern is the fact that that the participants with the top 20% adherence in this study (3 or fewer missed surveys) answered 35% of the surveys while away and those whose adherence was lowest (40 or more missed surveys) answered 40% of their surveys while away from home. There was no HC sample, although the EMA results for the participants with SCZ in this study were very similar to those seen in participants in previous comparative studies with HC. As noted above, previous studies also have found similar patterns of overestimation of abilities and similar correlates of these overestimations in HC and SCZ. Self-reported depressed mood was also correlated with self-reports of both poorer social functioning and poorer social cognition in HC and in participants with SCZ, in an independent sample using the same rating strategies and scales (Oliveri et al., 2019). We did not measure the full syndrome of depression in this study on a momentary basis. We have shown in other EMA studies that momentary reports of sadness collected with EMA are cross sectionally and longitudinally correlated with in-person clinical assessments (Targum et al., in press).

Another important interpretive issue is the quality of social interactions, and not just the quantity. In an analysis of the quality of behaviors, including social ones, in this database (Strassnig et al., submitted), there were some relevant findings. Explicitly social activities, including “social activities” or “visiting family” constituted 2.1% of all survey responses. The second finding was that unproductive activities (sleeping, smoking, just sitting, nothing) accounted for 32% of all surveys, but were significantly less likely when the participant was not alone. Productive activities, home or away, were more common when not alone, thus suggesting a generalized positive association with social contact and productivity. In a previous EMA study it was shown that suicidal ideation was not associated with the number of social contacts or perceptions of competence during the interactions, but rather the belief that social interactions were just not worth the effort (Depp et al., 2016). Thus, the association between social contact, motivation to engage in social behavior, and other productive outcomes clearly requires further attention.

4.1. Conclusions.

This study provides additional information suggesting that participants with severe mental illness do not rely on their actual longitudinal activities when generating global self-reports of functioning. However, observer ratings of social functioning were significantly correlated with daily socially-relevant activities measured longitudinally with EMA. Participants reporting very low levels of momentary sadness tended to generate ratings of their functioning that were positively biased compared to those generated by observers. Similarly, participants with lower levels of momentary social engagement (i.e., home and alone) generated ratings that were also positive biased, particularly compared to participants who were away from home with someone else. These data suggest that self-reports of everyday functioning coming directly from participants with SMI may be less useful then data collected on a momentary basis as a phenotype for the correlates of neurobiological or genomic variables or the effects of treatment.

Acknowledgments.

All authors who contributed to this paper are listed as authors. No professional medical writer was involved in any portion of the preparation of the manuscript.

This research was supported by NIMH grant RO1MH112620 to Dr. Pinkham.

Conflict of interest statement

In the last year, Dr. Durand has received consulting fees or travel reimbursements from Otsuka/Lundbeck Italy and Spain. Dr. Strassnig has served as a consultant to Signant Health. Dr. Raeanne C. Moore is a co-founder of KeyWise AI, Inc. and a consultant for NeuroUX. Dr. Harvey has received consulting fees or travel reimbursements from Alkermes, Bio Excel, Boehringer Ingelheim, Intra-Cellular Therapies, Minerva Pharma, Otsuka America, Regeneron, Roche Pharma, and Sunovion Pharma. He receives royalties from the Brief Assessment of Cognition in Schizophrenia and the MATRICS Consensus Battery. He has a research grant from Takeda and from the Stanley Medical Research Foundation. Dr. Pinkham has served as a consultant for Roche Pharma. The other authors have no potential Biomedical Conflicts of Interest.

Footnotes

The data in this study are being deposited in the NIMH RDOC repository. 6 Months after data lock they will be available for public access. In the interim, the authors are happy to share the data that underlie this paper.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

5.0 References

  1. Ben-Zeev D, McHugo GJ, Xie H, Dobbins K, Young MA. 2012.Comparing retrospective reports to real-time/real-place mobile assessments in individuals with schizophrenia and a nonclinical comparison group. Schizophr Bull. 38(3):396–404. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Charlson FJ, Ferrari AJ, Santomauro DF, et al. 2018. Global epidemiology and burden of schizophrenia: findings from the global burden of disease study 2016. Schizophr Bull. 44(6):1195–1203. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Depp CA, Bashem J, Moore RC, et al. 2019. GPS mobility as a digital biomarker of negative symptoms in schizophrenia: a case control study. NPJ Digit Med;2:108. doi: 10.1038/s41746-019-0182-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Depp CA, Moore RC, Perivoliotis D, et al. 2016. Social behavior, interaction appraisals, and suicidal ideation in schizophrenia: The dangers of being alone. Schizophr Res.172(1–3):195–200 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Durand D, Strassnig M, Sabbag S, et al. 2015. Factors influencing self-assessment of cognition and functioning in schizophrenia: implications for treatment studies. Eur Neuropsychopharmacol. 5(2):185–91. doi: 10.1016/j.euroneuro.2014.07.008. Epub 2014 Jul 25. PMID: 25104226; PMCID: PMC4305496. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. First MB, Williams JBW, Karg RS, Spitzer RL. 2015. Structured Clinical Interview for DSM-5—Research Version (SCID-5 for DSM-5, Research Version; SCID-5-RV). Arlington, VA, American Psychiatric Association. [Google Scholar]
  7. Frank E, Nimgaonkar VL, Phillips ML, Kupfer DJ. 2014. All the world’s a (clinical) stage: rethinking bipolar disorder from a longitudinal perspective. Mol psychiatry July 22. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Gawęda L, Woodward TS, Moritz S Kokoszka A, 2013. Impaired action self-monitoring in participants with schizophrenia with auditory hallucinations. Schizophr. Res, 144(1–3), pp.72–79. doi: 10.1016/j.schres.2012.12.003 [DOI] [PubMed] [Google Scholar]
  9. Gould F, McGuire LS, Durand D, et al. 2015. Self-assessment in schizophrenia: Accuracy of evaluation of cognition and everyday functioning. Neuropsychology. ep;29(5):675–82. doi: 10.1037/neu0000175. Epub 2015 Feb 2. PMID: 25643212; PMCID: PMC4522405. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Gould F, Sabbag S, Durand D, Patterson TL, Harvey PD. 2013. Self-assessment of functional ability in schizophrenia: milestone achievement and its relationship to accuracy of self-evaluation. Psychiatry Res May 15;207(1–2):19–24. doi: 10.1016/j.psychres.2013.02.035. Epub 2013 Mar 26. PMID: 23537844; PMCID: PMC3640769. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Granholm E, Holden JL, Mikhael T, et al. 2020. What Do People With Schizophrenia Do All Day? Ecological Momentary Assessment of Real-World Functioning in Schizophrenia. Schizophr Bull. 46(2):242–251. doi: 10.1093/schbul/sbz070 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Halverson TF, Hajdúk M, Pinkham AE, et al. 2020. Psychometric properties of the Observable Social Cognition Rating Scale (OSCARS): Self-report and informant-rated social cognitive abilities in schizophrenia [published online ahead of print, 2020 Feb 26]. Psychiatry Res 286:112891. doi: 10.1016/j.psychres.2020.112891 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Harvey PD, Deckler E, Jones MT, Jarskog LF, Penn DL, Pinkham AE. 2019. Autism symptoms, depression, and active social avoidance in schizophrenia: Association with self-reports and informant assessments of everyday functioning. J Psychiatr Res 115:36–42. doi: 10.1016/j.jpsychires.2019.05.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Harvey PD, & Pinkham A 2015. Impaired self-assessment in schizophrenia: Why patients misjudge their cognition and functioning. Current Psychiatry, 14(4), 53–59. [Google Scholar]
  15. Harvey PD, Siever LJ, Huang GD, et al. 2014.The genetics of functional disability in schizophrenia and bipolar illness: Methods and initial results for VA cooperative study #572. Am J Med Genet B Neuropsychiatr Genet; 165B(4):381–9. doi: 10.1002/ajmg.b.32242. Epub 2014 May 5. PMID: 24798943. [DOI] [PubMed] [Google Scholar]
  16. Harvey PD, Twamley EW, Pinkham AE, Depp CA, Patterson TL. 2017. Depression in Schizophrenia: Associations with Cognition, Functional Capacity, Everyday Functioning, and Self-Assessment. Schizophr Bull. 43(3):575–582. doi: 10.1093/schbul/sbw103 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Healey KM, Combs DR, Gibson CM, et al. , 2015. Observable Social Cognition--A Rating Scale: an interview-based assessment for schizophrenia. Cogn. Neuropsychiatry 20(3), 198–221. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. IBM Corporation. 2020. Statistical package for the social sciences (SPSS) Edition 26, Armonk, NY: 2020 [Google Scholar]
  19. Jastak S: 1993. Wide-Range Achievement Test, 3rd ed. San Antonio, TX, Wide Range, Inc [Google Scholar]
  20. Jones MT, Deckler E, Laurrari C, et al. 2019. Confidence, performance, and accuracy of self-assessment of social cognition: A comparison of schizophrenia patients and healthy controls. Schizophr Res Cogn 19:002–2. Published 2019 Feb 8. doi: 10.1016/j.scog.2019.01.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Koren D, Poyurovsky M, Seidman LJ, et al. . 2005. The neuropsychological basis of competence to consent in first-episode schizophrenia: a pilot metacognitive study. Biol. Psychiatry;57(6):609–616. doi: 10.1016/j.biopsych.2004.11.029 [DOI] [PubMed] [Google Scholar]
  22. Kother U, Veckenstedt R, Vitzthum F, et al. , 2012. “Don’t give me that look” — overconfidence in false mental state perception in schizophrenia. Psychiatry Res 196, 1–8. doi: 10.1016/j.psychres.2012.03.004 [DOI] [PubMed] [Google Scholar]
  23. Kruger J, Dunning D. 1999. Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J Pers Soc Psychol 77(6):1121–34. doi: 10.1037//0022-3514.77.6.1121. PMID: 10626367. [DOI] [PubMed] [Google Scholar]
  24. Merikangas KR, Swendsen J, Hickie IB, et al. 2019. Real-time Mobile Monitoring of the Dynamic Associations Among Motor Activity, Energy, Mood, and Sleep in Adults With Bipolar Disorder. JAMA Psychiatry. 76(2):190–198. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Moritz S, Göritz AS, Gallinat J, et al. 2015. Subjective competence breeds overconfidence in errors in psychosis. A hubris account of paranoia. J. Behav. Ther. Exper. Psychiat, 48, 118–124. doi: 10.1016/j.jbtep.2015.02.011 [DOI] [PubMed] [Google Scholar]
  26. Moritz S, Woznica A, Andreou C Köther U, 2012. Response confidence for emotion perception in schizophrenia using a Continuous Facial Sequence Task. Psychiatr. Res, 200(2–3), 202–207. doi: 10.1016/j.psychres.2012.07.007 [DOI] [PubMed] [Google Scholar]
  27. Oliveri LN, Awerbuch AW, Jarskog LF, et al. . 2020. Depression predicts self-assessment of social function in both patients with schizophrenia and healthy people. Psychiatry Res 2020; 284:112681. doi: 10.1016/j.psychres.2019.112681 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Orfei MD, Piras F, Banaj N, et al. 2017. Unrealistic self-overconfidence in schizophrenia is associated with left presubiculum atrophy and impaired episodic memory. Cortex, 86, 132–139. doi: 10.1016/j.cortex.2016.10.017 [DOI] [PubMed] [Google Scholar]
  29. Parrish EM, Depp CA, Moore RC, Harvey PD, Mikhael T, Holden J, Swendsen J, Granholm E. Emotional determinants of life-space through GPS and ecological momentary assessment in schizophrenia: What gets people out of the house? Schizophr Res 2020. October;224:67–73. doi: 10.1016/j.schres.2020.10.002. Epub 2020 Oct 24. PMID: 33289659. [DOI] [PubMed] [Google Scholar]
  30. Penn DL, Sanna LJ, Roberts DL. 2008. Social cognition in schizophrenia: an overview Schizophr. Bull., 34 (3) 408–411. doi. 10.1093/schbul/sbn014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Perez MM, Tercero BA, Penn DL, et al. 2020. Overconfidence in social cognitive decision making: Correlations with social cognitive and neurocognitive performance in participants with schizophrenia and healthy individuals. Schizophr Res 2020 October;224:51–57. doi: 10.1016/j.schres.2020.10.005. Epub 2020 Oct 21. PMID: 33097367; PMCID: PMC7722219. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Pinkham AE, Penn DL, Green MF, Harvey PD, 2016. Social Cognition Psychometric Evaluation: Results of the Initial Psychometric Study. Schizophr. Bull. 42(2), 494–504 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Pinkham AE, Harvey PD, Penn DL. 2018. Social Cognition Psychometric Evaluation: Results of the Final Validation Study. Schizophr Bull. 44(4):737–748. doi: 10.1093/schbul/sbx117 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Raugh IM, James SH, Gonzalez CM, et al. 2020. Geolocation as a Digital Phenotyping Measure of Negative Symptoms and Functional Outcome [published online ahead of print, 2020 Aug 27]. Schizophr Bull.;sbaa121. doi: 10.1093/schbul/sbaa121 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Savla GN, Vella L, Armstrong CC, et al. 2013. Deficits in domains of social cognition in schizophrenia: a meta-analysis of the empirical evidence. Schizophr Bull 39(5), 979–992. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Schneider LC, Struening EL, 1983. SLOF: a behavioral rating scale for assessing the mentally ill. Soc. Work Res. Abstr 19(3), 9–21. [DOI] [PubMed] [Google Scholar]
  37. Sheehan DV, Lecrubier Y, Harnett-Sheehan K, et al. , 1998. The mini international neuropsychiatric interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview. J. Clin. Psychiatry; 59, 22–33. [PubMed] [Google Scholar]
  38. Silberstein JM, Harvey PD. 2019. Impaired introspective accuracy in schizophrenia: an independent predictor of functional outcomes. Cogn Neuropsychiatry. 24:28–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Silberstein JM, Pinkham AE, Penn DL, Harvey PD. 2018. Self-assessment of social cognitive ability in schizophrenia: association with social cognitive test performance, informant assessments of social cognitive ability, and everyday outcomes. Schizophr Res 2018; 199:75–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Siu CO, Harvey PD, Agid O, et al. , 2015. Insight and subjective measures of quality of life in chronic schizophrenia. Schizophr Res. Cogn, 2(3), 127–132. doi: 10.1016/j.scog.2015.05.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Strassnig MT, Harvey PD, Miller ML, Depp CA, Granholm E. 2020. Real World Sedentary Behavior and Activity Levels in Patients with Schizophrenia and Controls: An Ecological Momentary Assessment Study. Ment Health Phys Activ, 20, 100364. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Strassnig MT, Kotov R, Fochtmann L, et al. 2018. Associations of independent living and labor force participation with impairment indicators in schizophrenia and bipolar disorder at 20-year follow-up. Schizophr Res; 197:150–155. doi: 10.1016/j.schres.2018.02.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Strassnig MT, Miller ML, Moore RC, et al. (under review). Evidence for avolition in bipolar disorder? A 30-Day Ecological Momentary Assessment Comparison of Daily Activities in Bipolar Disorder and Schizophrenia. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Strassnig MT, Raykov T, O’Gorman C, et al. 2015. Determinants of different aspects of everyday outcome in schizophrenia: The roles of negative symptoms, cognition, and functional capacity. Schizophr Res;165(1):76–82. doi: 10.1016/j.schres.2015.03.033 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Strauss GP, Esfahlani FZ, Granholm E, et al. 2020. Mathematically Modeling Anhedonia in Schizophrenia: A Stochastic Dynamical Systems Approach [published online ahead of print, Feb 26]. Schizophr Bull. 2020; sbaa014. doi: 10.1093/schbul/sbaa014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Targum SD, Sauder C., Evans M, Saber JB, Harvey PD. In press. Ecological momentary assessment as a measurement tool in depression trials. J Psychiatr. Res [DOI] [PubMed] [Google Scholar]
  47. Ustün TB, et al. , 2010. Measuring ealth and isability: Manual for W isability Assessment Schedule WHODAS 2.0. World Health Org, Geneva. [Google Scholar]

RESOURCES