Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Dec 1.
Published in final edited form as: J Psychiatr Res. 2022 Nov 4;156:594–601. doi: 10.1016/j.jpsychires.2022.10.069

Accuracy of Immediate Self-assessment of Neurocognitive Test Performance: Associations with Psychiatric Diagnosis and Longitudinal Psychotic Symptoms

Orly Morgan a, Martin T Strassnig a, Raeanne C Moore b,c, Colin A Depp b,c, Robert A Ackerman d, Amy E Pinkham d, Philip D Harvey a,e
PMCID: PMC9899150  NIHMSID: NIHMS1849244  PMID: 36372002

Abstract

Participants with schizophrenia (SCZ) and bipolar disorder (BD) have challenges in self-evaluation of their cognitive and functional abilities, referred to as introspective accuracy (IA). Although psychotic symptoms are commonly found to be uncorrelated with cognitive performance, many models of the development of delusions focus on failures in self-assessment and responses biases during momentary monitoring. We performed a single 4-test cognitive assessment on 240 participants (schizophrenia n=126; bipolar disorder n=114) and asked them to make a judgment about their performance immediately after completion of each task. We related performance and these judgments to results of Ecological Momentary Assessments (EMA) of the momentary occurrence of psychotic symptoms (Voices, paranoid ideas, other delusions) collected over up to 90 surveys over a 30 days prior to the single cognitive assessment. We examined test performance and the accuracy of self-assessment at that assessment, looking at diagnostic differences in performance and mis-estimation of performance. Participants with bipolar disorder had better cognitive performance, but there were no differences in mis-estimation. Analyses of the correlation between cognitive performance and self-assessment were all significant and better cognitive performance predicted reduced errors in self-assessment. Examination of the 30-day course of psychotic symptoms and IA could only be performed in participants with schizophrenia, revealing correlations between more common occurrences of all three psychotic symptoms and increased absolute values for IA errors. These data are consistent with theories of cognitive response biases and the formation of delusions.

Keywords: schizophrenia, bipolar disorder, introspective accuracy, psychotic symptoms, ecological momentary assessment, voices, paranoia

1.0. Introduction

Cognitive and social cognitive deficits are a determinant of poor social, occupational, and everyday functioning in both schizophrenia and bipolar disorder (Bowie et al., 2010; Depp et al., 2012; Fett et al., 2011; Tabarés-Seisdedos et al., 2008). Cognitive deficits occur across clinical states (Bonnín et al., 2010; Fioravanti et al., 2005; Mann-Wrobel et al., 2011), are present in participants diagnosed with schizophrenia and bipolar disorder, and are found in otherwise unaffected relatives as well (Arts, et al., 2008; Bora,et al., 2009; 2017; Hochberger et al., 2016). In fact, cognitive deficits are identified as a core characteristic of both schizophrenia and bipolar disorder (Harvey et al., 2010). The profile of impairment in mood disorders and schizophrenia has been reported to be very similar at the time of the first episode (Reichenberg et al., 2009). Stefanopoulou et al. (2009) concluded that differences between diagnostic groups are quantitative rather than qualitative. A very large-scale comparative study of schizophrenia and bipolar disorder (n= 10,160; Harvey et al., 2016) showed the factor structure of cognition in schizophrenia and bipolar disorder to be best characterized as a single domain across both diagnoses. GWAS analyses supported these results, which found similar genomic correlations of the cognition latent trait and overlap with genomic polygenic scores in the general population (Harvey et al., 2020).

An additional shared characteristic of schizophrenia and bipolar disorder is a challenge in self-assessment of multiple abilities, commonly referred to as introspective accuracy (IA). The concept of introspective accuracy reflects the awareness of one's abilities, skills, performance, and decisions (Harvey and Pinkham, 2015) compared to objective reality. Impaired IA applies broadly to self-assessment in serious mental illness, which spans self-assessments of clinical symptoms (Amador et al., 1993), functional abilities (Gould et al., 2013), cognitive performance (Jones et al., 2021) and social cognitive abilities (Silberstein et al., 2018). Multiple studies have found impaired IA in both schizophrenia and bipolar disorder. For example, Strassnig et al. (2018) reported that participants with schizophrenia and bipolar disorder who were not living independently reported the same level of disability as those who were paying for their own housing. Durand et al. (2021) noted that self-reports of social functioning were uncorrelated with the actual performance of socially relevant activities over a 30-day, 90-survey ecological momentary assessment (EMA) study of participants with bipolar disorder and schizophrenia, although observers' ratings of social functioning were correlated with these outcomes.

While the dominant IA error appears to be an overestimation of abilities relative to various reference points (Gould et al., 2013; Jones et al., 2019; Moritz et al., 2014; 2015), and studies have shown that overestimation of both cognitive performance (Gould et al., 2015) and social cognitive performance (Silberstein et al., 2018) contribute to impairments in everyday functioning, underestimation is found in as many as 30% of participants (Gohari et al., 2022; Bowie et al., 2007). In order to examine the specific cognitive substrate of IA, Tercero et al., (2021) used a metacognitive version of the Wisconsin Card Sorting Test (WCST) and found that both schizophrenia and bipolar disorder participants overestimated the accuracy of their responses on a trial x trial basis over the 64-card sorting procedure. However, the participants with bipolar disorder generated summary ratings of their performance that were correlated with their objective task performance, while this correlation was completely absent in the participants with schizophrenia. Thus, IA appears to have several components (i.e., momentary judgments, confidence in those momentary judgments, and the ability to aggregate judgments and feedback into global self-assessments of performance). IA seems broadly impaired in schizophrenia and bipolar disorder, but perhaps with impairments at different stages of the self-assessment stream.

Some studies have found that delusional thinking correlates with overly optimistic self-assessments and tendencies toward arriving at conclusions based on limited external information (Moritz et al., 2014; 2015). A recent study suggested that particularly unrealistic self-assessments of global abilities may also be related to the momentary occurrence of psychotic symptoms. In that study (Gohari et al., 2022 ), participants with schizophrenia who were commonly home and alone over a 30-day, 90-survey EMA period self-reported better performance in everyday activities and work than participants who were more commonly away from home and with other people. Many of the activities on which they reported themselves to be highly competent can only be performed away from home (Taking public transportation, shopping), despite objective evidence that these same participants were almost always at their home during the observation period. The same participants also reported the more common occurrence of delusions on a momentary basis. Gohari et al. suggested that self-reporting high levels competence for things they were clearly not doing could be considered delusional thinking instead of an erroneous self-assessment. Thus examination of whether the more frequent occurrence of psychotic symptoms is associated with impairments in momentary competence judgments is of considerable interest.

The present study expands on previous results in several ways. First, we included participants with both schizophrenia and bipolar disorder to link psychotic experiences and momentary judgments. Instead of asking for global self-assessments of competence as an outcome, we examined the convergence between objective performance at an endpoint assessment of cognitive abilities and self-assessments of that performance, collected immediately after the tests were completed. These performance and self-assessment data were collected at the end of a 30-day longitudinal EMA period and were correlated with the momentary occurrences of psychotic symptoms, including delusions and hallucinations, collected in that EMA assessment period. Thus, we related individual’s cognitive impairments and immediate judgments of the quality of cognitive performance to their longitudinal experience of momentary psychotic symptoms. Historically, performance on neuropsychological assessments is minimally related to the severity of psychotic symptoms (e.g., Keefe et al., 2006; n=1332, r=.03). However, those studies did not measure the momentary occurrence of psychotic symptoms, relying instead on clinical ratings based on the retrospective reports of experiencing symptoms. Thus, a momentary assessment of psychosis could yield a more precise test of this correlational result and is worth examining.

We tested several hypotheses in this study. We expected that more frequent momentary occurrences of psychotic symptoms, particularly delusions and specifically paranoid ideation, would correlate positively with challenges in immediate self-assessment after performing an assessment of cognitive abilities. We further hypothesized that poorer objectively measured cognitive performance would predict greater challenges in self-assessment but would not eliminate the other potential influences on immediate self-assessment. Based on previous results, we expected that immediate mis-estimation of cognitive abilities would relate to more frequent momentary occurrences of delusions, particularly those with paranoid ideation. Finally, we planned to explore differential effects in schizophrenia vs. bipolar disorder, hypothesizing that both cognition and IA would be more impaired in the participants with schizophrenia.

2.0. Methods:

2.1. Over-view of the study.

These data come from a longitudinal study examining the 30-day course of activities, mood symptoms, and psychosis and endpoint assessments of cognitive performance measured with several different tasks. Self-assessments of cognitive performance were collected immediately after the assessment. This study started before the COVID-19 pandemic and 101 participants with schizophrenia and 76 with bipolar disorder were seen before the study was paused. After resumption, an additional 25 participants with schizophrenia and 43 participants with bipolar disorder were examined and are included in this report. The study was IRB approved at each of the three sites and participants signed an informed consent form. Participants were compensated $1.00 for each EMA survey and $50.00 for the endpoint assessment. Supplemental Figure 1 presents a flow chart for this study.

2.2. Participants:

The study included a total of 321 participants (n=163; schizophrenia, n=158; bipolar disorder). Endpoint cognitive assessments were performed in 240 participants (n=126; schizophrenia, n=114; bipolar disorder). See Table 1 for full demographic information. The study was started in 2018, interrupted by the COVID pandemic in early 2020, and restarted across the three sites in mid-2020 in line with local regulations.

Table 1.

Demographic and Clinical Information for the Schizophrenia and Bipolar Participant Samples.

Schizophrenia
(n = 126)		 Bipolar Disorder
(n = 114)		 Statistics, p
Age [M (SD)] 41.91 (10.74) 38.45 (11.69) t = 2.38, .018
Sex (% Female) 49% 68% χ2 = 8.05, .02
Racial status χ2 = 22.24, < .001
 White (%) 35% 57%
 Black (%) 52% 24%
 Other (%) 17% 19%
Ethnicity (% Hispanic) 22% 27% χ2 = 0.62, .43
Education (years) [M (SD)] 12.67 (2.39) 14.23 (2.42) t = 5.02, < .001
Mother’s education (years) [M (SD)] 12.67 (3.16) 13.77 (3.23) t = 2.59, .007
Employment χ2 = 14.07, < .001
 Full Time (%) 8% 27%
 Part Time (%) 22% 24%
Unemployed/disabled (%) 70% 49%
Unemployed for more than 12 months (%) 87% 76% χ2 = 4.01, .045
MADRS [M (SD)] 10.71 (10.86) 13.40 (10.57) t = 1.93, .055
YMRS [M (SD)] 1.01 (3.48) 3.15 (4.63) t = 4.06, < .001
PANSS [M (SD)]
 Positive symptoms 15.88 (5.04) 10.32 (4.23) t = 10.10, < .001
 Reduced experience 6.23 (4.08) 4.69 (2.21) t = 4.82, < .001
 Reduced expression 5.99 (2.46) 4.76 (2.29) t = 3.97, < 001
Open in a new tab

Note. MADRS = Montgomery–Åsberg Depression Rating Scale, YMRS = Young Mania Rating Scale, PANSS = Positive and Negative Syndrome Scale.

2.21. Inclusion/ Exclusion criteria:

Inclusion criteria were as follows: (1) Participants meeting diagnostic criteria for schizophrenia or bipolar disorder (diagnosed with DSM-5, APA, 2013) and who were clinically stable (i.e., no hospitalizations, stable medication regimen for a minimum of 6 weeks with no dose changes >20% for a minimum of 2 weeks. (2) Individuals with bipolar disorder had to have experienced at least one mood episode recurrence or incomplete remission from a first episode according to the staging approach by Frank et al. (2014).

Exclusion criteria included: (1) a history of, or current, medical or neurological disorders potentially affecting cognition (e.g., CNS tumors, seizures, or loss of consciousness for over 15 minutes), (2) a history of, or current, intellectual disability (IQ<70) or pervasive developmental disorder according to the DSM-5 criteria, (3) the presence of substance use disorder not in remission for at least six months, (4) visual or hearing impairments that interfere with assessment, and (5) lack of proficiency in English. Participants with a Wide Range Achievement Test-3rd edition (WRAT-3; Jastak, 1993) grade equivalent score of less than 8th grade were also not enrolled.

2.3. EMA Procedures.

A Samsung smartphone with Android OS was used to deliver the EMA surveys, with participants using their own phone or one provided by the research team. Participants received text messages with links to EMA surveys three times daily for 30 days. Survey data were not stored on the device but were sent to an encrypted, HIPPA compliant cloud storage location in Amazon Web Services, with data plans provided through the study. This system allowed researchers to access participant data in real-time and monitor progress daily (Amazon Webservices, 2020). The text notifications occurred at stratified random intervals that varied from day to day within, on average, 4.0-hour windows starting at approximately 9:00 AM and ending at 9:00 PM each day. The first and last daily assessment times were adjusted to accommodate each participant's typical sleep and wake schedules. All responses were time-stamped and were only allowed within 1 hour following the signal, although participants had the option of silencing alarms for 30-minute intervals (e.g., driving, naps, classes). An inperson training session (typically <20 min) was provided on operating and charging the device and responding to surveys, including the meaning of all questions and response choices. We selected this one-hour window prior to the start of this study in contrast to other intervals (e.g., 15 minutes) because of findings in previous studies that participants commonly engaged in only one activity in the past hour (Strassnig et al., 2021b) and were also either home or away for the entire past hour in 85% of the surveys returned (Granholm et al., 2020; Parrish et al., 2020).

The first question in each EMA survey asked participants whether they were home or away, and the second asked whether they were alone or with someone else and with whom. A customized survey followed, tailored to activities performed at home, with or without another person, or away from home. The first assessment of the day set the timeframe as "today," and the subsequent two surveys asked about "Since the last survey ". At each time point, participants were asked to report their mood (sad, happy, relaxed, anxious) and positive symptoms of schizophrenia (hearing voices, special powers, receiving messages, mind-reading [including having one's mind read and reading others' minds), and paranoid ideas. The items were self-reported on a 7-point (1-7) scale, with one being not at all and seven, extreme for both moods and symptoms. In the interim pre-COVID sample (Harvey et al., 2021), EMA-based ratings of psychosis were significantly correlated with clinical ratings generated with the Positive and Negative Syndrome Scale (PANSS; Kay et al., 1987). There was also no reactive effect of EMA surveys upon clinical severity, with EMA psychosis items remaining stable over up to 90 surveys over 30 days. We focused on the psychotic symptoms only, having reported on mood states previously (Dalkner et al., in press; [full sample] Jones et al., 2021; [Pre-COVID sample]). We aggregated and averaged the frequency of occurrence of the delusion domains other than paranoia at each EMA survey into an ‘other delusions’ variable so that we ended up analyzing three psychosis variables: paranoia, hearing voices, and other delusions, collected at each EMA survey

2.4. Neurocognitive performance and introspective accuracy probes.

A cognitive, clinical, and functional assessment was performed once, at the end of the 30-day EMA period. The cognitive tests were tasks from the MATRICS Consensus Cognitive Battery (MCCB; Nuechterlein et al., 2008) assessing psychomotor processing speed (Trail Making Test Part A, Symbol Coding), semantic fluency (Animal Naming Test), auditory working memory (Letter-Number Sequencing), and verbal learning and memory (Hopkins Verbal Learning Test). Symbol coding was not administered to the remotely assessed participants and is not included in the current analyses. We asked subjects to rate their performance on a predefined scale indexed directly to the metric of the test immediately after completing the test (to measure immediate judgments). Participants were provided with a realistic range of possible performance for each test, such as the Hopkins Verbal Learning Test: "There are 36 possible correct responses. How many did you get correct? " Trail Making Test Part A IA Question: How many seconds did it take you to complete the task? (up to 300 seconds).

To have easily interpretable scores, we created z-scores in the overall sample of participants with schizophrenia and bipolar disorder for each of the cognitive performance variables and averaged them into a single composite score representing cognitive performance. For introspective accuracy, we took the raw-score difference between self-reported performance and actual performance on each test and subtracted the differences such that higher scores reflected over-estimation compared to actual performance. We then created an absolute value score for IA for each cognitive test and then, like for cognitive performance, generated standard scores across the two samples and averaged them into a single score. The greater the absolute IA value compared to 0, the larger the bi-directional self-assessment of one's own cognitive abilities (misjudgment).Thus, higher scores on cognitive performance reflect better performance, and positive scores on absolute introspective accuracy reflect greater inaccuracy compared to objective performance. Both scores are standard scores and thus can be compared directly.

2.5. Clinical Assessments

All participants were assessed on the day of their cognitive testing with structured clinical rating scales for depression, Mania, and Psychotic Symptoms.

2.5.1. Clinical ratings of Mood Symptoms.

Participants were rated with the Montgomery-Asberg Depression Rating scale (MADRS; Montgomery and Asberg, 1979 ) and the Young Mania Rating Scale (YMRS; Young et al.,1978 ). We present endpoint scores on these measures and relate these endpoint scores to EMA measures. Raters were trained to high levels of inter-rater reliability and generated ratings while unaware of the results of the EMA surveys.

2.5.2. Schizophrenia-Related Symptoms.

Severity of symptoms was evaluated with the Positive and Negative Syndrome Scale (PANSS; Kay et al., 1987), which was administered in its entirety by trained raters and on the same day as the cognitive assessments. These raters had extensive experience in other studies of participants with severe mental illness and were trained to high reliability (ICC>.80) by the study PI (Pinkham). The PANSS consists of 30 items and each item was scored on a 7-point Likert scale ranging from 1 to 7:

2.5.3. Negative Symptom Models.

Khan and colleagues (2017) generated a two-factor model of negative symptoms measured by the PANSS, identifying dimensions of expressive deficits and experiential deficits. This model is clinically relevant as the reduced emotional experience factor has been shown to predict variance in everyday functioning in very large samples (Harvey et al., 2017). The items in the PANSS Reduced Emotional Experience factor are: Emotional Withdrawal (N2), Passive/Apathetic Social Withdrawal (N4) and Active Social Avoidance (G16).

2.6. Data Analyses.

Data analyses were performed using SPSS edition 28 (IBM Corporation, 2021). Our first analysis compared performance and introspective accuracy data for cognition. We used t-tests to compare the participants on individual and composite cognitive performance and test by test introspective bias. We also compared the absolute value IA scores for the individual tests and the composite cognitive performance scores. We also tested the differences of absolute value IA compared to a score of zero, such that a significant result would reflect overall significant self-assessment bias. We then correlated task performance with self-assessed task performance and their difference with Pearson Correlations in each diagnostic group separately.

To perform a sophisticated examination of all the symptom data and its association with self-assessment variables, we used mixed-effects hierarchical linear modeling (HLM). We examined the time course of three psychosis variables to see if there was within-subjects variance in the three variables over time. We created an aggregate score for each of the symptoms for each participant and subtracted each individual EMA observation from that mean, creating a variable that indexed the dispersion of each EMA data point from the overall mean for the participant. If there were time effects, we planned to examine the association of the occurrence of these symptoms over the 90 assessments with endpoint cognitive performance and self-assessments, by using the momentary occurrence of the three different psychotic symptoms over up to 90 EMA surveys to predict endpoint cognitive performance and IA scores. If there were no time effects, we planned to create an aggregate frequency score for the occurrence of these symptoms within individuals and use correlational analyses to address the association of the symptom variables and the cognitive and self-assessment variables.

There were two critical dependent variables for HLM: composite cognition and absolute value IA. For the time effects analyses, we entered day (1–30) and survey (1–3) as the repeated measures (Sampled at each survey) in addition to the present-absent occurrence of each of the three psychosis symptoms, entering subject as a random intercept. We used the omnibus fit test for each model to determine that the fitted model improved on the intercept-only model. Gender and age were entered as fixed factors.

Cases that did not complete the endpoint clinical assessments were excluded, and all other cases were examined. Missing survey data for cases who completed the endpoint assessments were addressed by using full-information maximum likelihood procedures.

3.0. Results

There were 21,019 surveys sent, and 15,423 answered (73%) with all the needed information. Table 1 gives the demographic information for the sample. There were diagnostic differences in age, educational attainment, and mothers’ education; participants with bipolar disorder were younger and had higher scores on the education variables. Indicators of disability also were more substantial in the participants with schizophrenia and all three PANSS variables were more severe in this sample as well. No differences were seen in MADRS scores and the participants with bipolar disorder had higher YMRS scores, although their scores were in the remitted range other than for 1 participant.

Supplemental Table 1 presents medication information on participants. Given the nature of our study, we could not confirm whether participants were adherent to the medications that they were prescribed and did not perform any additional analyses on those data.

The cognitive and self-report scores of the participants collected at the end of the EMA period are presented in Table 2. Participants with bipolar disorder had better cognitive performance than participants with schizophrenia on all tasks and on the composite. There was only one difference in self-reported performance, however, with the participants with bipolar disorder reporting that they performed better on letter-number sequencing although that difference would not survive correction for multiplicity. There were no differences in absolute value IA across the samples. The absolute IA score was significantly greater than 0 for both participants with schizophrenia, t(125)=12.98, p<.001, d=1.10 ,and bipolar disorder, t(113)= 14.85, p<.001, d=1.41, reflecting a statistically significant and large effect size mis-estimation of performance. Mean MCCB normative t scores for performance are also presented for reference purposes each of the variables in each group. Participants with schizophrenia were on average 1.0 SD below HC norms and participants with bipolar disorder were 0.5 SD below. In a large sample study (n=2616) of participants participating in controlled studies of pharmacological cognitive enhancement (Georgiades et al, 2017), the MCCB converted t scores for performance on trail-making, letter-number sequencing, animal naming and HVLT were 37.6, 37.3, 40.4, and 36.2. It is not surprising to see slightly better cognitive performance for participants in a complicated study such as this with required adherence and multiple assessments.

Table 2.

Actual and Self-Reported Cognitive Test Performance, Introspective Accuracy, and Momentary Clinical Symptoms.

Schizophrenia
(n = 126)
 Bipolar Disorder
(n = 114)
Test M SD MCCB
T-score		 M SD MCCB
T-score		 t (p), d
TMT A
 Actual 33.78 12.65 40.8 31.29 11.88 46.7 2.10 (.037), 0.27
 Self-reported 21.89 17.73 21.10 16.61 1.20 (.20), 0.17
LNS
 Actual 12.51 3.90 39.2 13.75 3.91 41.6 2.70 (.007), 0.35
 Self-reported 13.61 5.70 15.10 5.18 1.99 (.048), 0.26
ANT
 Actual 21.08 5.79 44.7 23.60 5.70 48.9 3.50 (< .001), 0.46
 Self-reported 18.94 8.41 19.33 7.55 0.43 (.34), 0.06
HVLT
 Actual 20.82 5.60 36.7 23.67 6.24 42.9 3.81 (< .001), 0.50
 Self-reported 19.47 8.12 21.19 8.87 1.39 (.09), 0.18
Absolute Value IA (Z) .85 .37 .83 .31 1.64 (.10), 0.36
Composite Cognition (Z) −.14 .65 40.4 .20 .66 45.0 4.17 (< .001), 0.98
Momentary Clinical Symptom
Prevalence Across Surveys (%)		 Schizophrenia
(n = 126)		 Bipolar Disorder
(n = 114)
 Hallucinations 42% 8%
 Paranoia 52% 18%
 Other delusions 42% 11%
Open in a new tab

Note. MCCB = MATRICS Consensus Cognitive Battery, TMT A = Trail Making Test Part A, LNS = Maryland Letter-Number Sequencing Test, ANT = Animal Naming Test, HVLT = Hopkins Verbal Learning Test, IA = Introspective accuracy.

To understand the importance of the demographic variables that differed between the samples, we computed Pearson Product moment correlations within each sample between age, years of education, mothers’ years of education, and the two critical outcomes variables: composite cognitive performance and absolute value IA. For the participants with schizophrenia, younger age and more education were correlated with better scores on composite cognition (r’s=−.25 and .34, p<.002). Mothers’ education was not correlated with composite cognitive performance, (r=.049, p=.63). Reduced emotional experience, but none of the other clinical or mood variables, manifested a nominally significant correlation with composite cognitive performance, r=−.19, p=.035. None of the demographic variables correlated with absolute introspective accuracy. For the participants with bipolar disorder, the findings were essentially identical. Younger age and more education were correlated with better scores on composite cognition (r’s=−.39 and .38, p<.001), Mothers’ education was not correlated with composite cognitive performance, (r=.12, p=.11). Reduced emotional experience, but none of the other clinical variables, manifested a nominal correlation with composite cognitive performance, r=.23, p=.025. None of the demographic variables correlated with absolute IA.

When the two demographic variables that correlated with composite cognitive performance were entered as covariates in an analysis of covariance testing diagnosis on composite cognitive performance, the effect of diagnosis was still significant, F(1,238)= 12.69, p<.001,η2=.07. However, the covariate effects of both age, F(1,238)=25.04, p<.001,η2=.13 and education, F(1,238)=31.66, p<.001, η2=.16, were statistically significant. For the reduced emotional experience PANSS variable, there was also a significant covariate effect, F(1,238)=10.88, p<.001, η2=.05, which did not eliminate the effects of diagnosis on composite cognitive performance, F(1,238)=10.96, p<.001, η2=.05.

The percentage of answered surveys endorsing the presence of momentary psychotic experiences is also presented in Table 2. As can be seen in the table, the participants with bipolar disorder endorsed considerably fewer surveys for all three symptom domains than the participants with schizophrenia. No predictive analyses on participants with bipolar disorder were performed as a result.

Table 3 presents the results of correlations between performance and self-reported performance. For both groups, self-reports of performance were correlated with objective performance for 7/8 correlations. Better objective performance was negatively correlated with greater misestimation for 7/8 variables as well.

Table 3.

Pearson Correlations of Cognitive Performance and Self-reported Performance Across Participants with Schizophrenia or Bipolar Disorder.

Self-Reported Performance
Schizophrenia (n = 126)
 Bipolar Disorder (n = 114)
Objective
Performance		 Raw Score Difference Score Raw Score Difference
Score
r p r p r p r p
TMT A* .35 < .001 −.13 .17 .32 < .001 −.39 < .001
LNS .13 .17 −.51 < .001 .22 .01 −.48 < .001
ANT .45 < .001 −.27 .002 .42 < .001 −.34 < .001
HVLT .47 < .001 −.25 .005 .72 < .001 −.07 .51
Open in a new tab

Note.

*

Scoring reversed for correlation. TMT A = Trail Making Test Part A, LNS = Maryland Letter-Number Sequencing Test, ANT = Animal Naming Test, HVLT = Hopkins Verbal Learning Test.

Table 4 presents the results of the analyses of time effects on the within-subjects symptom variables in participants with schizophrenia. As can be seen in the table, all three analyses had significant overall omnibus effects and there were significant effects of survey day, but not time of day, on the within person variance on all three domains of symptoms. Thus, within-person variability in psychotic symptoms over the 30-day period is confirmed. There was evidence of significant kurtosis, with scores skewed toward to the positive. In order to ensure that the composite “other delusions” was reliable because the individual items were reduced in prevalence, we used the VARCOMP procedure in SPSS 28.0 to estimate the between-person reliability in that variable. The resulting reliability coefficient (referred to as Rkf ) uses generalizability theory to estimate the reliability of a composite variable (consisting of m items; (in our case, 3 items) averaged across a number of k time points (in our case, 90 time points).

Table 4.

Within-Person Variance of the Psychosis Variables Over Time in Participants with Schizophrenia.

Symptom
Voices
 Paranoia
 Other Delusions
χ 2 df p χ 2 df p χ 2 df p
Omnibus 40.42 31 < .001 77.70 31 < .001 94.00 31 < .001
Intercept 0.16 1 .67 0.17 1 .66 0.21 1 .64
Day 68.42 29 < .001 79.63 29 < .001 91.70 29 < .001
Time of Day 2.31 2 .32 1.05 2 .59 2.63 2 .27
Variance in Individual EMA Symptom Reports Centered Around Participant Means.
Symptom M SD Range Kurtosis
Voices 0.00 0.98 −5.21 ↔ 5.94 5.58
Paranoia 0.00 1.12 −5.54 ↔ 5.81 4.80
Other Delusions 0.00 1.90 −12.73 ↔ 17.28 10.04
Open in a new tab

The equation is: (person variance + (person*item variance/m )/ (person variance + (person*item variance/m ) + (error variance/k*m)) and we found that the between-person reliability of the “other delusions” variable was . 9984143 for present-absent scores and 0.9987534 for full-range scores.

Table 5 presents the results for the examination of the occurrences of psychotic symptoms and cognition and absolute value introspective accuracy at the end point assessment. The model predicting the cognition composite did not improve on the fit of the null model, X2 (34)=36.44, p= .64, so no additional analyses are presented. For absolute value IA, all three symptom variables entered the model, and the frequency of occurrence of all three symptoms was associated with greater absolute introspective inaccuracy. Adding cognition to the model as a covariate reduced the p values for all symptom variables and improved the predictive power across indices of overall model fit. We also computed these analyses using severity scores, in contrast to present/absent occurrences from the three domains. The results were essentially identical (see supplementary Table 2), but we present the present absent values because of the ease of interpretation of the EM means. See supplemental figures 2 and 3 for scatterplots of the relationships between absolute value IA, the cognitive composite score, and the three types of psychotic symptoms.

Table 5.

Prediction of Cognitive Performance and Absolute Value Introspective Accuracy by the Longitudinal Occurrence of Psychotic Symptoms Measured with EMA.

Absolute Value IA Absolute Value IA Adjusted for
Cognition
Variable X 2 df p X 2 df p
Omnibus 201.32 34 < .001 489.92 34 < .001
Intercept 12595.70 1 < .001 11658.74 1 < .001
Hallucinations 38.04 1 < .001 44.30 1 < .001
Paranoia 90.18 1 < .001 49.85 1 < .001
Other Delusions 187.03 1 < .001 161.37 1 < .001
Composite Cognition - - - 293.72 1 < .001
Adjusted Means for Absolute Value IA.
EM Mean SEM
Hallucinations Y 0.51 0.01
N 0.45 0.01
Paranoia Y 0.51 0.01
N 0.44 0.01
Other Delusions Y 0.55 0.01
N 0.41 0.01
Open in a new tab

Note. IA = Introspective accuracy.

4.0. Discussion

Our study found that participants with schizophrenia had more frequent momentary occurrences of psychotic symptoms and poorer overall cognitive performance than participants with bipolar disorder. Both sets of participants manifested significant absolute introspective inaccuracy, with the magnitude of absolute inaccuracy not differing across the participant groups. Momentary psychotic symptoms in the participants with schizophrenia were found to vary over the 30-day, 90 survey EMA assessment period, but the symptoms did not predict cognitive performance. Momentary psychotic symptoms were not frequent enough to allow for analyses in the participants with bipolar disorder indicating that they could not be the origin of the IA deficits.

More frequently experiences three different psychotic symptoms, including two groups of delusions and auditory hallucinations, was associated with greater absolute errors in immediate self-assessment of neuropsychological test performance in participants with schizophrenia. Partially in line with our predictions, the more frequent momentary experience of delusional symptoms and paranoid ideation was associated with greater absolute errors in immediate self-assessment of cognitive performance. These findings are consistent with previous hypotheses regarding challenges in self-assessment and the development and maintenance of delusions. While we did not find that paranoid symptoms were more strongly associated with challenges in IA than other delusions, these symptom ratings were self-generated by participants with schizophrenia and not by trained raters. Hence, differentiation between different types of delusional content may not have the same validity as ratings generated on the basis of a structured interview. More common momentary experiences of hallucinations were also significantly associated with challenges in introspective accuracy. As expected, poorer cognitive performance exerted a significant covariate effect on absolute introspective accuracy but did not degrade the fit statistics for the influence of momentary psychotic symptoms. Thus, across zero order correlation analyses in both participant groups suggested that better cognitive performance was associated with better introspective accuracy, but poor cognitive performance did not fully explain IA impairments in participants with schizophrenia when the momentary occurrence of psychotic symptoms was considered.

There is very little previous data on immediate self-assessment of cognitive test performance and even less information on what the correlates of mis-estimation may be. Most previous studies on cognitive self-assessment accuracy used global self-report ratings of cognitive performance (Burton et al., 2016; Medalia and Thysen, 2008) and related them to test performance collected at some prior time.

A final important implication is that challenges in introspective accuracy may arise from different sources in bipolar disorder and schizophrenia. Although momentary inaccuracy was equivalent in the two samples, the low level of psychosis in the bipolar participants suggests that this could not be the origin of IA challenges in that group. In an analysis of the current data set focusing on emotional experiences (Dalkner et al., in press) we found that the longitudinal momentary experience of greater negative affect was associated with greater challenges in introspective accuracy in participants with bipolar disorder, but not in participants with schizophrenia. This finding was like a previous cross-sectional study where we found that participants with bipolar depression manifested a strong correlation between higher levels of self-reports of depression and underestimation of their cognitive and functional abilities compared both to objective data and high contact informant reports (Harvey et al., 2015). In another previous study (Tercero et al., 2021) bipolar and schizophrenia participants had similar levels of momentary overestimation of the accuracy of their responses while performing the Wisconsin card sorting test. However, the bipolar participants in that study overcame their momentary overestimation and generated global summary self-assessments of task performance that were strongly related to performance and not momentary response biases on a sort x sort basis.

There are several limitations to the study. The reduced frequency of occurrence of psychosis in participants with bipolar disorder did not allow for diagnostic comparisons of the association of psychosis with introspective accuracy or bias. Specifically recruiting bipolar disorder with psychosis would be helpful. Not all participants completed the endpoint cognitive assessment, which reduced the sample size. As expected, and previously reported, there are racial, ethnic, sex, and disability differences across the diagnostic groups, which are related to but do not fully account for differences in cognitive performance. Finally, although a healthy comparison group would be helpful for a normative understanding of misestimation of cognitive performance, such a group would not be helpful for the prediction of misestimation by momentary experiences of psychosis or even momentary variance in mood symptoms (Granholm et al., 2020).

This sample of participants also completed repeated momentary assessments of their verbal learning performance (the Mobile Variable Difficulty List Memory Test; VLMT Parrish et al., 2021) three times per week over the 30-day EMA period, accompanied by a detailed momentary examination of introspective accuracy and bias. These data were not reported here and thus, a true momentary examination of cognitive performance and self-assessment biases will be forthcoming. The full array of momentary performance and self-assessment variables will be captured along with momentary data regarding social context, mood states, psychosis, and these endpoint cognitive and IA assessments. At that time, we will be able to expand our understanding of the accuracy of momentary self-assessment of cognitive performance.

Supplementary Material

1

Acknowledgments.

All authors who contributed to this paper are listed as authors. No professional medical writer was involved in any portion of the preparation of the manuscript.

This research was supported by NIMH grant RO1MH112620 to Dr. Pinkham.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of interest statement

Dr. Raeanne C. Moore is a co-founder of KeyWise AI, Inc. and a consultant for NeuroUX. Dr. Harvey has received consulting fees or travel reimbursements from Alkermes, Bio Excel, Boehringer Ingelheim, Minerva Pharma, and Sunovion Pharma. He receives royalties from the Brief Assessment of Cognition in Schizophrenia, which is owned by WCG-Verasci and is part of the MATRICS Consensus Battery. He is chief scientific officer of iFunction, Inc. Dr. Pinkham has served as a consultant for Roche Pharma. The other authors have no potential Biomedical Conflicts of Interest.

The data in this study are being deposited in the NIMH RDOC repository. 6 Months after data lock they will be available for public access. In the interim, the authors are happy to share the data that underlie this paper upon request.

5.0 References

  1. Amador XF, Strauss DH, Yale SA, Flaum MM, Endicott J, Gorman JM 1993. Assessment of insight in psychosis. Am J Psychiatry. 150, 873–9. 10.1176/ajp.150.6.873 [DOI] [PubMed] [Google Scholar]
  2. American Psychiatric Association. 2013. Diagnostic and statistical manual of mental disorders: 5th Edition. Washington, DC: Author [Google Scholar]
  3. Arts B, Jabben N, Kraddebdam L, Van Os J 2008. Meta- analyses of cognitive functioning in euthymic bipolar patients and their first- degree relatives. Psychol. Med 38, 781–785. 10.1017/S0033291707001675 [DOI] [PubMed] [Google Scholar]
  4. Bonnín CM, Martínez-Arán A, Torrent C, et al. 2010. Clinical and neurocognitive predictors of functional outcome in bipolar euthymic patients: a long-term, follow-up study. J Affect Disord. 121, 156–60. 10.1016/j.jad.2009.05.014 [DOI] [PubMed] [Google Scholar]
  5. Bora E 2017. A comparative meta-analysis of neurocognition in first-degree relatives of patients with schizophrenia and bipolar disorder. Eur. psychiatry 45, 121–128. 10.1016/j.eurpsy.2017.06.003 [DOI] [PubMed] [Google Scholar]
  6. Bora E, Yucel M, Pantelis C. 2009.Cognitive endophenotypes of bipolar disorder: a meta-analysis of neuropsychological deficits in euthymic patients and their first-degree relatives. J Affect Disord. ; 113: 1–20. [DOI] [PubMed] [Google Scholar]
  7. Bowie CR, Depp C, McGrath JA, et al. 2010. Prediction of real-world functional disability in chronic mental disorders: a comparison of schizophrenia and bipolar disorder. Am J Psychiatry. 167, 1116–24. 10.1176/appi.ajp.2010.09101406 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bowie CR, Twamley EW, Anderson H, Halpern B, Patterson TL, Harvey PD 2007. Self-assessment of functional status in schizophrenia. J. Psychiatric Res 41(12), 1012–1018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Burton CZ, Harvey PD, Patterson TL, Twamley EW 2016. Neurocognitive insight and objective cognitive functioning in schizophrenia. Schizophr. Res, 777(1-3), 131–136. 10.1016/j.schres.2016.01.021 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Dalkner N, Moore RC, Depp CA, et al. In press. Negative Mood States as a Correlate of Cognitive Performance and Self-assessment of Cognitive Performance in Bipolar disorder versus Schizophrenia. Schizophr. Res, 2022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Depp CA, Mausbach BT, Harmell AL, et al. 2012. Meta-analysis of the association between cognitive abilities and everyday functioning in bipolar disorder. Bipol. Disord 14(3), 217–226. 10.1111/j.1399-5618.2012.01011.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Durand D, Strassnig MT, Moore RC, et al. 2021. Self-reported social functioning and social cognition in schizophrenia and bipolar disorder: Using ecological momentary assessment to identify the origin of bias. Schizophr. Res, 230, 17–23. 10.1016/j.schres.2021.02.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Fett AK, Viechtbauer W, Dominguez MD, Penn DL, Van Os J, Krabbendam L 2011. The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: a meta-analysis. Neurosci Biobehav Rev. 35. 573–88. 10.1016/j.neubiorev.2010.07.001 [DOI] [PubMed] [Google Scholar]
  14. Frank E, Nimgaonkar VL, Phillips ML, Kupfer DJ, 2015. All the world's a (clinical) stage: rethinking bipolar disorder from a longitudinal perspective. Mol. Psychiatr 20. 23–31. 10.1038/mp.2014.71 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Georgiades A, Davis VG, Atkins AS, et al. 2017. Psychometric characteristics of the MATRICS Consensus Cognitive Battery in a large pooled cohort of stable schizophrenia patients. Schizophr Res.; 190:172–179. doi: 10.1016/j.schres.2017.03.040 [DOI] [PubMed] [Google Scholar]
  16. Gohari E, Moore RC, Depp CA, Ackerman RA, Pinkham AE, Harvey PD 2022. Momentary severity of psychotic symptoms predicts overestimation of competence in domains of everyday activities and work in schizophrenia: An ecological momentary assessment study. Psychiatry Res. , 310, 114487. 10.1016/j.psychres.2022.114487 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Gold JM, Goldberg RW, McNary SW, Dixon LB, Lehman AF 2002. Cognitive correlates of job tenure among patients with severe mental illness. Am J Psychiatry. 159. 1395–402. 10.1176/appi.ajp.159.8.1395. [DOI] [PubMed] [Google Scholar]
  18. Gould F, Sabbag S, Durand D, Patterson TL, Harvey PD 2013. Self-assessment of functional ability in schizophrenia: Milestone achievement and its relationship to accuracy of self-evaluation. Psychiatry Res. 207. 19–24, 10.1016/j.psychres.2013.02.035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Gould F, McGuire LS, Durand D, et al. 2015. Self-assessment in schizophrenia: Accuracy of evaluation of cognition and everyday functioning. Neuropsychology. 29. 675–682. 10.1037/neu0000175. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Granholm E, Holden JL, Mikhael T, L., et al. 2020. What Do People With Schizophrenia Do All Day? Ecological Momentary Assessment of Real-World Functioning in Schizophrenia. Schizophr Bull. 46. 242–251. 10.1093/schbul/sbz070 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Harvey PD, Aslan M, Du M, et al. 2016. Factor structure of cognition and functional capacity in two studies of schizophrenia and bipolar disorder: Implications for genomic studies. Neuropsychology, 30(1), 28–39. 10.1037/neu0000245 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Harvey PD, Paschall G, Depp C 2015. Factors influencing self-assessment of cognition and functioning in bipolar disorder: a preliminary study. Cogn Neuropsychiatry. 2015;20(4):361–371. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Harvey PD, Miller ML, Moore RC, Depp CA, Parrish EM, Pinkham AE 2021. Capturing Clinical Symptoms with Ecological Momentary Assessment: Convergence of Momentary Reports of Psychotic and Mood Symptoms with Diagnoses and Standard Clinical Assessments. Innovations in clinical neuroscience, 75(1-3), 24–30. [PMC free article] [PubMed] [Google Scholar]
  24. Harvey PD and Pinkham A 2015. Impaired self-Assessment in schizophrenia: why patients misjudge their cognition and functioning. Curr Psychiatr 14. 53–59. [Google Scholar]
  25. Harvey PD, Sun N, Bigdeli TB, et al. 2020. .Genome-wide association study of cognitive performance in U.S. veterans with schizophrenia or bipolar disorder. Am J Med Genet B Neuropsychiatr Genet. 183. 181–194. 10.1002/ajmg.b.32775 [DOI] [PubMed] [Google Scholar]
  26. Harvey PD, Twamley EW, Pinkham AE, Depp CA, Patterson TL 2017. Depression in Schizophrenia: Associations with Cognition, Functional Capacity, Everyday Functioning, and Self-Assessment. Schizophr Bui. 43. 575–582. 10.1093/schbul/sbw103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Harvey PD, Wingo AP, Burdick KE, Baldessarini RJ 2010. Cognition and disability in bipolar disorder: lessons from schizophrenia research. Bipol. Disord 12(4), 364–375. 10.1111/j.1399-5618.2010.00831.x [DOI] [PubMed] [Google Scholar]
  28. Hochberger WC, Hill SK, Nelson CL, et al. 2016. Unitary construct of generalized cognitive ability underlying BACS performance across psychotic disorders and in their first-degree relatives. Schizophr. Res, 170(1), 156–161. 10.1016/j.schres.2015.11.022 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Jastak S, 1993. Wide-Range Achievement Test, 3rd ed. San Antonio, TX, Wide Range, Inc. [Google Scholar]
  30. Jones MT, Deckler E, Laurrari C, Jarskog LF, Penn DL, Pinkham AE, Harvey PD, 2019. Confidence, performance, and accuracy of self-assessment of social cognition: A comparison of schizophrenia patients and healthy controls. Schizophr Res Cogn., 19. 002–2. 10.1016/j.scog.2019.01.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Jones SE, Moore RC, Depp CA, Ackerman RA, Pinkham AE, Harvey PD 2021. Daily ecological momentary assessments of happy and sad moods in people with schizophrenia and bipolar disorders: What do participants who are never sad think about their activities and abilities?, Schizophr Res Cogn. 26. 10.1016/j.scog.2021.100202. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Kay SR, Fiszbein A, Opler LA 1987. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull. 13. 261–276. 10.1093/schbul/13.2.261. [DOI] [PubMed] [Google Scholar]
  33. Khan A, Liharska L, Harvey PD, Atkins A, Ulshen D, Keefe RSE 2017. Negative Symptom Dimensions of the Positive and Negative Syndrome Scale Across Geographical Regions: Implications for Social, Linguistic, and Cultural Consistency. Innov. Clin. Neurosci 14(11-12), 30–40. [PMC free article] [PubMed] [Google Scholar]
  34. Keefe RS, Bilder RM, Harvey PD, et al. 2006. Baseline neurocognitive deficits in the CATIE schizophrenia trial. Neuropsychopharmacology;31(9):2033–2046. doi: 10.1038/sj.npp.1301072 [DOI] [PubMed] [Google Scholar]
  35. Mann-Wrobel MC, Carreno JT, Dickinson D 2011. Meta-analysis of neuropsychological functioning in euthymic bipolar disorder: an update and investigation of moderator variables. Bipolar Disord. 13. 334–42. 10.1111/j.1399-5618.2011.00935.x. [DOI] [PubMed] [Google Scholar]
  36. Medalia A, Thysen J 2008. Insight into neurocognitive dysfunction in schizophrenia. Schizophr Bull., 34(6), 1221–1230. 10.1093/schbul/sbm144 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Montgomery SA, Asberg M, 1979. A new depression scale designed to be sensitive to change. Br. J. Psychiatry 134, 382–389. [DOI] [PubMed] [Google Scholar]
  38. Moritz S, Göritz AS, Gallinat J,et al. ., 2015. Subjective competence breeds overconfidence in errors in psychosis. A hubris account of paranoia. J. Behav. Ther. Exper. Psychiat, 48, 118–124 [DOI] [PubMed] [Google Scholar]
  39. Moritz S, Ramdani N, Klass H, et al. 2014. Overconfidence in incorrect perceptual judgments in participants with schizophrenia. Schizophr. Res.: Cognition, 1(4), 165–170. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Nuechterlein KH, Green MF, Kern RS, et al. 2008. The MATRICS Consensus Cognitive Battery, part 1: test selection, reliability, and validity. Am J Psychiatry. 165. 203–13. 10.1176/appi.ajp.2007.07010042. [DOI] [PubMed] [Google Scholar]
  41. Parrish EM, Depp CA, Moore RC, et al. 2020. Emotional determinants of life-space through GPS and ecological momentary assessment in schizophrenia: What gets people out of the house? Schizophr Res. 224. 67–73. 10.1016/j.schres.2020.10.002. [DOI] [PubMed] [Google Scholar]
  42. Parrish EM, Kamarsu S, Harvey PD, Pinkham A, Depp CA, Moore RC. 2021. Remote Ecological Momentary Testing of Learning and Memory in Adults With Serious Mental Illness. Schizophr Bull. 2021;47(3):740–750. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Reichenberg A, Harvey PD, Bowie CR, et al. 2009. Neuropsychological Function and Dysfunction in Schizophrenia and Psychotic Affective Disorders, Schizophr Bui. 35. 1022–1029. 10.1093/schbul/sbn044 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Silberstein JM, Pinkham AE, Penn DL, Harvey PD, 2018. Self-assessment of social cognitive ability in schizophrenia: Association with social cognitive test performance, informant assessments of social cognitive ability, and everyday outcomes. Schizophr. Res, 199. 75–82. 10.1016/j.schres.2018.04.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Strassnig M, Kotov R, Fochtmann L, Kalin M, Bromet EJ, Harvey PD 2018. Associations of independent living and labor force participation with impairment indicators in schizophrenia and bipolar disorder at 20-year follow-up. Schizophr. Res, 197, 150–155. 10.1016/j.schres.2018.02.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Strassnig MT, Miller ML, Moore R, Depp CA, Pinkham AE, Harvey PD 2021. Evidence for avolition in bipolar disorder? A 30-day ecological momentary assessment comparison of daily activities in bipolar disorder and schizophrenia. Psychiatry Res., 300, 113924. 10.1016/j.psychres.2021.113924 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Tabarés-Seisdedos R, Balanzá-Martínez V, Sánchez-Moreno J, et al. 2008. Neurocognitive and clinical predictors of functional outcome in patients with schizophrenia and bipolar I disorder at one-year follow-up. J Affect Disord. 109. 286–99. 10.1016/j.jad.2007.12.234. [DOI] [PubMed] [Google Scholar]
  48. Tercero BA, Perez MM, Mohsin N, et al. 2021. Using a Meta-cognitive Wisconsin Card Sorting Test to measure introspective accuracy and biases in schizophrenia and bipolar disorder. J Psychiatr Res. 140. 436–442. 10.1016/j.jpsychires.2021.06.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Young RC, Biggs JT, Ziegler VE, Meyer DA 1978. A rating scale for mania: reliability, validity and sensitivity. Br. J Psychiatry ;133:429–35. doi: 10.1192/bjp.133.5.429. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1849244-supplement-1.docx (402.1KB, docx)

RESOURCES