Self-Assessment of Functional Ability in Schizophrenia: Milestone Achievement and Its Relationship to Accuracy of Self-Evaluation

Abstract

Between 50–80% of patients with schizophrenia do not believe they have any illness and self assessment of cognitive impairments and functional abilities is also impaired compared to other information, including informant reports and scores on performance-based ability measures. The present paper explores self- assessment accuracy in reference to real world functioning as measured by milestone achievement such as employment and independent living. Our sample included 195 people with schizophrenia examined with a performance-based assessment of neurocognitive abilities and functional capacity. We compared patient self-assessments across achievement of milestones, using patient performance on cognitive and functional capacity measures as a reference point. Performance on measures of functional capacity and cognition was better in people who had achieved employment and residential milestones. Patients with current employment and independence in residence rated themselves as more capable than those who were currently unemployed or not independent. However, individuals who had never had a job rated themselves as at least as capable as those who had been previously employed. These data suggest that lifetime failure to achieve functional milestones is associated with overestimation of abilities. As many patients with schizophrenia never achieve milestones, their self-assessment may be overly optimistic as a result

1. Introduction

Schizophrenia is an illness characterized by four domains of dysfunction: positive symptoms, negative symptoms, cognitive impairment, and affective symptoms. These four domains have been found to predict deficits in psychosocial and occupational functioning in schizophrenia (Bowie et al., 2008). Studies have suggested that negative symptoms have a greater impact on real-world functioning than other symptoms (Rabinowitz et al., 2012) and that depression is associated with impairments in the performance of everyday functional skills independent from the influence of cognition and functional capacity (Sabbag et al., 2012). Patients with schizophrenia have been shown to lack awareness of impairments associated with illness in several areas, including symptoms (Amador et, al., 1994), cognitive abilities (Medalia and Thysen, 2008), functional capacity (Sabbag et al., 2011), and everyday functioning (Bowie et al., 2007), largely through a tendency to underestimate the significance and severity of symptoms and to overestimate their functional abilities and current everyday functioning.

Poor insight is a core feature of schizophrenia and includes reduced awareness of having a mental disorder, the need for treatment, understanding the consequences of the illness and the attribution of symptoms to the disorder. Between 50–80% of patients with schizophrenia do not believe they have any illness or impairments (Amador and Gorman, 1998). There are multiple strategies previously employed for assessing real-world functioning and its correlates, including rating scales completed by informants and patients (Leifker et al., 2011), direct observations by trained clinicians (Kleinman et al., 2009), and performance-based measures aimed at the ability to perform critical everyday skills (Harvey et al., 2007). Studies have shown that self-reports of cognition and everyday functioning in schizophrenia often do not converge with objective evidence, obtained both from performance-based assessments (Bowie et al., 2007; Keefe et al. 2006; Sabbag et al., 2012) or the reports of other evaluators (Keefe et al., 2006; McKibbin et al., 2004; Sabbag et al., 2011). Studies have shown that unawareness of cognitive deficits (Medalia and Thysen, 2008; 2010) is also common. In a recent investigation, we (Sabbag et al., 2011), found that the clinician ratings of the severity of real-world impairment were more strongly correlated with performance-based data relevant to outcomes than impairment ratings generated by friends, relatives, or the patients themselves, suggesting that the characteristics of the specific observer is important as well.

Impaired accuracy of self-assessment is not limited to persons with severe mental illness. The accuracy of self-assessment in both clinical populations and healthy individuals appears to be limited. Healthy individuals tend to consistently overestimate their abilities, with poor performers having a particularly positive bias (Dunning and Story, 1991; Ehrlinger et al., 2008). In contrast, otherwise healthy individuals with mild depressive symptoms tend to be more accurate in their self- assessments (Alloy and Abramson, 1979), with moderate to severe depression associated with underestimation of functioning (Bowie et al., 2007). Recently, we (Sabbag et al., 2012) found that higher levels of depressive symptoms within the moderate ranges in people with schizophrenia were associated with less overestimation of everyday functioning. Clinical ratings of delusions, suspiciousness, grandiosity and poor rapport predicted over- estimation of self-reported functioning compared to interviewer judgments. Such findings mirror those from studies of people with neurological and neuropsychiatric conditions, including bipolar disorder (Burdick et al., 2005), multiple sclerosis (Carone et al., 2005), and traumatic brain injury (Spikman and van der Naalt, 2010). Across all conditions, individuals with poorer neuropsychological (NP) test performance tend to underestimate their impairment.

Everyday functioning can be defined in several ways. The rating scales typically rate the level of success in performance of various skilled acts, such as cleaning, cooking, or financial management. Many of these rating scales do not have items that directly assess milestone achievements, such as marriage and financial responsibility for maintenance of a residence (Harvey et al., 2012). Milestone achievement can be defined across numerous domains. Living without supervision, being financially responsible, obtaining competitive employment, or having a stable relationship comparable to marriage signifies successful outcome. Such milestones have long been an integral part of clinical assessments and conceptions of recovery in severe mental illness (Harvey & Bellack, 2009), but have less often been systematically analyzed in the course of research on real world functioning. In patients with schizophrenia, milestone achievement rates are typically low, particularly when achievement of more than one milestone is examined (Harvey et al., 2012). However, given the relative ease of validly measuring milestones (both their lifetime achievement and sustained maintenance) and their clear validity, when achieved, as indices of successful real world functioning, it is important to evaluate what distinguishes milestone achievers from their peers who do not achieve these milestones on a lifelong or current basis.

We have previously shown, in the current sample (Harvey et al., 2011) and in previous studies with other samples (Bowie et al., 2008; 2010; Mausbach, Bowie et al 2007; Mausbach, Harvey et al 2007; Mausbach et al., 2011) that performance on neuropsychological tests and measures of functional capacity were correlated with both ratings on functional status rating scales and achievement of functional milestones in residential and vocational domains. These data suggest that real-world functional outcomes, assessed with rating scales and indexed by achievement of milestones, have determinants in ability variables as well as environmental and social factors (Rosenheck et al., 2006)

Achieving specific milestones themself may have the potential to impact on accuracy of self-assessment, in that individuals who have had a job in the past or live independently have experience with the demands and challenges associated with achieving these goals (Bryson et al., 2002). Milestone achievers may have information that allows them to be more realistic with respect to self-assessment of their abilities. The current paper expands our research on the accuracy of self-assessment in patients with schizophrenia to the association of self-assessment with milestone achievement, using data from the results of the VALERO study phase 1 (Leifker et al., 2011). In this study, the achievement of functional milestones in domains of residential functioning, social outcomes and productive activities was examined, with a goal of examining the association between achievement of milestones and of self-assessment of real world everyday functioning. We compared patients who had and had not achieved functional milestones on their self-assessments of their everyday functioning on two different everyday functioning rating scales: the Specific Levels of Functioning (SLOF; Schneider and Stuening, 1983) and the Quality of Life Scale (QLS; Heinrichs et al., 1994). Interviewer ratings on subscales were previously shown to be related to milestone achievement in specific functional domains (Harvey et al., 2012). We then used performance on neuropsychological tests and performance-based measures of functional capacity to further compare patients who had and had not achieved milestones, in order to provide an objective reference point for the self-reported levels of competence of the patients.

2. Methods

2.1 Subjects

Study participants were patients with schizophrenia who were receiving treatment at one of three different outpatient service delivery systems, two in Atlanta and one in San Diego. All research participants provided signed, informed consent, and this research study was approved by local IRBs. In Atlanta, patients were either recruited at an intensive psychiatric rehabilitation program (Skyland Trail) or from the general outpatient population of the Atlanta VA Medical Center. The San Diego patients were recruited from the UCSD Outpatient Psychiatric Services clinic, which is a large public mental health clinic, from other local community clinics, and by word of mouth.

All patients with schizophrenia were administered either the Structured Clinical Interview for the DSM-IV (SCID; First et al., 1995: Atlanta sites) or the MINI International Neuropsychiatric Interview (MINI; Sheehan et al., 1998: San Diego) by a trained interviewer. All diagnoses were subjected to a consensus procedure at the local site. Patients were excluded for a history of traumatic brain injury with unconsciousness >10 minutes, brain disease such as seizure disorder or neurodegenerative condition, or the presence of another DSM-IV-TR diagnosis that would exclude the diagnosis of schizophrenia. None of the patients were experiencing their first psychotic episode. Substance abuse was not an exclusion criterion, in order to capture a broad array of patients, but patients who appeared intoxicated were rescheduled. Inpatients were not recruited, but patients resided in a wide array of unsupported, supported, or supervised residential locations. Descriptive information on patients has been previously presented and is contained in online supplemental material (See supplemental table 1).

2.2 Procedure

All patients were examined with a performance-based assessment of neurocognitive abilities and functional capacity which has been reported on previously (Harvey et al., 2011). They also provided self-reports of social, residential, and vocational functioning on six different functional outcome scales, which were either administered to them as interviews by a trained rater or completed in a questionnaire format, depending on the instructions of the scales. Patients received $50.00 for their time and effort.

2.2.1 Real-World Functional Outcomes

The initial phase of the VALERO study included a RAND panel that selected 6 functional outcome scales from a much larger group of candidate scales, as most suitable for current use at the time of the panel (see Leifker et al., 2011 for detailed descriptions of these instruments). Two of these scales, the QLS and the SLOF, had subscales directly targeted at the domains of functioning in which we were interested: vocational, social, and everyday living skills; they are the focus of this report. These instruments were modified by deletion of some of their subscales following the suggestions of the RAND panel. The social acceptability and personal care subscales of the SLOF were excluded because of reduced relevance and potential ceiling effects and for the QLS the intrapsychic foundations subscale was not included in the analyses of the data because it measures deficit (i.e., negative symptoms). The SLOF subscales that we examined were interpersonal relations, work skills, and everyday activities. The QLS subscales were intrapersonal relationships, instrumental role functioning, and objects and activities. See Table 1 for sample items from each subscale for each rating scale.

Table 1.

Sample Items from the Functional Status Rating Scales

2.2.2 Functional Milestone Achievements

We collected information from patients, informants, and medical records on the achievement of various functional milestones. In cases of uncertainty, a consensus was obtained through discussion with the local principal investigator and the interviewer. We recorded details about achievements (e.g., first job, most recent job, best job) in order to increase accuracy of reporting. These milestones included social outcomes such as marriage or an equivalent long-term relationship. For employment, we collected current and lifetime history of supported or competitively obtained employment (either part or full time), regardless of duration or reason for termination. For residential status, we determined whether the individuals were currently living without supervision and whether they were financially responsible for their housing (even if they used disability compensation to pay their bills) using the methods we previously employed (Harvey et al., 2012). These two residential outcomes were examined separately because they are not necessarily overlapping.

2.2.3 Performance-Based Ability Assessments

2.2.3.1 Neurocognition

We examined cognitive performance with a modified version of the MATRICS consensus cognitive battery (MCCB, Nuechterlein et al., 2008). For this study, we did not include the social cognition measure, from the MCCB, the Mayer–Salovey–Caruso Emotional Intelligence Test Managing Emotions, because two separate meta-analyses (Fett et al., 2011 and Ventura et al., in press) demonstrated that social cognition has a clearly different relationship with real-world outcomes in than neurocognition. We calculated a composite score, an average of 9 age-corrected t-scores based on the MCCB normative program, as our critical dependent variable.

2.2.3.2 Functional Capacity

We administered the Brief version of the UCSD Performance-based Skills Assessment (UPSA-B Mausbach, Harvey, et al., 2007). The UPSA-B is a measure of functional capacity in which patients are asked to perform everyday tasks related to communication and finances. During the Communication subtest, participants role-play exercises using an unplugged telephone (e.g., emergency call; dialing a number from memory; calling to reschedule a doctor’s appointment). For the Finance subtest, participants count change, read a utility bill, and write and record a check for the bill. The UPSA-B requires approximately 10–15 minutes, and raw scores are converted into a total score ranging from 0–100, with higher scores indicating better functional capacity.

2.3 Data Analyses

We dichotomized the three different domains of functional milestones, separating the patients into those who had achieved a long-term relationship or not, were currently employed or not, were ever employed or not, were currently living independently or not, and were financially responsible for their dwelling or not. We compared patients who had and had not achieved functional milestones on the specific SLOF and QLS subscales aimed at that functional domain with t-tests (using the Bonferroni correction for multiple comparisons). Finally, we compared performance on the modified MCCB and the UPSA-B across the groups who differed in their milestone achievements as well.

In our previous study (Harvey et al., 2012), we found that milestone achievements were consistently associated with subscale scores in the same domains and that global scores did not predict milestones; there was also no cross-over prediction across domains (ratings of vocational functioning did not predict social or residential outcomes, etc.). Thus, for the sake of clarity and parsimony, we present the comparisons of self-assessments across levels of achievement on only those scales previously found to be associated with achievement of functional milestones.

3. Results

We first compared patients who had and had not achieved each of the 5 functional milestones (ever married, ever employed, currently employed, currently living independently, and currently financially responsible) on age and educational attainment. Of the 10 t-tests, only one was significant, in that patients who had been previously married were older than those who had not (m=48.1, sd=8.6 vs. 40.6, SD= 12.8), t(194)=4.57, p<.001. We also correlated PANSS total, negative, and positive subscale scores with the 6 self-report functional outcomes measures with Pearson correlations. Of the 18 total correlations, 4 were significant at a level that would have exceeded Bonferroni criteria (p<.001). All involved self-reported social functioning and in every case, more severe symptoms were correlated with reduced levels of self-reported social deficit.

Table 2 provides means and standard deviations on the SLOF and QLS subscales tapping social functioning for patients who achieved the milestone of marriage or equivalent and for those who did not. There were no differences in self-assessed social functioning between those patients who had and had not ever experienced a long term relationship; consistent with considerable recent research (Bowie et al., 2007; Leifker et al., 2009), there were no differences in neurocognition or functional capacity either. Table 3 provides means and standard deviations on the SLOF and QLS subscales aimed at vocational functioning and the results of analyses comparing patients who have the milestones of current or previous employment and those who did not. As can be seen in the table, both UPSA and MCCB scores were higher for patients who had achieved either current or lifetime employment. However, patients who had never been employed rated themselves as significant more capable on the QLS instrumental role subscale than those who had had a job at some time in their lives. For the other two functional subscales, there was no difference in self-reported vocational ability between those who had and had not ever had a job. In contract, currently employed patients rated themselves as more capable than those who were not currently employed across all three relevant functional status subscales.

Table 2.

Self Assessment of Social Functioning as a Function of Milestone Achievement

Ever Married	No (N=90)		Yes (N=105)		t	p
	M	SD	M	SD
SLOF Interpersonal Relations	25.69	5.90	25.26	6.34	0.48	.63
QLS Intrapersonal Relations	25.34	11.76	26.80	11.16	0.82	.41
MCCB Composite	38.30	7.09	37.57	6.94	0.71	.48
UPSA-B Total score	75.16	14.10	77.94	11.73	1.45	.15

Table 3.

Self Assessment of Vocational Functioning as a Function of Milestone Achievement

Ever Employed	No (N=33)		Yes (N=162)		t	p
	M	SD	M	SD
SLOF Work Skills	24.69	5.66	23.68	4.51	1.03	.31
QLS Instrumental Role	16.32	4.23	12.57	5.45	2.69	.009
QLS Objects and Activities	7.82	1.78	8.23	1.84	1.00	.32
MCCB Composite	34.31	7.41	41.67	6.96	4.96	.001
UPSA-B Total Score	74.14	13.16	79.45	13.48	3.01	.007
Currently Employed	No (N =171)		Yes (N=24)		t	p
	M	SD	M	SD
SLOF Work Skills	23.51	4.70	26.65	3.41	3.27	.001
QLS Instrumental Role	12.56	5.41	16.63	3.93	3.32	.001
QLS Objects and Activities	7.80	1.87	8.58	1.77	2.07	.046
MCCB Composite	37.58	7.04	39.98	5.92	2.66	.005
UPSA-B Total Score	75.99	13.39	81.11	10.56	2.16	.037

Finally, as seen in Table 4, individuals who were currently either financially responsible or living independently had higher scores on both self-assessed functional abilities and on performance-based measures of ability. Thus, current residential functioning status, defined in terms of both independence and financial responsibility, was associated with self -reports of both functioning and ability scores that were higher than those seen in individuals not living independently or being financially responsible.

4. Discussion

The data presented in this paper make several points regarding self-assessment of functioning in people with schizophrenia. When asked to self-report their functional abilities, patients with current functional achievement in residential and vocational domains rate themselves as more capable in the immediately relevant functional areas than those individuals who were not succeeding in those domains. These reports of better functional skill were corroborated by better performance on tests of neurocognition and functional capacity. However, patients who had never been employed rated themselves as more (or equivalently) capable in vocational skills than those who had been employed in the past or the present. These data suggest that the experience of work, with its associated challenges and demands, may lead to beneficial modification of opinions regarding functional abilities. An additional possibility, consistent with previous research, is that the experience of work itself may in fact lead to improvements in everyday functioning (Bryson et al., 2002). Studies of the maintenance of cognitive remediation suggests that employment is sustained long after the end of cognitive remediation, suggesting that work may be sustained cognitive benefits (McGurk et al., 2007). The results also suggest that the current achievement is correlated with assessments of current functional abilities that are more accurate than assessments of lifetime achievement. Further, previous findings of globally inaccurate self-assessment of functioning may be related to the fact that these assessments were performed on a global basis, collapsing across different functional domains.

There are limitations in this dataset that require explanation. First, the sample was not selected on the basis of milestone achievements and the rates of achievement of these milestones may not be representative of the population of people with schizophrenia. Rates of current employment are consistent with other research, but the rate of marriage and independence in residence may be slightly higher than the general population of people with schizophrenia. Also, we did not measure other abilities, such as social cognition, that may be relevant to some of our milestones, particularly social outcomes (Fett et al., 2011; Ventura et al., in press).

While the differences across milestone achievement in the performance-based measures were statistically significant, the effect sizes were moderate or smaller. Thus, there is likely to be considerable overlap in performance on ability measures across patients who have and have not achieved functional milestones. This is consistent with the larger research literature indicating that environmental factors such as disability compensation, opportunities, and the expense associated with maintenance of a residence have a notable impact on achievement in people with schizophrenia (Rosenheck et al., 2006).

Achievement of relationship milestones was unassociated with performance on NP and FC measures, as well as self-reports of social abilities. It is possible that these achievements are related to other abilities, such as social cognition (See Harvey and Penn, 2010 for a discussion). Also, the current rating scales do not specifically assess achievement of milestones such as marriage, which is a clear shortcoming. Their focus is more on the intermediate achievements such as socializing, conversing, and participating in group social activities.

In summary, these results suggest that there are circumstances where self-report of abilities on the part of people with schizophrenia is generally accurate. At the same time, this accuracy in self-assessment is present in cases with current milestone achievements and better scores on performance-based variables. As achievement of current work was quite rare in the sample (as in schizophrenia patients in general; McGurk and Mueser, in press), the majority of patients are actually providing inaccurate reports of their abilities. Patients who were never employed report that they have better work skills than people who have been employed in the past (and equivalent to the reports of currently employed patients). For people with schizophrenia who have never achieved specific functional milestones, performance-based assessment of skills seems more likely to yield a valid result than asking for self-reports of how capable they are.

Table 4.

Self Assessment of Residential Functioning as a Function of Milestone Achievement

Currently Independent	No (N=76)		Yes (N=119)		t	p
	M	SD	M	SD
SLOF Everyday Activities	50.22	4.19	52.13	(5.84)	2.80	.006
QLS Objects and Activities	7.22	1.66	8.36	(1.79)	4.62	.001
QLS Instrumental Role	11.53	5.31	16.18	(4.85)	4.09	.001
MCCB Composite	35.20	7.06	39.32	(6.86)	4.00	.001
UPSA-B Total Score	70.93	13.36	81.11	(13.05)	5.66	.001
Financially Responsible	No (N	= 109)	Yes (N	= 86)	t	p
	M	SD	M	SD
SLOF Everyday Activities	50.58	5.35	52. 72	5. 34	2.66	.008
QLS Objects and Activities	7.56	1.79	8.39	1.77	3.44	.001
QLS Instrumental Role	11.98	5.27	17.38	4.38	4.50	.001
MCCB Composite	35.19	7.34	38.52	6.44	2.70	.001
UPSA-B Total Score	74.05	13.33	79.64	12.90	2.87	.001