Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Mar 1.
Published in final edited form as: Neuropsychology. 2014 Jul 7;29(2):282–291. doi: 10.1037/neu0000113

THE ROLE OF LOW COGNITIVE EFFORT AND NEGATIVE SYMPTOMS IN NEUROPSYCHOLOGICAL IMPAIRMENT IN SCHIZOPHRENIA

Gregory P Strauss 1,*, Lindsay F Morra 1, Sara K Sullivan 1, James M Gold 2
PMCID: PMC4286525  NIHMSID: NIHMS600292  PMID: 25000322

Abstract

Objective

Two experiments were conducted to examine whether insufficient effort, negative symptoms (e.g., avolition, anhedonia), and psychological variables (e.g., anhedonia and perception of low cognitive resources) predict generalized neurocognitive impairment in individuals with schizophrenia (SZ).

Method

In Experiment 1, participants included 97 individuals with SZ and 63 healthy controls (CN) who completed the Victoria Symptom Validity Test (VSVT), the MATRICS Consensus Cognitive Battery (MCCB), and self-report anhedonia questionnaires. In Experiment 2, participants included 46 individuals with SZ and 33 CN who completed Green’s Word Memory Test (WMT), the MCCB, and self-reports of anhedonia, defeatist performance beliefs, and negative expectancy appraisals.

Results

Results indicated that a low proportion of individuals with SZ failed effort testing (1.0% Experiment 1; 15.2% Experiment 2); however, global neurocognitive impairment was significantly predicted by low effort and negative symptoms.

Conclusions

Findings indicate that low effort does not threaten the validity of neuropsychological test results in the majority of individuals with schizophrenia; however, effort testing may be useful in SZ patients with severe negative symptoms who may be more likely to put forth insufficient effort due to motivational problems. Although the base rate of failure is relatively low, it may be beneficial to screen for insufficient effort in SZ and exclude individuals who fail effort testing from pharmacological or cognitive remediation trials.

Keywords: Effort, Cognition, Anhedonia, Psychosis, Schizophrenia

Introduction

Neuropsychological impairment has long been considered a core characteristic of schizophrenia (SZ) (Kraepelin, 1919). Meta-analyses of neuropsychological test performance confirm that individuals with SZ display neurocognitive impairments on the order of 1.0 standard deviation below healthy controls (Dickinson, Ramsey, & Gold, 2007; Fioravanti, Carlone, Vitale, Cinti, & Clare, 2005). Although the field has searched for focal deficits in specific cognitive domains for over 40 years now, it is clear that there is no distinctive pattern of differential deficits in SZ (Reichenberg & Harvey, 2007). Rather, SZ patients display impairments across a range of cognitive domains and there are generally moderate interrelationships among individual tests (Dickinson, 2008; Dickinson, Iannone, Wilk, & Gold, 2004; Dickinson, Ragland, Gold, & Gur, 2008). Such findings are consistent with a generalized neurocognitive deficit in SZ (Dickinson & Harvey, 2009; Dickinson, et al., 2008).

Several accounts have been proposed to explain the generalized neurocognitive deficit in SZ, including grey and white-matter abnormalities, impaired integration of signals across neural networks, cellular-level neuropathology that causes diffuse effects across brain regions (e.g., GABA internenurons or NMDA receptor dysfunction), and abnormalities in inflammatory, metabolic, and oxidative stress processes (Dickinson & Harvey, 2009). Although central nervous system and “general systems” theories of the generalized cognitive impairment in SZ are compelling, it is possible that psychological factors also play a role. One long held clinical view of neuropsychological impairment in SZ is that problems with motivation undermine the ability to put forth adequate effort on cognitively demanding tasks. If a substantial proportion of individuals with SZ do in fact exhibit insufficient effort on cognitive testing, one might expect a generalized deficit that spans across most areas of cognitive functioning. Relatively few studies have empirically evaluated the impact of effort on neuropsychological test performance in SZ, despite the existence of several effort tests that have been well-validated in clinical and nonclinical populations (Bush, et al., 2005). These effort measures are designed to look like rather difficult tests of memory, whereas in fact they are quite easy. Indeed, even individuals with severe memory impairments or severe Traumatic Brain Injury are capable of achieving near perfect performance on measures of effort (Green, Rohling, Lees-Haley, & Allen, 2001), supporting the notion that low scores on these tests reflect reduced effort rather than true cognitive impairments. Low effort on such tests is often found in assessment contexts where examinees have a clear motive to perform poorly (e.g., litigation, disability determination) (Mittenberg, Patton, Canyock, & Condit, 2002). Studies administering standardized effort tests to individuals with SZ have produced mixed results, with the proportion of patients falling below clinically derived effort cutoffs ranging from 0–20% (Arnold, et al., 2005; Avery, Startup, & Calabria, 2009; Back, 1996; Duncan, 2005; Egeland, et al., 2003; Gierok, Dickson, & Cole, 2005; Hunt, Root, & Bascetta, 2014; Moore, et al., 2013; Pivovarova, Rosenfeld, Dole, Green, & Zapf, 2009; Schroeder & Marshall, 2011) to as high as 60–72% (Gorissen, Sanz, & Schmand, 2005; Hunt, et al., 2014). Inconsistent findings may be due to differences in sample characteristics (e.g., symptom severity and heterogeneity), inpatient vs. outpatient status, whether freestanding vs. embedded effort tests were used, and differences in the sensitivity and specificity of effort tests administered. Importantly, schizophrenia patients who fail effort testing are not thought to be malingering (i.e., feigning cognitive impairment), but rather to have invested insufficient effort due to motivational impairment inherent to the disease. Significant correlations between clinical ratings of negative symptoms (e.g., avolition, anhedonia) and performance on effort tests support this interpretation (Avery, et al., 2009; Gorissen, et al., 2005).

In addition to negative symptoms and effort (Gorissen, et al., 2005; Harvey, Koren, Reichenberg, & Bowie, 2006), several psychological processes may also play a role in neuropsychological impairment in SZ. Grant, Beck, Rector and colleagues (Beck & Rector, 2005; Grant & Beck, 2009; Rector, Beck, & Stolar, 2005) proposed that dysfunctional beliefs play an important role in neurocognitive deficits in SZ. These include defeatist performance beliefs (i.e., overly generalized negative conclusions regarding task performance) and negative expectancy appraisals, such as perceptions of limited cognitive resources, negative beliefs related to one’s ability to persist in difficult tasks, and reduced expectations for success (Couture, Blanchard, & Bennett, 2011; Grant & Beck, 2009; Horan, et al., 2010). Several studies have reported an association between neurocognitive deficits and dysfunctional beliefs in SZ, including defeatist performance beliefs and negative expectancy appraisals (Couture, et al., 2011; Grant & Beck, 2009; Horan, et al., 2010). There is also a significant association between neurocognitive impairment and anhedonia in SZ. In particular, neurocognitive impairments may be related to reduced anticipation of how much pleasure can be experienced from future activities (Strauss & Gold, 2012). It is currently unclear whether dysfunctional beliefs and anhedonia are associated with insufficient effort during neuropsychological testing. In the only study examining such associations to date, effort on the Word Memory Test was significantly correlated with anhedonia, but not negative expectancy appraisals measured in relation to beliefs about performance on the Trail Making Part B test (Avery, et al., 2009). Thus, there is some evidence that a significant proportion of variance in neuropsychological test scores is accounted for by effort, anhedonia, and dysfunctional beliefs; however, the extent to which these variables make individual and combined contributions to neuropsychological impairment in SZ is unclear.

In the current study, we conducted two experiments to determine whether neuropsychological impairment in SZ is predicted by insufficient effort and psychological variables. We hypothesized that a small proportion of individuals with SZ would fail standard effort tests, and that effort, anhedonia, dysfunctional beliefs, and negative symptoms would explain a significant proportion of variance in neuropsychological test scores in SZ.

Experiment #1

In Study 1, we administered a well-validated measure of effort, the Victoria Symptom Validity Test (VSVT: Slick et al. 1997), to a sample of schizophrenia patients and healthy controls. The VSVT is sensitive to detecting low effort and has been shown to differentiate malingerers from healthy controls and neurological patients (Slick et al, 1996). Although the VSVT is presented as a memory test, it is not a true memory test, but a test of effort. Patients with neurological disorders typically obtain near perfect performance on the VSVT (Slick et al., 2003). The VSVT would therefore be expected to be an adequate test of effort in SZ, even though these patients have memory deficits.

Participants

Participants included 97 individuals meeting DSM-IV-TR criteria for schizophrenia or schizoaffective disorder and 63 healthy controls. Patient and control groups did not differ on age, gender, ethnicity, or parental education (Table 1). Individuals with SZ were recruited from the Maryland Psychiatric Research Center outpatient clinics and other local treatment programs. Exclusion criteria included: (1) history of substance abuse or dependence in the past 6 months, (2) history of a head injury, and (3) history of a neurological disorder. Patients were clinically stable at the time of assessment. Stability was defined by constant dosage and type of medication for a period of at least 4 weeks and as judged by treatment providers. Consensus diagnosis was established with a best-estimate approach based on a diagnostic interview and medical records, and subsequently confirmed using the Structured Clinical Interview for DSM–IV (SCID; (First et al., 2001). All patients were prescribed antipsychotic medications.

Table 1.

Participant Demographics and Test Performance for Experiment 1

SZ (n = 97) CN (n = 63) Test Statistic η2partial
Demographics
 Age 42.1 (9.7) 41.5 (9.7) F = 0.17 .001
 Parental Education 13.2 (2.91) 13.5 (2.5) F = 0.43 .220
 Participant Education 12.5 (2.3) 14.9 (1.9) F = 47.45*** .002
 % Male 69.1% 66.7% χ2 = 0.10 --
 Ethnicity χ2 = 1.87 --
  Caucasian 57.7% 61.9%
  African-American 36.1% 34.9%
  American-Indian 2.1% 0%
Neuropsychological Performance
 WTAR SS 95.8 (16.2) 108.5 (12.5) F = 25.99*** .149
 WASI Full-Scale IQ 95.2 (14.5) 113.63 (10.9) F = 68.73*** .310
 VSVT
  Accuracy Easy Items 23.6 (1.0) 24.0 (0.2) F = 10.4*** .060
  Accuracy Difficult Items 21.9 (2.5) 23.7 (0.6) F = 30.3*** .152
  RT Easy Items 1.9 (0.7) 1.4 (0.5) F = 30.7*** .172
  RT Difficult Items 2.9 (1.1) 2.2 (0.7) F = 20.4*** .121
  Right-Left Preference −0.01 (0.06) 0.00(0.03) F = 1.20 .006
Open in a new tab

Note. SZ = Schizophrenia; CN = Control; WTAR SS = Wechsler Test of Adult Reading Scaled Score; WASI = Wechsler Abbreviated Scale of Intelligence; VSVT = Victoria Symptom Validity Test; RT = Reaction Time;

***

= p < 0.001

Healthy control participants were recruited from the community by means of random digit dialing, newspaper advertisements, and word of mouth. In addition to the aforementioned exclusionary criteria, control subjects did not meet criteria for current Axis I or II diagnoses as determined by the Structured Clinical Interview for DSM-IV Axis I Disorders (First et al., 2001) and Structured Interview for DSM-IV Personality (SID-P) (Pfol et al., 1997), reported no family history of psychosis, and were not taking psychotropic medications. Subjects were screened using urine toxicology to assess for current substance use. All participants signed an informed consent for a research protocol approved by the University of Maryland Institutional Review Board.

Measures

Participant completed the Wechsler Test of Adult Reading (Wechsler, 2001) and the MATRICS Cognitive Consensus Battery (MCCB; (Nuechterlein, et al., 2008). The WTAR is a measure of word reading often used as an estimate of premorbid intellectual functioning. The MCCB consists of 10 tests that yield domain scores in relation to 7 areas of cognition (speed of processing, attention/vigilance, working memory, verbal learning, visual learning, reasoning and problem solving, social cognition), as well as a global impairment score.

Participants also completed the Victoria Symptom Validity Test (Slick et al., 1997), a computerized cognitive test used to assess inadequate effort in neuropsychological testing. The VSVT is a 48-item test separated into three blocks of 16 trials. Examinees are presented with a recognition trial consisting of a 5 digit number, followed by a brief retention interval that varies across trials (trials 1–16 = 5 seconds, trials 17–32 = 10 seconds, trials 33–48 = 15 seconds). After the retention interval, subjects receive a forced-choice recognition trial in which they are presented with the original 5-digit number and a 5-digit foil and asked to indicate which of the two numbers was originally presented. Recognition distractors are either easy or difficult, as determined by their similarity to the original stimulus. A measure of inadequate effort was derived from examining performance across these difficult items in comparison to VSVT standardization norms and established cut-scores for inadequate effort. Difficult item correct scores ≥ 16 indicate sufficient effort. By chance alone, individuals would be expected to perform above this score on difficult items approximately 3 out of 100 times. Thus, even though the difficult items are designed to look harder than easy items, difference in performance between easy and difficult items are minimal and “difficult” items are in fact quite easy. Discrepancies between the easy and difficult items or low scores on the difficult items have been shown to predict insufficient effort (Slick, Hopp, Strauss, & Spellacy, 1996).

A clinical interview was performed to assess symptom severity and functional outcome, after which the following scales were completed by clinicians trained to MPRC reliability criteria using gold-standard training videos (Inter-rater reliability > 0.80 on each scale): Scale for the Assessment of Negative Symptoms (SANS; (Andreasen et al., 1983), Brief Psychiatric Rating Scale (BPRS; (Overall & Gorham, 1962), and Level of Function Scale (LOF: (Hawk et al., 1975). Questionnaires administered to explore potential relationships between psychological variables and VSVT performance included: the Revised Chapman Scales for Physical and Social Anhedonia (Chapman, Chapman, & Raulin, 1976; Eckblad, 1982).

Results

VSVT Performance

One individual with SZ and 0 CN failed the VSVT. The proportion of SZ vs. CN who failed the VSVT was nonsignificant (SZ: 1.0% vs. CN: 0.0%), χ2 (1, n = 160) =0.65, p= 1.00. The sole patient who failed the VSVT received scores of 22/24 and 11/24 on the easy and difficult items, respectively. This person was retested approximately 1 month later and passed the second administration, receiving scores of 23/24 (easy) and 18/24 (difficult) (this subject’s original scores are included in analyses).

One-way ANOVAs indicated that SZ had significantly lower accuracy and slower reaction time (RT) than CN on VSVT easy and difficult items (see Table 1). On the VSVT, easy item RTs > 3 seconds have been shown to predict questionable or invalid performance; our patients and CN had mean RTs much faster than 3 seconds. Additionally, larger discrepancies in RT to easy and difficult items are associated with low effort (e.g., >5 seconds); much like CN, our SZ patients displayed minimal differences in RT between easy and difficult items (~ 1 second)

The VSVT also yields a Right-Left Preference score, which indicates the extent to which the examinee displayed a bias toward selecting the left or right item more frequently. Correct responses are evenly distributed across sides, so side preference reflects biased or random responding. Scores < −0.6 indicate a left sided bias and scores > 0.6 indicate a right sided bias. Evaluation of the mean Right-Left preference scores indicated that neither group displayed a left or right-sided bias; ANOVA revealed that groups did not differ on the Right-Left preference score.

Multiple Regression

VSVT difficult accuracy, Chapman anhedonia scores, and SANS total scores were used in stepwise (backward) multiple regression analysis to predict MCCB global scores in SZ. The correlations among the variables are presented in Supplemental Materials (see Tables S1 and S2). In SZ, poorer performance on VSVT difficult items (i.e., lower effort) was associated with greater severity of physical anhedonia and neuropsychological impairment. Poorer neuropsychological test performance was associated with greater severity of physical anhedonia. The prediction model contained 2 of the 4 predictors and was reached in 3 steps with two variables removed. The model was statistically significant and accounted for approximately 26% of variance in MCCB global scores. Cognition was significantly predicted by low effort and negative symptoms (see Table 2).

Table 2.

Regression Models Predicting Global MCCB Performance (Experiment 1)

β F R2
Step 1 7.96*** .28
 Constant
 VSVT Difficult Accuracy .36***
 SANS Total −.27**
 PA −.16
 SA .03
Step 2 10.71*** .28
 Constant
 VSVT Difficult Accuracy .36***
 SANS Total −.26**
 PA −.14
Step 3 14.79*** .26
 Constant
 VSVT Difficult Accuracy .39***
 SANS Total −.30**
Open in a new tab

Note.

***

p < 0.001;

**

p< 0.01;

MCCB = MATRICS Consensus Cognitive Battery; VSVT = Victoria Symptom Validity Test; SANS = Scale for the Assessment of Negative Symptoms; PA = Chapman Scale Physical Anhedonia; SA = Chapman Scale Social Anhedonia

Experiment #2

Results of Study # 1 indicated that only 1% of SZ patients failed the VSVT. Given that the VSVT has been found to be less sensitive to detecting low effort than some other freestanding effort measures (Tan, Slick, Strauss, & Hultsch, 2002), we conducted a second experiment where participants completed a test that is known to be more sensitive to detecting subtle reductions in effort, the Word Memory Test (WMT: (Green, 2003). In addition to examining anhedonia and effort as predictors of cognitive impairment, we also examined defeatist performance beliefs and negative expectancy appraisals. Specifically, we evaluated beliefs in relation to performance on the VSVT and obtained reports of expected success, effort, and pleasure on the WMT after participants heard standard instructions. We also obtained self-reports on questionnaires validated to assess defeatist performance beliefs and negative expectancy appraisals.

Methods

Participants

Participants included 46 individuals with SZ and 33 CN. SZ and CN did not significantly differ on age, parental education, sex, or ethnicity (see Table 3). All patients were prescribed antipsychotic medications. Recruitment, diagnostic, and exclusionary procedures were identical to Experiment 1. All participants signed an informed consent for a research protocol approved by the University of Maryland Institutional Review Board.

Table 3.

Participant Demographics and Clinical Characteristics for Experiment 2

SZ (n = 46) CN (n = 33) Test-Statistic η2partial SZ-PASS WMT (n = 39) SZ-FAIL WMT (n = 7) Test-Statistic η2partial
Age 41.66 (10.72) 39.06 (11.29) F = 0.715 .010 40.36 (11.18) 43.00 (8.37) F = 0.39 .013
Parental Education 13.39 (2.71) 13.42 (1.81) F = 0.05 .001 13.56 (2.36) 11.50 (1.52) F = 2.16 .054
Participant Education 12.43 (2.16) 14.75 (1.90) F = 27.03*** .257 13.69 (2.22) 10.17 (0.98) F = 18.62*** .333
% Male 60.9% 63.6% χ2 = 0.06 -- 62.5% 57.1% χ2 = 0.11 --
Ethnicity χ2 = 0.90 -- χ2 = 1.51 --
 Caucasian 60.9% 57.6% 58.3% 71.4%
 African-American 37.0% 42.4% 38.5% 28.6%
 American-Indian 2.2% 0.0% 2.6% 0.0%
Open in a new tab

Note. SZ = Schizophrenia; CN = Control; SZ-PASS WMT = Schizophrenia patients who Passed the Word Memory Test; SZ-FAIL WMT = Schizophrenia patients who failed the Word Memory Test; Statistical comparison reported in relation to SZ-PASS and SZ-FAIL reflect differences among 3 groups: CN, SZ-FAIL, and SZ-PASS.

***

= p < 0.001.

Measures

In addition to the SCID and SID-P, all participants completed a battery of psychiatric rating instruments, questionnaires, and neuropsychological tests. Questionnaires included: (1) Revised Chapman Physical and Social Anhedonia Scales(Chapman, et al., 1976; Eckblad, 1982), (2) Success and Resource Appraisals Questionnaire (SARA-Q: (Couture, et al., 2011), a self-report measure of expectation for success and appraisal of cognitive resources, and (3) the Defeatist Performance Beliefs subscale of the Defeatist Attitudes Scale, which evaluates overgeneralized conclusions about one’s ability to perform tasks (DPB: (Grant & Beck, 2009; Weissman, 1978)). Symptom rating scales included: (1) Brief Negative Symptom Scale (BNSS: (Kirkpatrick, et al., 2011; Strauss, Hong, et al., 2012; Strauss, Keller, et al., 2012) and (3) Brief Psychiatric Rating Scale (BPRS: (Overall & Gorham, 1962). Functional outcome was assed using the Level of Function Scale (LOF: (Hawk et al., 1975). Neuropsychological status was evaluated using the MATRICS Consensus Cognitive Battery (MCCB: (Nuechterlein, et al., 2008). Of note, 3 participants did not complete the MCCB (SZ: n=2; CN: n=1). All subjects who did not complete the MCCB passed the WMT.

Green’s Word Memory Test (WMT: (Green, 2003)) was administered to measure effort. The WMT consists of 6 subtests, including Immediate Recognition (IR), Delayed Recognition (DR), Multiple Choice (MC), Paired Associates (PA), Free Recall (FR), and Long Delayed Free Recall (LDFR). Participants are shown 20 word pairs on the screen, one at a time, and then tested for immediate recognition (IR). After a 30-minute delay, delayed recognition (DR) is tested using different foils. The consistency between the IR and DR scores is assessed (CNS). Three memory subtests are then given. In Multiple Choice (MC) testing, participants are shown the first word of the pair and asked to select its match from 8 choices. In Paired Associates (PA) a word is given from each pair and the participant is asked to provide its match. In Free Recall (FR) the participant is asked to recall as many words as possible without prompting. Effort is determined by performance on variables that are extremely easy to pass (immediate recognition, delayed recognition and response consistency scores), and on which adults with neurological disease or brain injury can obtain scores of approximately 95% (Green, et al., 2001). Individuals were considered to fail the WMT according to standardized procedures outlined in the WMT manual: scoring below the 82.5% cutoff on any of the 3 WMT effort scale (i.e., Immediate Recognition, Delayed Recognition, Consistency). A composite effort measure was calculated by averaging the 3 WMT effort subtests.

After the standardized WMT instructions were read, participants were asked three questions to determine whether psychological factors predict performance on effort testing: (1) how much they expected to enjoy the test, (2) how successful they expected to be on the test, and (3) how much effort it would take to complete the test. Each self-report was made on a 1 (not at all) to 7 (extremely) scale. Self-reports were not obtained on 3 participants (SZ: n = 1; CN: n = 2). All subjects who did not complete the self-report questions passed the WMT.

Results

WMT Performance

A total of 7/46 individuals with SZ and 0/33 CN failed the WMT. The proportion of SZ vs. CN that failed the WMT was statistically significant (SZ: 15.2% vs. CN: 0.0%), χ2 (1, n = 79) =5.51, p= 0.038. Any individuals who failed the WMT were subsequently administered the VSVT- all participants who failed the WMT scored above the VSVT low effort cut-off score.

Figures 1A and 1B present mean group performance for each of the WMT subtests. A 2 X 6 (Group [SZ, CN] X Subtest [IR, DR, MC, PA, FR, LDFR]) repeated measures ANOVA indicated a significant group x WMT subtest interaction [F(2.89, 211.10) = 33.17, p < .001, ηp2 = .31], within-subjects effect of WMT subtest [F(2.89, 211.10) = 170.17, p < .001, ηp2 = .70], and between-subjects effect of group [F(1, 73) = 51.27, p < .001, ηp2 = .41]. Follow-up one-way ANOVAs revealed that the schizophrenia group performed poorer than the healthy control group on all 6 WMT subtests (all p-values < .01).

Figure 1.

Figure 1

Open in a new tab

WMT Performance among Groups

Note. SZ = Schizophrenia; CN = Control; SZ_Fail = Schizophrenia patients who failed the Word Memory Test; SZ_PASS = Schizophrenia patients who passed the Word Memory Test; IR = Immediate Recognition; DR = Delayed Recognition; MC = Multiple Choice; PA = Paired Associates; FR = Free Recall

Self-Report for WMT Performance and Experience

Patient and healthy control groups were compared on their self-reported ratings of expected enjoyment, success, and effort on the WMT (see Figure 2A). A 2 X 3 (Group [SZ, CN] X WMT Question [enjoy, success, effort]) repeated measures ANOVA revealed a significant within-subjects effect of WMT question [F(2, 148) = 8.41, p < .001, ηp2 = .10] and a significant group effect [F(1, 74) = 8.98, p = .004, ηp2 = .11]. However, the Group X WMT Question interaction was nonsignificant [F(2, 148) = 1.76, p = .18]. Follow-up one-way ANOVAs indicated that compared to CN, SZ expected less enjoyment from the test [F(1, 74) = 7.00, p = .01] and to be less successful on the WMT [F(1, 74) = 11.04, p = .01]; groups did not differ on how much effort they expected it would take to complete the test [F(1, 74) = .364, p = .55].

Figure 2.

Figure 2

Open in a new tab

Self-reports of Expected Enjoyment, Success, and Effort on the WMT among Groups

Note. SZ = Schizophrenia; CN = Control; SZ_Fail = Schizophrenia patients who failed the Word Memory Test; SZ_PASS = Schizophrenia patients who passed the Word Memory Test; Enjoy = Self-report of How much participants expected to enjoy the Word Memory Test; Success = How well participants expected to do on the Word Memory Test; Effort = How much effort participants expected it would take to complete the Word Memory Test

A 3 X 3 (Group [SZ-PASS, SZ- FAIL, CN] X WMT Question [enjoy, success, effort]) repeated measures ANOVA was also conducted, and revealed significant within-subjects [F(2, 146) = 6.85, p = .001, ηp2 = .09] and group [F(2, 73) = 4.55, p = .014, ηp2 = .11] effects; there was a trend toward a significant Group X WMT Question interaction [F(4, 146) = 2.19, p = .073]. Again, groups differed significantly on how much they expected to enjoy the test [F(2, 73) = 5.30, p = .007] and how successful they expected to be on the test [F(2, 73) = 3.66, p = .03], but not on how much effort they expected it would take to complete the test [F(2, 73) = .18, p = .835]. Post-hoc LSD contrasts revealed that only the SZ-pass and the CN groups differed significantly on how much they expected to enjoy the test [p = .003], such that the CN group expected to enjoy the WMT more. There was a trend toward SZ-PASS expecting to enjoy the WMT less than SZ-FAIL (p = .07). Additionally, the SZ-PASS [p = .03] and SZ-FAIL [p = .03] groups expected to be less successful on the WMT than CN (see Figure 2B); however, SZ-PASS and SZ-FAIL groups did not differ on expected success (p = .35).

Neuropsychological Test Performance

One-way ANOVAs revealed significant group differences between SZ and CN on all MCCB subtests, as well as the composite score [p < .001] such that SZ had significantly greater impairment (see Figure 3A). The magnitude of impairment was approximately 2 SDs below healthy controls, which is lower than the 1 SD difference typically found in meta-analyses examining performance on individual tests. This difference in magnitude of impairment may reflect that composite scores are more sensitive than any individual test typically included in meta-analyses.

Figure 3.

Figure 3

Open in a new tab

Neuropsychological Test Performance among Groups

Note. SZ = Schizophrenia; CN = Control; SZ_Fail = Schizophrenia patients who failed the Word Memory Test; SZ_PASS = Schizophrenia patients who passed the Word Memory Test; Proc speed = Processing Speed; Att/Vig = Attention/Vigilance; Work Mem = Working memory; Verb Lrn = Verbal learning; Vis Lrn = Visual learning; Reasoning/PS = Reasoning/Problem-solving; Social cog = Social cognition; Overall = Global t-score.

A 3 X 8 (Group [SZ-PASS, SZ-FAIL, CN] X MCCB Subtest [processing speed, attention/vigilance, working memory, verbal learning, visual learning, reasoning/problem solving, social cognition, overall score]) repeated measures ANOVA was also conducted to determine whether groups displayed different patterns of impairment across MCCB subtests. Group means are shown in Figure 3B. There was a significant effect of MCCB subtest [F(5.08, 365.81) = 11.08, p < .001, ηp2 = .13], Group [F(12, 72) = 51.23, p < .001, ηp2 = .59], and Group X Subtest interaction [F(10.16, 365.81) = 4.07, p < .001, ηp2 = .10]. Follow-up one-way ANOVAs revealed significant group differences on all MCCB subtests, and on the overall MCCB score [all p-values < .001]. Post-hoc LSD contrasts revealed that CN had significantly better performance than SZ-FAIL and SZ-PASS on all 7 MCCB domains and the global score (p < .01 for all). SZ-FAIL performed significantly worse than SZ-PASS on the MCCB processing speed, working memory, visual learning, and global score (p’s < .05); however, there were no differences on attention/vigilance, verbal learning, reasoning/problem solving, and social cognition (p’s > .19).

Multiple Regression

The WMT composite, chapman physical and social anhedonia, SARA-Q, WMT success expectancy, and BNSS total scores were used in stepwise (backward) multiple regression analysis to predict MCCB global scores in SZ. Correlations among the variables are presented in Supplemental materials (Tables S3 and S4). The prediction model contained 2 of the 6 predictors and was reached in 5 steps with 4 variables removed. The model was statistically significant and accounted for approximately 36% of variance in MCCB global scores. Cognition was predicted by low effort and negative symptoms (see Table 4).

Table 4.

Regression Models Predicting Global MCCB Performance (Experiment 2)

β F R2
Step 1 4.26** .44
 WMT .36*
 Success Expectancy .07
 BNSS Total −.37*
 SARA-Q .04
 PA −.39
 SA .28
Step 2 5.25*** .44
 WMT .35*
 Success Expectancy .08
 BNSS Total −.35*
 PA −.38
 SA .28
Step 3 6.60*** .43
 WMT .38**
 BNSS Total −.36*
 PA −.39*
 SA .29
Step 4 7.70*** .39
 WMT .42**
 BNSS Total −.29
 PA −.21
Step 5 10.36*** .36
 WMT .47***
 BNSS Total −.38**
Open in a new tab

Note.

***

p < 0.001;

**

p< 0.01;

MCCB = MATRICS Consensus Cognitive Battery; WMT = Word Memory Test; BNSS = Brief Negative Symptom Scale; SARA-Q = Success and Resource Appraisals Questionnaire; PA = Chapman Scale Physical Anhedonia; SA = Chapman Scale Social Anhedonia; Success Expectancy = How well participants expected to do on the Word Memory Test

General Discussion

Across two experiments, we examined the role of effort in neuropsychologcal test performance in SZ. Results indicated that only 1% of SZ patients failed the VSVT in Experiment 1 and 15.2% failed the WMT in Experiment 2. These findings suggest that the majority of individuals with SZ put forth adequate effort during neuropsychological testing. Low rates of insufficient effort were also found in most prior SZ studies, with similar findings across studies using freestanding and embedded effort tests (Avery, et al., 2009; Moore, et al., 2013; Schroeder & Marshall, 2011). Inconsistencies in effort rates between our two experiments may reflect variation in the sensitivity of the tests used. The WMT is known to be more sensitive to subtler forms of low effort (Tan, et al., 2002), such as those that might be expected to occur in SZ as result of motivational impairments rather than malingering. This explanation appears viable given that when we administered the VSVT to patients who had failed the WMT in Experiment 2, there were no patients who failed both tests.

Interestingly, patients who failed the WMT performed significantly worse than those who passed on only 3 MCCB subtests: processing speed, working memory, and visual learning. There were no differences between those who passed and those who failed on attention/vigilance, verbal learning, reasoning/problem solving, and social cognition. This suggests that although low effort contributes to reduced test scores, it cannot fully account for the generalized cognitive impairment that is observed in SZ. A combination of low effort, CNS, and general systems factors (e.g., inflammation, oxidative stress, metabolic factors) may explain the generalized cognitive deficit in SZ.

Results of our two experiments, as well as most other published studies using standard effort tests in SZ (Arnold, et al., 2005; Avery, et al., 2009; Back, 1996; Duncan, 2005; Egeland, et al., 2003; Gierok, et al., 2005; Hunt, et al., 2014; Moore, et al., 2013; Pivovarova, et al., 2009; Schroeder & Marshall, 2011), stand in contrast to the results of Gorrisen et al. (Gorissen, et al., 2005) who found that 72% of patients diagnosed with SZ failed the WMT. The high proportion of WMT failures led Gorrisen et al. (2005) et al to conclude that insufficient effort represents a serious threat to the validity of neuropsychological test data in SZ. However, given the mounting evidence for relatively low rates of failure on effort tests in SZ (generally <25%), we disagree with this conclusion. Several factors may explain the striking differences in failure rates between Gorrisen et al. (2005) and other published SZ studies. First, patients in the Gorrisen et al. (2005) sample may not be representative of the general population of individuals with SZ. Compared to patients in our samples, those in Gorrisen et al. (2005) were older, more cognitively impaired, and referred for neuropsychological evaluation due to suspected cognitive impairment. Patients in our studies were recruited from an outpatient research center where they received pharmacological and psychosocial treatment- they were not referred for neuropsychological evaluation purposes. Additionally, Gorrisen et al. (2005) included a sample of inpatients that were ostensibly more symptomatic than the stable outpatients in our sample, which tended to have relatively mild positive, negative, and general psychiatric symptoms. Given the remarkable heterogeneity of schizophrenia and potential differences in cognition across phases of the illness, additional studies examining effort in SZ are needed to resolve discrepancies within this literature.

Although the majority of SZ patients passed the effort tests administered in the current study, stepwise regression analyses indicated that global neuropsychological impairment was significantly predicted by low effort and negative symptoms. Self-reported anhedonia was also predictive of neuropsychological impairment, but to a lesser extent. These findings may suggest that SZ patients who fail effort tests and do poorly on neuropsychological assessments exhibit insufficient effort due to abnormal motivational or affective processes. Patients failing the WMT also had significantly lower personal education and a trend toward lower parental education. They may therefore have a history of not performing well during testing sessions and may have low expectations for academic/testing achievement. Unlike many patients who fail effort testing in clinical settings, we do not suspect that individuals with SZ who fail effort tests are feigning cognitive impairment. Rather, insufficient effort travels with negative symptoms (e.g., avolition, anhedonia) that affect a minority of patients with the disease. It may therefore be important to evaluate effort in SZ patients with severe negative symptoms to ensure the validity of their neuropsychological test results. Furthermore, the MCCB domains that correlated with symptom validity test performance included subtests that require sustained effort (e.g., symbol-coding, animal fluency). These results raise caution for examiners, indicating that some patients should be prompted to keep trying on more demanding tests despite features of their illness that make these tests more difficult.

Consistent with hypotheses, dysfunctional beliefs were meaningfully associated with low effort in SZ. In both experiments, greater severity of self-reported physical anhedonia was associated with lower effort in both studies, and in experiment 2 low effort was related to negative expectancy appraisals. Expectations for reduced success on the WMT were most related to low effort, whereas defeatist performance beliefs and SARA-Q scores were not associated with WMT effort. These findings provide only modest support for recent psychological models of SZ (Beck & Grant, 2008; Grant & Beck, 2009). Based on these psychological theories, one would expect that defeatist performance beliefs and beliefs of limited cognitive resources, broadly held, would strongly predict low effort. In our study, only beliefs of low success on the WMT predicted low effort on the WMT, potentially suggesting that situation-specific rather than more global perceptions of limited cognitive resources may predict insufficient effort during neuropsychological testing. In developing future cognitive remediation programs for SZ, it may therefore be valuable to target task-specific low success expectancies using cognitive therapy techniques.

Although the base rate is relatively low, it may be worth screening for insufficient effort and excluding those patients who fail effort tests from clinical trials examining the efficacy of cognitive enhancing drugs or cognitive rehabilitation programs. The exclusion of such patients may increase the likelihood of detecting small to moderate treatment effects. Alternatively, patients failing effort testing could be retained in treatment trials and effort testing could be used to identify which patients might benefit from individually tailored cognitive remediation approaches that incorporate incentives to maximize effort. Freestanding effort tests may be valuable additions to comprehensive neuropsychological test batteries such as the MCCB that are used in cognitive enhancing clinical trials. Given differences in beta weights that were observed between the WMT and VSVT for predicting global neurocognitive impairment on the MCCB, the WMT may be the more sensitive test. Indeed, it has an extensive literature and normative database for score comparisons with both psychiatric and neurological disorders and it has been shown to have better sensitivity and specificity than some other measures (Tan, et al., 2002). However, current guidelines for effort testing recommend the use of multiple measures. Therefore it would be ideal to include other freestanding measures and examine embedded effort indices when performing neuropsychological evaluations on individuals with SZ.

Supplementary Material

Tables

Acknowledgments

Research supported in part by US National Institutes of Mental Health Grants: R01MH080066 to Dr. Gold

References

  1. Arnold G, Boone KB, Lu P, Dean A, Wen J, Nitch S, et al. Sensitivity and specificity of finger tapping test scores for the detection of suspect effort. Clin Neuropsychol. 2005;19(1):105–120. doi: 10.1080/13854040490888567. [DOI] [PubMed] [Google Scholar]
  2. Avery R, Startup M, Calabria K. The role of effort, cognitive expectancy appraisals and coping style in the maintenance of the negative symptoms of schizophrenia. Psychiatry Res. 2009;167(1–2):36–46. doi: 10.1016/j.psychres.2008.04.016. [DOI] [PubMed] [Google Scholar]
  3. Back C, Brauer Boone K, Edwards C. The performance of schizophrenics on three cognitive tests of malingering, Rey 15-Item Memory test, Rey Dot Counting, and Hiscock Forced-Choice Method. Assessment. 1996;3:449–457. [Google Scholar]
  4. Beck AT, Grant PM. Negative self-defeating attitudes: factors that influence everyday impairment in individuals with schizophrenia. Am J Psychiatry. 2008;165(6):772. doi: 10.1176/appi.ajp.2008.08020229. author reply 772. [DOI] [PubMed] [Google Scholar]
  5. Beck AT, Rector NA. Cognitive approaches to schizophrenia: theory and therapy. Annu Rev Clin Psychol. 2005;1:577–606. doi: 10.1146/annurev.clinpsy.1.102803.144205. [DOI] [PubMed] [Google Scholar]
  6. Bush SS, Ruff RM, Troster AI, Barth JT, Koffler SP, Pliskin NH, et al. Symptom validity assessment: practice issues and medical necessity NAN policy & planning committee. Arch Clin Neuropsychol. 2005;20(4):419–426. doi: 10.1016/j.acn.2005.02.002. [DOI] [PubMed] [Google Scholar]
  7. Chapman LJ, Chapman JP, Raulin ML. Scales for physical and social anhedonia. J Abnorm Psychol. 1976;85(4):374–382. doi: 10.1037//0021-843x.85.4.374. [DOI] [PubMed] [Google Scholar]
  8. Couture SM, Blanchard JJ, Bennett ME. Negative expectancy appraisals and defeatist performance beliefs and negative symptoms of schizophrenia. Psychiatry Res. 2011;189(1):43–48. doi: 10.1016/j.psychres.2011.05.032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Dickinson D. Digit symbol coding and general cognitive ability in schizophrenia: worth another look? Br J Psychiatry. 2008;193(5):354–356. doi: 10.1192/bjp.bp.108.049387. [DOI] [PubMed] [Google Scholar]
  10. Dickinson D, Harvey PD. Systemic hypotheses for generalized cognitive deficits in schizophrenia: a new take on an old problem. Schizophr Bull. 2009;35(2):403–414. doi: 10.1093/schbul/sbn097. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Dickinson D, Iannone VN, Wilk CM, Gold JM. General and specific cognitive deficits in schizophrenia. Biol Psychiatry. 2004;55(8):826–833. doi: 10.1016/j.biopsych.2003.12.010. [DOI] [PubMed] [Google Scholar]
  12. Dickinson D, Ragland JD, Gold JM, Gur RC. General and specific cognitive deficits in schizophrenia: Goliath defeats David? Biol Psychiatry. 2008;64(9):823–827. doi: 10.1016/j.biopsych.2008.04.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Dickinson D, Ramsey ME, Gold JM. Overlooking the obvious: a meta-analytic comparison of digit symbol coding tasks and other cognitive measures in schizophrenia. Arch Gen Psychiatry. 2007;64(5):532–542. doi: 10.1001/archpsyc.64.5.532. [DOI] [PubMed] [Google Scholar]
  14. Duncan A. The impact of cognitive and psychiatric impairment of psychotic disorders on the test of memory malingering (TOMM) Assessment. 2005;12(2):123–129. doi: 10.1177/1073191105275512. [DOI] [PubMed] [Google Scholar]
  15. Eckblad ML, Chapman LJ, Chapman JP, Mishlove M. Unpublished test. University of Wisconsin; Madison: 1982. The Revised Social Anhedonia Scale. [Google Scholar]
  16. Egeland J, Sundet K, Rund BR, Asbjornsen A, Hugdahl K, Landro NI, et al. Sensitivity and specificity of memory dysfunction in schizophrenia: a comparison with major depression. J Clin Exp Neuropsychol. 2003;25(1):79–93. doi: 10.1076/jcen.25.1.79.13630. [DOI] [PubMed] [Google Scholar]
  17. Fioravanti M, Carlone O, Vitale B, Cinti ME, Clare L. A meta-analysis of cognitive deficits in adults with a diagnosis of schizophrenia. Neuropsychol Rev. 2005;15(2):73–95. doi: 10.1007/s11065-005-6254-9. [DOI] [PubMed] [Google Scholar]
  18. Gierok SD, Dickson AL, Cole JA. Performance of forensic and non-forensic adult psychiatric inpatients on the Test of Memory Malingering. Arch Clin Neuropsychol. 2005;20(6):755–760. doi: 10.1016/j.acn.2005.04.008. [DOI] [PubMed] [Google Scholar]
  19. Gorissen M, Sanz JC, Schmand B. Effort and cognition in schizophrenia patients. Schizophr Res. 2005;78(2–3):199–208. doi: 10.1016/j.schres.2005.02.016. [DOI] [PubMed] [Google Scholar]
  20. Grant PM, Beck AT. Defeatist beliefs as a mediator of cognitive impairment, negative symptoms, and functioning in schizophrenia. Schizophr Bull. 2009;35(4):798–806. doi: 10.1093/schbul/sbn008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Green P. The Word Memory Test. Green’s Publishing Inc; Seattle: 2003. [Google Scholar]
  22. Green P, Rohling ML, Lees-Haley PR, Allen LM., 3rd Effort has a greater effect on test scores than severe brain injury in compensation claimants. Brain Inj. 2001;15(12):1045–1060. doi: 10.1080/02699050110088254. [DOI] [PubMed] [Google Scholar]
  23. Harvey PD, Koren D, Reichenberg A, Bowie CR. Negative symptoms and cognitive deficits: what is the nature of their relationship? Schizophr Bull. 2006;32(2):250–258. doi: 10.1093/schbul/sbj011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Horan WP, Rassovsky Y, Kern RS, Lee J, Wynn JK, Green MF. Further support for the role of dysfunctional attitudes in models of real-world functioning in schizophrenia. J Psychiatr Res. 2010;44(8):499–505. doi: 10.1016/j.jpsychires.2009.11.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Hunt S, Root JC, Bascetta BL. Effort testing in schizophrenia and schizoaffective disorder: validity indicator profile and test of memory malingering performance characteristics. Arch Clin Neuropsychol. 2014;29(2):164–172. doi: 10.1093/arclin/act069. [DOI] [PubMed] [Google Scholar]
  26. Kirkpatrick B, Strauss GP, Nguyen L, Fischer BA, Daniel DG, Cienfuegos A, et al. The brief negative symptom scale: psychometric properties. Schizophr Bull. 2011;37(2):300–305. doi: 10.1093/schbul/sbq059. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kraepelin E. In: Dementia Praecox and Paraphrenia. BRM, translator. New York: 1919. [Google Scholar]
  28. Mittenberg W, Patton C, Canyock EM, Condit DC. Base rates of malingering and symptom exaggeration. J Clin Exp Neuropsychol. 2002;24(8):1094–1102. doi: 10.1076/jcen.24.8.1094.8379. [DOI] [PubMed] [Google Scholar]
  29. Moore RC, Davine T, Harmell AL, Cardenas V, Palmer BW, Mausbach BT. Using the repeatable battery for the assessment of neuropsychological status (RBANS) effort index to predict treatment group attendance in patients with schizophrenia. J Int Neuropsychol Soc. 2013;19(2):198–205. doi: 10.1017/S1355617712001221. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Nuechterlein KH, Green MF, Kern RS, Baade LE, Barch DM, Cohen JD, et al. The MATRICS Consensus Cognitive Battery, part 1: test selection, reliability, and validity. Am J Psychiatry. 2008;165(2):203–213. doi: 10.1176/appi.ajp.2007.07010042. [DOI] [PubMed] [Google Scholar]
  31. Pivovarova E, Rosenfeld B, Dole T, Green D, Zapf P. Are Measures of Cognitive Effort and Motivation Useful in Differentiating Feigned from Genuine Psychiatric Symptoms? International Journal of Forensic Mental Health. 2009;8:271–278. [Google Scholar]
  32. Rector NA, Beck AT, Stolar N. The negative symptoms of schizophrenia: a cognitive perspective. Can J Psychiatry. 2005;50(5):247–257. doi: 10.1177/070674370505000503. [DOI] [PubMed] [Google Scholar]
  33. Reichenberg A, Harvey PD. Neuropsychological impairments in schizophrenia: Integration of performance-based and brain imaging findings. Psychol Bull. 2007;133(5):833–858. doi: 10.1037/0033-2909.133.5.833. [DOI] [PubMed] [Google Scholar]
  34. Schroeder RW, Marshall PS. Evaluation of the appropriateness of multiple symptom validity indices in psychotic and non-psychotic psychiatric populations. Clin Neuropsychol. 2011;25(3):437–453. doi: 10.1080/13854046.2011.556668. [DOI] [PubMed] [Google Scholar]
  35. Slick DJ, Hopp G, Strauss E, Spellacy FJ. Victoria Symptom Validity Test: efficiency for detecting feigned memory impairment and relationship to neuropsychological tests and MMPI-2 validity scales. J Clin Exp Neuropsychol. 1996;18(6):911–922. doi: 10.1080/01688639608408313. [DOI] [PubMed] [Google Scholar]
  36. Strauss GP, Gold JM. A new perspective on anhedonia in schizophrenia. Am J Psychiatry. 2012;169(4):364–373. doi: 10.1176/appi.ajp.2011.11030447. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Strauss GP, Hong LE, Gold JM, Buchanan RW, McMahon RP, Keller WR, et al. Factor structure of the Brief Negative Symptom Scale. Schizophr Res. 2012;142(1–3):96–98. doi: 10.1016/j.schres.2012.09.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Strauss GP, Keller WR, Buchanan RW, Gold JM, Fischer BA, McMahon RP, et al. Next-generation negative symptom assessment for clinical trials: validation of the Brief Negative Symptom Scale. Schizophr Res. 2012;142(1–3):88–92. doi: 10.1016/j.schres.2012.10.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Tan JE, Slick DJ, Strauss E, Hultsch DF. How’d they do it? Malingering strategies on symptom validity tests. Clin Neuropsychol. 2002;16(4):495–505. doi: 10.1076/clin.16.4.495.13909. [DOI] [PubMed] [Google Scholar]
  40. Wechsler D. Wechsler Test of Adult Reading (WTAR) Toronto, Ontario, Canada: Pearson Canada Assessment; 2001. [Google Scholar]
  41. Weissman A. The Dysfunctional Attitudes Scale: A Validation Study. Philadelphia, Pa: University of Pennsylvania; 1978. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Tables
NIHMS600292-supplement-Tables.docx (25.5KB, docx)

RESOURCES