Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Mar 1.
Published in final edited form as: Schizophr Res. 2015 Jan 9;162(0):205–210. doi: 10.1016/j.schres.2014.12.033

Predictors of Neuropsychological Effort Test Performance in Schizophrenia

Lindsay F Morra 1, James M Gold 2, Sara K Sullivan 1, Gregory P Strauss 1,*
PMCID: PMC4339544  NIHMSID: NIHMS654950  PMID: 25583248

Abstract

There is some evidence that insufficient effort may be common in schizophrenia, posing significant threats to the validity of neuropsychological test results. Low effort may account for a significant proportion of variance in neuropsychological test scores and the generalized cognitive deficit that characterizes the disorder. The current study evaluated clinical predictors of insufficient effort in schizophrenia using an embedded effort measure, the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) Effort Index (EI). Participants were 330 patients meeting DSM-IV-TR criteria for schizophrenia, schizoaffective disorder, or another psychotic disorder who received a battery of neuropsychological tests, including: Wechsler Test of Adult Reading (WTAR), Wechsler Abbreviated Scale of Intelligence (WASI), and RBANS. Clinical assessments designed to measure functional outcome, symptoms, and premorbid adjustment were also obtained. Results indicated that 9.4% of patients failed the EI. Patients who failed had lower full-scale, verbal, and performance IQ, as well as poorer performance on RBANS domains not included in the EI (immediate memory, language, and visuospatial/construction). Patients who failed the EI also displayed poorer community-based vocational outcome, greater likelihood of having “deficit schizophrenia” (i.e., primary and enduring negative symptoms), and increased severity of positive symptoms. Regression analyses revealed that insufficient effort was most significantly predicted by a combination of low IQ, negative symptoms, and positive symptoms. Findings suggest that although insufficient effort may be relatively uncommon in schizophrenia, it is associated with important clinical outcomes. The RBANS EI may be a useful tool in evaluating insufficient effort in schizophrenia.

Keywords: Psychosis, Effort, Cognition, Negative Symptoms, Intelligence

1.0. Introduction

Neuropsychological impairment is common in schizophrenia, and has long been considered a core feature of the illness (Kraepelin, 1919). Meta-analyses indicate that individuals with schizophrenia display neurocognitive impairments approximately one standard deviation below the mean for healthy controls (Dickinson et al., 2007; Fioravanti et al., 2005). Despite such pervasive cognitive impairments, there is no distinct pattern of differential deficits that characterizes most individuals with schizophrenia (Reichenberg and Harvey, 2007). Rather, schizophrenia patients display neurocognitive impairments of similar magnitude across most cognitive domains, suggesting a generalized neurocognitive deficit (Dickinson, 2008; Dickinson et al., 2004; Dickinson et al., 2008).

Several theories have been proposed to account for this generalized neurocognitive deficit, including central nervous system (e.g., grey and white-matter abnormalities, impaired integration of signals across neural networks, cellular-level neuropathology) and “general systems” (e.g., inflammatory, metabolic, and oxidative stress processes) abnormalities that can negatively impact cognition (Dickinson and Harvey, 2009). However, it is also possible that psychological factors contribute substantially to the neurocognitive impairments observed in schizophrenia. One possibility is that problems with motivation result in inadequate effort on measures of neurocognition, particularly on tasks that are more cognitively demanding. To date, relatively few studies have examined insufficient effort during neuropsychological testing in schizophrenia and whether such abnormalities are associated with motivational problems. Those studies that have been conducted have produced inconsistent results, with the majority indicating that a small proportion of individuals with schizophrenia (∼20%) perform below clinically established cut-off scores for valid effort (Arnold et al., 2005; Avery et al., 2009; Back, 1996; Duncan, 2005; Egeland et al., 2003; Gierok et al., 2005; Hunt et al., 2014; Moore et al., 2013; Pivovarova et al., 2009; Schroeder and Marshall, 2011), and other studies indicating that up to 60-72% of the sample may fail effort testing (Gorissen et al., 2005; Hunt et al., 2014).

Despite these inconsistencies regarding rates of effort test failure, there is reliable evidence that certain clinical variables predict low effort in schizophrenia. For example, multiple studies have found that global scores on negative symptom rating scales, such as the Scale for the Assessment of Negative Symptoms (Andreasen, 1983) or the Brief Negative Symptom Scale (Kirkpatrick, 2011), account for a substantial proportion of variance in effort test performance (Avery et al., 2009; Gorissen et al., 2005; Strauss et al., 2014). Several psychological variables also differentiate patients who pass and fail effort measures, including self-reported anhedonia and the perception of low cognitive resources (Avery et al., 2009; Strauss et al., 2014). These findings suggest that negative symptoms and psychological processes associated with negative symptoms may be core to diminished effort during neuropsychological testing.

However, negative symptoms are both multi-dimensional and multi-determined and it is currently unclear which aspects of negative symptoms are associated with low effort. There is consistent evidence for the multi-dimensionality of negative symptoms, such that 2 distinct negative symptom factors are commonly identified, one reflecting diminished motivation (anhedonia, avolition, asociality) and the other diminished expressivity (alogia and restricted affect) (Blanchard, 2006; Horan et al., 2011; Strauss et al., 2012). These two dimensions have different demographic and clinical correlates (Strauss et al., 2013), with more severe volitional pathology generally predicting worse outcomes. Given that neuropsychological impairment has been associated with the motivational dimension more strongly than the diminished expressivity dimension (Fervaha, 2014), one might expect effort test performance to be specifically linked to greater severity of motivational symptoms. Furthermore, it is now generally accepted that negative symptoms are multi-determined— two patients can display identical scores on negative symptom rating scales for very different reasons. This notion was highlighted in the seminal work of Carpenter and colleagues (1988), which demonstrated that negative symptoms can result from either primary or secondary factors. Primary negative symptoms are those that are idiopathic to the illness, whereas secondary negative symptoms result from processes such as paranoid social withdrawal, depression, disorganization, hallucinations, and suspiciousness. If low effort is indeed critically linked to true motivational problems in schizophrenia, one might expect higher rates of effort test failure in patients who meet clinical diagnostic criteria for “deficit schizophrenia”, i.e., those with primary and clinically stable negative symptoms (Carpenter et al., 1988; Kirkpatrick, Buchanan, Ross, & Carpenter, 2001). Patients with deficit schizophrenia typically fall 1 SD below nondeficit schizophrenia patients and 2 SD below healthy controls on standard neuropsychological tests (Buchanan et al., 1994; Cohen et al., 2007); however, it remains to be seen whether patients meeting clinical criteria for deficit schizophrenia are more likely to fail effort tests than nondeficit patients.

In the current study, we explored rates of effort test failure in a large sample of individuals with schizophrenia using an embedded effort measure that has been well-validated in clinical populations, the RBANS Effort Index (Silverberg et al., 2007). Clinical predictors of insufficient effort were examined, with an emphasis on determining whether the motivational dimension and primary negative symptoms are most predictive of effort test failure. It was hypothesized that a small percentage of individuals with schizophrenia (< 20%) would fail the RBANS Effort Index and that patients falling below the low-effort cut-off would be more likely to meet clinical criteria for deficit schizophrenia, have greater severity of motivational symptoms, and poorer community-based functional outcome.

2.0. Method

2.1. Participants

Participants included 330 individuals meeting DSM-III or DSM-IV criteria for schizophrenia (n=289), schizoaffective disorder (n=32), or another psychotic disorder (n=9). Outpatients were recruited from the Maryland Psychiatric Research Center (MPRC) outpatient clinics and other local outpatient clinical care centers. Inpatients were recruited from the Treatment Research Program unit of the MPRC. Participants were excluded for: (1) history of substance abuse or dependence in the past 6 months, (2) history of a head injury, and (3) history of a neurological disorder. All patients were assessed in a research (rather than clinical) context, and therefore did not have any identifiable motivation to feign or exaggerate clinical and/or cognitive symptoms. Patients were not using results of neuropsychological evaluations for disability compensation or litigation purposes.

Participants were divided into those who passed (SZ-PASS) and failed (SZ-FAIL) the RBANS Effort Index according to established procedures (Silverberg et al., 2007). Demographic characteristics of the SZ-PASS and SZ-FAIL groups are presented in Table 1. The two groups did not differ on sex, race, or years of parental education. However, the SZ-FAIL group was significantly older and had fewer years of personal education than SZ-PASS.

Table 1. Participant Demographics By Group.

SZ-PASS (N=299) SZ-FAIL (N=31) Test Statistic, p value
Age 35.04 (10.36) 41.55 (14.45) F = 10.20, p <0.01
% Male 72.6% 77.4% χ2 = 0.34, p = 0.56
Ethnicity χ2 = 1.48, p = 0.83
 Caucasian 53.8% 61.3%
 African American 41.8% 32.3%
 American Indian 0.3% 0%
 Asian 1.7% 3.2%
 Other 2.3% 3.2%
Parental Education 12.48 (3.11) 11.46 (2.54) F = 2.32, p =0.13
Participant Education 12.24 (2.10) 11.38 (2.34) F = 4.39, p =0.04
Open in a new tab

Notes: SZ-PASS = participants with an EI>3, SZ-FAIL, participants with an EI≤3.

2.2. Measures

Participants completed a clinical interview, after which psychiatric rating instruments designed to measure community-based functional outcome, psychosis, disorganization, and negative symptoms were completed. Neuropsychological tests were also administered. Consensus diagnosis was established via a best-estimate approach based upon multiple interviews and a detailed psychiatric history. This diagnosis was subsequently confirmed using the Structured Clinical Interview for DSM (SCID). Symptom rating scales were completed by clinicians trained to MPRC reliability standards (reliability > 0.80; Schedule for the Deficit Syndrome kappa > 8/10).

Symptom severity measures included the Schedule for the Deficit Syndrome (SDS: Kirkpatrick et al., 1989) and the Scale for the Assessment of Positive Symptoms (SAPS: Andreasen, 1984). The SDS assesses the severity of six negative symptoms (restricted affect, poverty of speech, diminished emotional range, curbed interests, diminished sense of purpose, and diminished social drive), and whether those symptoms are primary/secondary and enduring (stable > 1 year). The measure is primarily intended to yield a categorization of “deficit” or “nondeficit” schizophrenia. Individuals meet criteria for the deficit syndrome if they have 2 or more negative symptoms that are clinically significant, and those symptoms are considered primary and stable. The SDS demonstrated good psychometric properties in the original study and subsequent studies have indicated strong inter-rater reliability and convergent validity (Amador et al., 1999; Fenton and McGlashan, 1992; Kirkpatrick, 1989). Factor analytic studies suggest that the SDS items assessing symptom severity load onto two coherent factors— avolition and diminished emotional expressivity (Kimhy et al., 2006; Strauss et al., 2013).

The SAPS consists of 30 items that assess hallucinations, delusions, and disorganization (Andreasen, 1984). Each SAPS item is scored on a 6-point scale (0-5). The Level of Function Scale (Hawk et al., 1975) measures community-based functional outcome in eight areas: Duration of Hospitalization, Frequency of Social Contacts, Quality of Social Relations, Employment, Work Quality, Symptoms, Ability to Meet own Basic Needs, and Fullness of Life. LOF ratings are assigned on a 0–4 scale with high scores indicating better functional outcome (and decreased symptoms for the symptoms domain). All of the domains, except symptom severity, are rated for the last six months; symptom severity is rated for the past month. Summary scores were calculated for total functional outcome, work outcome, and social outcome in the current study.

Neuropsychological measures included the (1) Wide Range Achievement Test- Third Edition (WRAT-3: Wilkinson, 1993), (2) Wechsler Adult Intelligence Scale- Third Edition (Information, Block Design, Arithmetic, and Symbol Search subtests) (WAIS-III: Wechsler, 1997), and (3) Repeatable Battery for the Assessment of Neuropsychological Status (RBANS: Randolph, 1998). The RBANS is a brief neurocognitive battery that consists of 12 subtests measuring cognitive abilities across five broad domains: Immediate Memory, Visuospatial Ability, Language, Attention, and Delayed Memory. A total score is also derived from summing index scores. The RBANS has demonstrated adequate sensitivity, reliability, and validity for neurocognitive screening of individuals with schizophrenia (Gold, 1999; Wilk, 2004).

The RBANS Effort Index (EI) was calculated to evaluate insufficient effort using standard procedures developed by Silverberg et al. (2007) that use the List Recognition and Digit Span subtests. Insufficient effort was defined as an EI >3.

3.0. Results

3.1. Insufficient Effort and Neuropsychological Performance on the RBANS

Thirty-one patients (9.4%) failed the RBANS Effort Index (EI). The distribution of patients falling at each EI vale was: 0 = 202 (61.2%), 1 = 31 (9.4%), 2 = 36 (10.9%), 3 = 30 (9.1%), 4 = 7 (2.1%), 5 = 10 (3.0%), 6 = 8 (2.4%), 7 = 5 (1.5%), 9 = 1 (0.3%).

Given that SZ-FAIL and SZ-PASS groups significantly differed on age, age was entered as a covariate in primary analyses. As expected, separate oneway ANCOVAs indicated that the SZ-PASS and SZ-FAIL groups significantly differed on both Digit Span, F(1, 327) = 71.90, p < 0.001, ηp2 =0.18, and List Recognition, F (1, 327) = 148.49, p < 0.001, ηp2 =0.31, subtests that are used to derive the EI.

Differences in neuropsychological test performance between groups are presented in Table 2. A MANCOVA, using age as a covariate, revealed that patients who failed the effort index had lower scores on the RBANS total score, as well as the five index scores (language, attention, visuospatial, immediate memory, and delayed memory) (p < .01). Effect sizes ranged from ηp2 =0.03 to ηp2 =0.18. Of those indexes that do not include subtests comprising the EI, effect sizes were the greatest for the Immediate Memory (ηp2 =0.16) and Language (ηp2 =0.09) Index scores. Consistent with the RBANS results, the patients who failed the Effort Index also demonstrated markedly lower IQ and WRAT reading scores than the patients who passed the EI.

Table 2. Neuropsychological Performance in Pass and Fail Groups.

SZ-PASS SZ-FAIL Test Statistic, p value Effect Sizes (ηp2)
Estimated FSIQ (WAIS-III) 84.66 (16.21) 68.48 (10.35) F = 29.56, p <0.001
WRAT-3 Scaled Scores 90.03 (15.16) 75.32 (17.93) F = 25.50, p <0.001
RBANS Total 72.24 (13.88) 51.13 (5.24) F = 73.56, p < 0.001 0.18
 Immediate Memory 75.65 (17.42) 50.19 (7.78) F = 61.19, p < 0.001 0.16
 Visuospatial 79.38 (17.63) 68.39 (13.48) F = 11.29, p = 0.001 0.03
 Language 82.88 (14.67) 69.61 (16.62) F = 31.24, p < 0.001 0.09
 Attention 74.47 (15.49) 51.71 (9.21) F = 72.28, p < 0.001 0.18
 Delayed Memory 77.40 (17.78) 49.84 (11.44) F = 70.99, p < 0.001 0.18
Open in a new tab

Note: FSIQ= Full Scale Intelligence Quotient, WAIS-III = Wechsler Adult Intelligence Scale, 3rd Edition, WRAT-3 = Wide Range Achievement Test, 3rd Edition, RBANS = Repeatable Battery for the Assessment of Neuropsychological Status, SZ-PASS = Individuals with schizophrenia that passed the RBANS Effort Index, SZ-FAIL = Individuals with schizophrenia that failed the RBANS Effort Index. A total of 330 patients had WAIS-III data: 299 SZ-PASS and 31 SZ-FAIL, a total of 329 patients had WRAT-3 data: 298 SZ-PASS and 31 SZ-FAIL, and a total of 330 patients had RBANS data: 299 SZ-PASS and 31 SZ-FAIL.

3.2. Clinical Symptoms and Functional Outcome

Patients in the SZ-FAIL and SZ-PASS groups differed significantly on several clinical symptom variables (see Table 3). The SZ-FAIL group was more likely to meet criteria for the deficit syndrome (p=0.02) and had greater severity of all 6 negative symptom subscales assessed by the SDS (all p-values ≤ 0.05). Group differences in severity of negative symptom subscales remained significant after controlling for SAPS total scores (all p-values ≤ 0.01), suggesting that these negative symptom effects cannot be attributed to the secondary factor of psychosis.

Table 3. Participant Clinical Characteristics By Group.

SZ-PASS SZ-FAIL Test Statistic, p value
SDS Global Categorization 18.5% Deficit 39.1% Deficit χ2 = 5.51 p = 0.02
SDS Severity Scores:
 Restricted Affect 1.07 (1.09) 1.67 (1.37) F = 6.26, p = 0.01
 Diminished Emotional Range 0.94 (1.04) 1.54 (1.50) F = 6.77, p = 0.01
 Poverty of Speech 0.69 (1.08) 1.29 (1.37) F = 6.41, p = 0.01
 Curbing of Interests 1.00 (1.16) 1.92 (1.53) F = 12.83, p < 0.01
 Diminished Sense of Purpose 1.18 (1.32) 2.29 (1.55) F = 14.90, p < 0.01
 Diminished Social Drive 1.21 (1.25) 1.75 (1.442) F = 3.97, p = 0.05
Positive Symptoms (SAPS)
 Hallucinations 1.97 (1.89) 3.05 (1.94) F = 6.52, p =0.01
 Delusions 2.34 (1.67) 2.68 (1.76) F = 0.85, p =0.36
 Bizarre Behavior 0.75 (1.11) 1.41 (1.40) F = 6.71, p =0.01
 Formal Thought Disorder 1.13 (1.31) 1.82 (1.87) F = 5.21, p =0.02
 SAPS Total Score 24.05 (20.51) 34.40 (22.79) F = 4.60, p =0.03
Functional Outcome
 LOF Social 3.28 (2.54) 2.67 (2.14) F = 1.31, p =0.25
 LOF Work 2.28 (2.46) 1.25 (1.29) F = 4.05, p =0.05
 LOF Total Score 16.20 (7.04) 12.38 (5.88) F = 6.65, p =0.01
Open in a new tab

Notes: SZ-PASS = participants with an EI > 3, SZ-FAIL, participants with an EI ≤ 3, SDS = Schedule for the Deficit Syndrome, SAPS = Scale for the Assessment of Positive Symptoms, LOF = Level of Functioning Scale. A total of 299 patients had SDS data: 268 SZ-PASS and 31 SZ-FAIL, a total of 249 patients had SAPS data: 229 SZ-PASS and 20 SZ-FAIL, and a total of 270 patients had LOF data: 246 SZ-PASS and 24 SZ-FAIL.

The SZ-FAIL group also had more severe hallucinations, bizarre behavior, formal thought disorder, and total positive symptoms (p-values≤ 0.03) relative to the SZ-PASS group, but did not have greater severity of delusions (p= 0.36). Groups also differed in community-based functional outcome. More specifically, the SZ-FAIL group demonstrated significantly lower scores on measures of work and overall functioning (p= 0.05 and p= 0.01, respectively). In contrast, groups did not differ on social outcome.

To investigate whether cognitive effort was related to the volitional and/or emotional expressivity dimensions of negative symptoms, two index scores were calculated using the SDS severity items. The avolition dimension was calculated by averaging curbing of interests, diminished sense of purpose, and diminished social drive items. The diminished expressivity dimension was calculated by averaging the severity scores for the restricted affect, diminished emotional range, and poverty of speech items. Analyses revealed that greater severity of both diminished emotional expression and volition were significantly associated with higher scores on the EI (r = 0.24, p < 0.001 and r = 0.25, p < 0.001, respectively). Differences remained statistically significant for diminished emotional expressivity (r = 0.28, p < 0.001) and volition (r = 0.30, p < 0.001) dimensions when age and SAPS total scores were partialed out.

3.3. Predictors of Insufficient Effort

SDS mean severity scores, SAPS total scores, and full-scale IQ scores were used in a binary logistic regression analysis to predict RBANS EI pass/fail group membership. Overall, the model correctly classified 92% of patients. The prediction model was statistically significant and accounted for approximately 30% (Nagerkle R2) of variance in EI failure, χ2 = 34.65, p < 0.001. Negative symptoms, positive symptoms, and IQ were all significant predictors (p < 0.02) (see Table 4).

Table 4. Logistic Regression Model Prediction RBANS Effort Index Failure Group.

95% Confidence Interval for Odds Ratio
B (SE) Wald Exp(B) Lower Upper
IQ -0.10 (0.03)*** 11.4 0.91 0.86 0.96
SAPS 0.03 (0.01)* 5.6 1.03 1.01 1.05
SDS 0.11 (0.04)** 7.9 1.11 1.03 1.20
Constant 3.00 (2.05) 2.14 20.25
Open in a new tab

Note.

***

p < 0.001,

**

p < 0.01,

*

p <0.05;

IQ = Wechsler Estimated Full-Scale Intelligence; SAPS = Scale for the Assessment of Positive Symptoms Total Score; SDS = Schedule for the Deficit Syndrome average severity score.

Table 5 presents correlations between dimensional RBANS EI scores and WAIS estimated full-scale IQ, SDS symptom severity, SAPS total scores, and LOF work, social, and total scores. Lower effort was associated with lower IQ, greater severity of volitional and expressive negative symptoms, poorer global functional outcome, and poorer work outcome. Correlations with positive symptoms and social outcome were nonsignificant.

Table 5. Correlations between Continuous RBANS Effort Index Scores and Clinical variables.

r, p-value
Full-Scale IQ -0.35***
SDS Diminished Expressivity 0.24***
SDS Volition 0.25***
SDS Total 0.27***
SAPS Total 0.08
LOF Social -0.11
LOF Work -0.14*
LOF Total -0.19
Open in a new tab

Note.

***

p < 0.001;

*

p <0.05;

SDS = Schedule for the Deficit Syndrome; SAPS = Scale for the Assessment of Positive Symptoms; LOF = Level of Functional Scale; Lower scores on the LOF reflect poorer functional outcome

4.0. Discussion

Results indicated that only 9.4% of schizophrenia patients fell below the cut-off for low effort on the RBANS EI. This suggests that the majority of our sample put forth sufficient effort during neuropsychological testing. The rate of patients exhibiting low effort in our study is somewhat lower than the majority of published studies, which have indicated that approximately 20% of people with schizophrenia fail neuropsychological effort tests. Like prior studies, we did not expect patients failing the EI to be malingering (i.e., feigning low performance). Rather, we expected that patients who failed the EI would display clinically significant motivational impairments that negatively impacted test performance. Indeed, this is what was observed- patients failing the effort index had a greater severity of clinically rated negative symptoms than patients who passed the EI. These findings are consistent with several prior studies reporting an association between negative symptoms and low effort (Avery et al., 2009; Gorissen et al., 2005; Strauss et al., 2014).

Most importantly, we extended the results of prior studies indicating an association between negative symptoms and low effort by teasing apart the contributions of two critical negative symptom factors- dimensionality and whether negative symptoms are attributed to primary vs. secondary causes. We hypothesized that low effort would be predicted by greater severity of volitional pathology and greater likelihood of meeting criteria for deficit schizophrenia (i.e., primary and enduring negative symptoms). Contrary to hypotheses, low effort showed a similar magnitude of correlation with the volitional and diminished expressivity dimensions. This suggests that low effort is associated with greater severity of negative symptoms broadly defined, rather than increased severity of volitional pathology specifically. Results did, however, support the hypothesis that there would be a greater proportion of patients meeting criteria for deficit schizophrenia among those who failed the EI than among those who passed. This finding suggests that patients with primary and enduring negative symptoms are more likely to exert low effort during cognitive testing, potentially explaining why many studies find that deficit schizophrenia patients perform on the order of one standard deviation below nondeficit schizophrenia patients on standard neuropsychological tests (Cohen et al., 2007). Furthermore, insufficient effort was related to several other clinical variables that are common secondary negative symptom causes, including positive and disorganized symptoms. When these variables were entered as covariates or partialed during correlations, negative symptom effects persisted. Thus, although associations were observed between low effort and several forms of psychopathology, results suggest that negative symptom effects observed are not likely to be attributable to common causes of secondary negative symptoms.

Interestingly, low IQ also predicted insufficient effort. Effort test results are typically thought to be valid indicators of performance in individuals with low or borderline intellectual functioning; however, there has been some evidence that neurological patients (e.g., epilepsy) with lower intellectual functioning fail a significantly greater number of effort measures than patients with higher intellectual functioning, even in the absence of any identifiable incentive to feign cognitive impairment (Dean et al., 2008). Our results suggest that schizophrenia patients with low IQ may also be highly susceptible to failing effort tests. These findings call into question whether effort tests are appropriate for use in schizophrenia patients with low IQ. It is unclear whether insufficient effort leads to low performance on IQ tests, or low IQ leads to greater rates of effort test failure. It is possible that the patients who failed the EI were affected by neurodevelopmental abnormalities years before the neuropsychological tests were administered; results indicating that patients who failed the EI had lower personal education and WRAT-3 reading scores than patients who passed are consistent with this notion. Thus, we suspect that either low IQ and neurodevelopmental abnormalities lead to greater rates of effort test failure, or that the same kind of motivational processes that affect performance on effort tests are also present early in development, resulting in poor academic attainment and difficulty learning to read.

Another important finding was that insufficient effort was associated with poorer neurocognitive performance across all five RBANS domains (language, attention, visuospatial, immediate memory, and delayed memory), as well as tests not used to compute the EI. These findings are consistent with other studies indicating that low effort may play a role in the generalized cognitive deficit that is observed in schizophrenia (Strauss et al., 2014). Furthermore, among the domains not used to compute the EI, there was some evidence for larger effect sizes on more cognitively demanding than less demanding tests. These findings are consistent with evidence indicating that psychological processes (e.g., low expectancies for success) may play a significant role in both neuropsychological impairment and insufficient effort (Couture et al., 2011; Horan et al., 2010; Strauss et al., 2014).

Although only a minority of our schizophrenia sample failed the EI, findings have important implications for clinical trials examining the efficacy of cognitive enhancing drugs or cognitive retraining programs. Specifically, it may be useful to screen for insufficient effort to identify patients who should be excluded from clinical trials. The exclusion of such patients may increase the likelihood of observing small to moderate treatment effects. Alternatively, effort testing could be used to identify patients in need of individually tailored protocols that take motivational difficulties into account. Well-validated embedded effort measures, such as the RBANS EI, may offer a screening method that is both time and cost effective.

Certain limitations and interpretative considerations should be taken into account when evaluating these findings. First, it is well-documented that various free-standing and embedded measures result in variable rates of effort test failure (Armistead-Jehle and Hansen, 2011). This may reflect differences in sensitivity and specificity across tests, complicating comparisons in the rate of insufficient effort across schizophrenia studies. Future studies could administer multiple free-standing and embedded tests to account for this problem and consider patients to have insufficient effort only if they fail multiple tests. Furthermore, additional work is needed to determine the validity of using standardized neuropsychological tests of effort in schizophrenia. The current results pose the question of why individuals with schizophrenia are failing these tests: does poor performance on effort tests result from having low IQ and poor cognition to begin with, or does low effort partially explain poor neuropsychological test performance? These questions are impossible to answer using the current study design; however, paradigms that manipulate receipt of incentives for adequate neuropsychological test performance may shed light onto whether effort test failure reflects a cognitive deficit or motivational problem. Additionally, many tests determine insufficient effort by calculating discrepancy scores among various subtests to identify statistically improbable performance that is thought to signal insufficient effort. Although these procedures may validly capture insufficient performance in populations that have discrete profiles of strengths and weaknesses (e.g., specific deficits in recall, but not recognition memory), these discrepancy based calculations may be of questionable validity for schizophrenia patients who display a generalized neurocognitive deficit across multiple cognitive domains and a magnitude of impairment that is statistically improbable in other disorders. These considerations are particularly important when using embedded measures such as the RBANS EI, which may capture both genuine cognitive impairment and poor motivation. Further research is clearly needed to evaluate whether standardized neuropsychological measures of effort can be validly applied to individuals with schizophrenia, or whether new measures are needed to evaluate insufficient cognitive effort as it occurs in this population.

Acknowledgments

The authors would like to thank the participants who completed the study, as well as staff at the Maryland Psychiatric Research Center who contributed to data collection.

Role of Funding Source: Research supported in part by US National Institutes of Mental Health Grant R01MH080066 to Dr. Gold

Footnotes

Contributors: Gregory Strauss and James Gold designed the study. Statistical analyses and writing of the first draft of the manuscript were performed by Lindsay Morra, Sara Sullivan, and Gregory Strauss. All authors contributed to and approved the final manuscript.

Conflict of Interest: Authors have no conflicts of interest relevant to the current study.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Amador XF, Kirkpatrick B, Buchanan RW, Carpenter WT, Marcinko L, Yale SA. Stability of the diagnosis of deficit syndrome in schizophrenia. The American journal of psychiatry. 1999;156(4):637–639. doi: 10.1176/ajp.156.4.637. [DOI] [PubMed] [Google Scholar]
  2. Andreasen N. The Scale for the Assessment of Negative Symptoms (SANS) The University of Iowa; Iowa City, Iowa: 1983. [Google Scholar]
  3. Andreasen N. Scale for the Assessment of Positive Symptoms (SAPS), Iowa City. Iowa City: University of Iowa; 1984. [Google Scholar]
  4. Armistead-Jehle P, Hansen CL. Comparison of the Repeatable Battery for the Assessment of Neuropsychological Status Effort Index and stand-alone symptom validity tests in a military sample. Archives of clinical neuropsychology : the official journal of the National Academy of Neuropsychologists. 2011;26(7):592–601. doi: 10.1093/arclin/acr049. [DOI] [PubMed] [Google Scholar]
  5. Arnold G, Boone KB, Lu P, Dean A, Wen J, Nitch S, McPherson S. Sensitivity and specificity of finger tapping test scores for the detection of suspect effort. Clin Neuropsychol. 2005;19(1):105–120. doi: 10.1080/13854040490888567. [DOI] [PubMed] [Google Scholar]
  6. Avery R, Startup M, Calabria K. The role of effort, cognitive expectancy appraisals and coping style in the maintenance of the negative symptoms of schizophrenia. Psychiatry Res. 2009;167(1-2):36–46. doi: 10.1016/j.psychres.2008.04.016. [DOI] [PubMed] [Google Scholar]
  7. Back C, Brauer Boone K, Edwards C. The performance of schizophrenics on three cognitive tests of malingering, Rey 15-Item Memory test, Rey Dot Counting, and Hiscock Forced-Choice Method. Assessment. 1996;3:449–457. [Google Scholar]
  8. Blanchard JJC, C AS. The structure of negative symptoms within schizophrenia: implications for assessment. Schizophrenia Bull. 2006;32(2):238–245. doi: 10.1093/schbul/sbj013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Buchanan RW, Strauss ME, Kirkpatrick B, Holstein C, Breier A, Carpenter WT., Jr Neuropsychological impairments in deficit vs nondeficit forms of schizophrenia. Archives of general psychiatry. 1994;51(10):804–811. doi: 10.1001/archpsyc.1994.03950100052005. [DOI] [PubMed] [Google Scholar]
  10. Carpenter WT, Jr, Heinrichs DW, Wagman AM. Deficit and nondeficit forms of schizophrenia: the concept. The American journal of psychiatry. 1988;145(5):578–583. doi: 10.1176/ajp.145.5.578. [DOI] [PubMed] [Google Scholar]
  11. Cohen AS, Saperstein AM, Gold JM, Kirkpatrick B, Carpenter WT, Buchanan RW. Neuropsychology of the deficit syndrome: New data and meta-analysis of findings to date. Schizophrenia Bull. 2007;33(5):1201–1212. doi: 10.1093/schbul/sbl066. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Couture SM, Blanchard JJ, Bennett ME. Negative expectancy appraisals and defeatist performance beliefs and negative symptoms of schizophrenia. Psychiat Res. 2011;189(1):43–48. doi: 10.1016/j.psychres.2011.05.032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Dean AC, Victor TL, Boone KB, Arnold G. The relationship of IQ to effort test performance. The Clinical neuropsychologist. 2008;22(4):705–722. doi: 10.1080/13854040701440493. [DOI] [PubMed] [Google Scholar]
  14. Dickinson D. Digit symbol coding and general cognitive ability in schizophrenia: worth another look? Br J Psychiatry. 2008;193(5):354–356. doi: 10.1192/bjp.bp.108.049387. [DOI] [PubMed] [Google Scholar]
  15. Dickinson D, Harvey PD. Systemic hypotheses for generalized cognitive deficits in schizophrenia: a new take on an old problem. Schizophr Bull. 2009;35(2):403–414. doi: 10.1093/schbul/sbn097. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Dickinson D, Iannone VN, Wilk CM, Gold JM. General and specific cognitive deficits in schizophrenia. Biol Psychiatry. 2004;55(8):826–833. doi: 10.1016/j.biopsych.2003.12.010. [DOI] [PubMed] [Google Scholar]
  17. Dickinson D, Ragland JD, Gold JM, Gur RC. General and specific cognitive deficits in schizophrenia: Goliath defeats David? Biol Psychiatry. 2008;64(9):823–827. doi: 10.1016/j.biopsych.2008.04.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Dickinson D, Ramsey ME, Gold JM. Overlooking the obvious: a meta-analytic comparison of digit symbol coding tasks and other cognitive measures in schizophrenia. Arch Gen Psychiatry. 2007;64(5):532–542. doi: 10.1001/archpsyc.64.5.532. [DOI] [PubMed] [Google Scholar]
  19. Duncan A. The impact of cognitive and psychiatric impairment of psychotic disorders on the test of memory malingering (TOMM) Assessment. 2005;12(2):123–129. doi: 10.1177/1073191105275512. [DOI] [PubMed] [Google Scholar]
  20. Egeland J, Sundet K, Rund BR, Asbjornsen A, Hugdahl K, Landro NI, Lund A, Roness A, Stordal KI. Sensitivity and specificity of memory dysfunction in schizophrenia: a comparison with major depression. J Clin Exp Neuropsychol. 2003;25(1):79–93. doi: 10.1076/jcen.25.1.79.13630. [DOI] [PubMed] [Google Scholar]
  21. Fenton WS, McGlashan TH. Testing systems for assessment of negative symptoms in schizophrenia. Archives of general psychiatry. 1992;49(3):179–184. doi: 10.1001/archpsyc.1992.01820030011002. [DOI] [PubMed] [Google Scholar]
  22. Fervaha G, Foussias G, Agid O, Remington G. Motivational and neurocognitive deficits are central to the prediction of longitudinal functional outcome in schizophrenia. Acta psychiatrica Scandinavica. 2014;130(4):290–299. doi: 10.1111/acps.12289. [DOI] [PubMed] [Google Scholar]
  23. Fioravanti M, Carlone O, Vitale B, Cinti ME, Clare L. A meta-analysis of cognitive deficits in adults with a diagnosis of schizophrenia. Neuropsychol Rev. 2005;15(2):73–95. doi: 10.1007/s11065-005-6254-9. [DOI] [PubMed] [Google Scholar]
  24. Gierok SD, Dickson AL, Cole JA. Performance of forensic and non-forensic adult psychiatric inpatients on the Test of Memory Malingering. Arch Clin Neuropsychol. 2005;20(6):755–760. doi: 10.1016/j.acn.2005.04.008. [DOI] [PubMed] [Google Scholar]
  25. Gold JM, Queern C, Iannone VN, Buchanan RW. Repeatable battery for the assessment of neuropsychological status as a screening test in schizophrenia I: sensitivity, reliability, and validity. Am J Psychiat. 1999;156(12):1944–1950. doi: 10.1176/ajp.156.12.1944. [DOI] [PubMed] [Google Scholar]
  26. Gorissen M, Sanz JC, Schmand B. Effort and cognition in schizophrenia patients. Schizophrenia research. 2005;78(2-3):199–208. doi: 10.1016/j.schres.2005.02.016. [DOI] [PubMed] [Google Scholar]
  27. Hawk AB, Carpenter WT, Jr, Strauss JS. Diagnostic criteria and five-year outcome in schizophrenia. A report from the International Pilot Study of schizophrenia. Archives of general psychiatry. 1975;32(3):343–347. doi: 10.1001/archpsyc.1975.01760210077005. [DOI] [PubMed] [Google Scholar]
  28. Horan WP, Kring AM, Gur RE, Reise SP, Blanchard JJ. Development and psychometric validation of the Clinical Assessment Interview for Negative Symptoms (CAINS) Schizophrenia research. 2011;132(2-3):140–145. doi: 10.1016/j.schres.2011.06.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Horan WP, Rassovsky Y, Kern RS, Lee J, Wynn JK, Green MF. Further support for the role of dysfunctional attitudes in models of real-world functioning in schizophrenia. Journal of psychiatric research. 2010;44(8):499–505. doi: 10.1016/j.jpsychires.2009.11.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Hunt S, Root JC, Bascetta BL. Effort testing in schizophrenia and schizoaffective disorder: validity indicator profile and test of memory malingering performance characteristics. Arch Clin Neuropsychol. 2014;29(2):164–172. doi: 10.1093/arclin/act069. [DOI] [PubMed] [Google Scholar]
  31. Kimhy D, Yale S, Goetz RR, McFarr LM, Malaspina D. The factorial structure of the schedule for the deficit syndrome in schizophrenia. Schizophr Bull. 2006;32(2):274–278. doi: 10.1093/schbul/sbi064. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Kirkpatrick B, Buchanan RW, Ross DE, Carpenter WT. A separate disease within the syndrome of schizophrenia. Archives of general psychiatry. 2001;58(2):165–171. doi: 10.1001/archpsyc.58.2.165. [DOI] [PubMed] [Google Scholar]
  33. Kirkpatrick B, Buchanan RW, McKenney PD, Alphs LD, Carpenter WT., Jr The Schedule for the Deficit syndrome: an instrument for research in schizophrenia. Psychiat Res. 1989;30(2):119–123. doi: 10.1016/0165-1781(89)90153-4. [DOI] [PubMed] [Google Scholar]
  34. Kirkpatrick B, Strauss GP, Nguyen L, Fischer BA, Daniel DG, Cienfuegos A, Marder SR. The brief negative symptom scale: psychometric properties. Schizophr Bull. 2011;37(2):300–305. doi: 10.1093/schbul/sbq059. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Kraepelin E. In: Dementia Praecox and Paraphrenia. B RM, translator. New York: 1919. [Google Scholar]
  36. Moore RC, Davine T, Harmell AL, Cardenas V, Palmer BW, Mausbach BT. Using the repeatable battery for the assessment of neuropsychological status (RBANS) effort index to predict treatment group attendance in patients with schizophrenia. J Int Neuropsychol Soc. 2013;19(2):198–205. doi: 10.1017/S1355617712001221. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Pivovarova E, Rosenfeld B, Dole T, Green D, Zapf P. Are Measures of Cognitive Effort and Motivation Useful in Differentiating Feigned from Genuine Psychiatric Symptoms? International Journal of Forensic Mental Health. 2009;8:271–278. [Google Scholar]
  38. Reichenberg A, Harvey PD. Neuropsychological impairments in schizophrenia: Integration of performance-based and brain imaging findings. Psychol Bull. 2007;133(5):833–858. doi: 10.1037/0033-2909.133.5.833. [DOI] [PubMed] [Google Scholar]
  39. Schroeder RW, Marshall PS. Evaluation of the appropriateness of multiple symptom validity indices in psychotic and non-psychotic psychiatric populations. Clin Neuropsychol. 2011;25(3):437–453. doi: 10.1080/13854046.2011.556668. [DOI] [PubMed] [Google Scholar]
  40. Silverberg ND, Wertheimer JC, Fichtenberg NL. An effort index for the Repeatable Battery For The Assessment Of Neuropsychological Status (RBANS) Clin Neuropsychol. 2007;21(5):841–854. doi: 10.1080/13854040600850958. [DOI] [PubMed] [Google Scholar]
  41. Strauss GP, Hong LE, Gold JM, Buchanan RW, McMahon RP, Keller WR, Fischer BA, Catalano LT, Culbreth AJ, Carpenter WT, Kirkpatrick B. Factor structure of the Brief Negative Symptom Scale. Schizophrenia research. 2012;142(1-3):96–98. doi: 10.1016/j.schres.2012.09.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Strauss GP, Horan WP, Kirkpatrick B, Fischer BA, Keller WR, Miski P, Buchanan RW, Green MF, Carpenter WT., Jr Deconstructing negative symptoms of schizophrenia: avolition-apathy and diminished expression clusters predict clinical presentation and functional outcome. Journal of psychiatric research. 2013;47(6):783–790. doi: 10.1016/j.jpsychires.2013.01.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Strauss GP, Morra LF, Sullivan SK, Gold JM. The Role of Low Cognitive Effort and Negative Symptoms in Neuropsychological Impairment in Schizophrenia. Neuropsychology. 2014 doi: 10.1037/neu0000113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Wilk CM, Gold JM, Humber K, Dickerson F, Fenton WS, Buchanan RW. Brief cognitive assessment in schizophrenia: normative data for the Repeatable Battery for the Assessment of Neuropsychological Status. Schizophrenia research. 2004;70(2-3):175–186. doi: 10.1016/j.schres.2003.10.009. [DOI] [PubMed] [Google Scholar]

RESOURCES