Abstract
Deficits in goal-directed decision making and motivation are hallmark characteristics of several neuropsychiatric disorders, including schizophrenia (SZ) and major depressive disorder (MDD). Studies using effort-based decision-making tasks have shown that both patients with SZ and MDD invest less physical effort in order to obtain rewards. However, how these motivational deficits relate to clinically assessed symptom dimensions such as apathy remains controversial. Using a grip-strength-based effort discounting task we assessed effort-based decision-making behavior in healthy controls (HC) (N = 18), patients with SZ (N = 42), and MDD (N = 44). We then investigated how effort discounting relates to different symptom dimensions. There were no differences in effort discounting between HC participants and patients with SZ or MDD. In addition, we did not observe a correlation between effort discounting and negative symptoms (NS) in patients with SZ or MDD. In conclusion, the current study does not support an association between effort discounting and NS in SZ or MDD. Further studies are needed to investigate effort discounting and its relation to psychopathological dimensions across different neuropsychiatric disorders.
Keywords: schizophrenia, major depressive disorder, effort discounting, negative symptoms
Introduction
Deficits in goal-directed decision making and motivation resulting in suboptimal clinical outcomes characterize several neuropsychiatric disorders, including schizophrenia (SZ) and major depressive disorder (MDD).1,2 These impairments significantly contribute to reduced functioning and quality of life and therefore understanding the mechanisms of abnormal decision-making processes in different neuropsychiatric disorders is imperative to the development of potential treatments.3–5
Effort-based decision-making paradigms have proven to be a powerful tool for unraveling different aspects of motivational processes.6 In effort-based decision-making tasks, participants make several decisions on how much effort to exert (eg, squeeze a hand grip or press a button a certain number of times) to obtain a given level of reward.7 One of the most consistent findings across studies is that compared to healthy controls (HC), both patients with SZ8–14 and MDD15,16 are less willing to invest physical effort in order to obtain monetary rewards.
How abnormal effort-based decision-making processes relate to psychopathological symptom dimensions such as negative symptoms (NS) is less clear. NS, ie, impairments in motivation, social interactions, ability to experience pleasure, but also the reduction in expression of emotions and speech, constitute a separate symptom dimension in patients with SZ, and are highly relevant to clinical outcome and functional impairments.17,18 Although NS were initially described in patients with SZ,19 several domains of NS such as motivational deficits and anhedonia are also prevalent in many other psychiatric disorders such as MDD.5,20 While various studies have shown that in patients with SZ effort expenditure is associated with NS,8,10,12,21 several others have found no associations9,13,22 (for review see 23). In patients with MDD, reduced effort expenditure and allocation have been associated with anticipatory anhedonia15,24 and the number of effortful choices has been shown to relate to cognitive function.25 However, similar to studies in patients with SZ, these associations have not been consistently observed or reported in MDD.16,26
To specifically investigate the relationship between decision making and NS, our group has recently developed a task that allows to measure effort discounting.27 This effort discounting task is based on a grip strength paradigm. Study participants first squeeze a hand grip with maximal strength for calibration. During the task, participants then make a series of decisions about whether to exert no effort for a small reward or a higher effort for a larger reward. This allows for a subjective measure of how monetary reward is devalued in proportion to effort. Hartmann et al showed that in patients with SZ, increased effort discounting was strongly correlated with lower apathy but not with diminished expression.27 In other words, apathetic patients were less likely to invest physical effort to obtain a reward.
In recent years, an increasing number of studies has begun disentangling the complex interactions between reward and motivation across diagnostic entities. However, studies directly comparing different neuropsychiatric disorders such as MDD and SZ, aiming at teasing apart common and different mechanisms of effort-based decision making and specifically how these relate to NS remain sparse.
The overall aim of the present study was to compare the discounting of monetary reward by physical effort between patients with SZ, MDD, and HC and investigate in these disorders how effort discounting relates to motivational deficits and anhedonia. We hypothesized that patients with SZ and MDD with higher apathy scores make fewer effortful choices for higher rewards than control participants. Based on previous studies,16,26 we further hypothesized patients with MDD to make fewer effortful choices than HC. Additionally, we expected patients’ performance to relate to anhedonia, ie, higher anhedonia levels would correlate with fewer effortful choices. Lastly, in an exploratory approach, we investigated potential associations between effort discounting and other psychopathological dimensions such as positive symptoms, symptoms of disorganization, and cognition.
Methods
Forty-two patients meeting the DSM-IV28 criteria for SZ, 44 patients with MDD and 18 HC participants were recruited. More patients (SZ and MDD) than controls were recruited to have adequate power for the correlational analyses. Patients were recruited from outpatient and inpatient units of the Psychiatric Hospital of the University of Zurich and affiliated institutions. Diagnoses were confirmed by conducting the Mini-International Neuropsychiatric Interview.29 All patients were clinically stable and under a stable dose of medication for at least 2 weeks prior to testing. Inpatients were at the end of their hospitalization and engaged in a multimodal therapy program and activities outside the hospital.27,30–33 The average duration of hospitalization for patients with SZ and MDD in Swiss psychiatric hospitals is longer than in most other countries, so the majority of patients would have been treated as outpatients in other health care systems. HC were recruited from the community via advertisement. All participants gave written informed consent and the project was approved by the Ethics Committee of the Canton of Zurich. The inclusion age was between 18 and 65 years.
We excluded patients with any disorder other than the abovementioned DSM-IV Axis I disorders (thus, patients with SZ did not have comorbid MDD and MDD patients did not have a history of psychotic disorder), use of benzodiazepines (except lorazepam equivalents of 1 mg or less per day) and acute psychotic symptoms. Chlorpromazine equivalents were calculated according to.34 Participants with any alcohol use disorder based on lifetime criteria and participants with a current abuse or dependency of cannabis or any other substance abuse were excluded. HC were excluded if any lifetime psychiatric diagnosis was present in the structured Mini-International Neuropsychiatric Interview.
Assessment of Psychopathology and Cognition
NS were assessed using the Brief Negative Symptom Scale (BNSS).35,36 Positive symptoms, symptoms of disorganization, and NS were assessed using the 5-factor model of the Positive and Negative Syndrome Scale (PANSS)37: The positive factor was computed from items P1, P3, P5, G9, the disorganized factor was calculated based on items P2, N5, G11, and the NS factor was calculated based on items N1, N2, N3, N4, N6, G7. Cognition was assessed with the Brief Neurocognitive Assessment (BNA).38 With the BNA, a cognitive score is computed for each participant by combining results from the Letter-Number-Span Test and the Symbol Coding Test. The BNA was shown to be highly correlated with the MATRICS Consensus Cognitive Battery (MCCB)39 and has similarly good validity criteria.40 Global level of functioning was assessed with the Personal and Social Performance (PSP) Scale.41
Effort Task
The experimental procedure closely followed the one described in Hartmann et al.27 Physical effort was assessed by an isometric dynamometer (Sensory-Motor Systems Laboratory, ETH Zurich; measuring range: 0–600 N). After the instructions were delivered, the experimental session began with 2 calibration trials. Participants had to squeeze the dynamometer with their dominant hand as hard as possible for 3.5 s. Maximum voluntary contraction (MVC) was defined as the mean force exerted during the period 1–3.5 s after the go signal in the 2 trials. The experiment began after 5 training trials.
On every trial, participants had to make a choice between (a) exerting no effort and receiving a default small amount of money (1 CHF, equivalent to 1 USD) or (b) exerting one of 4 levels of effort and receiving one of the larger rewards. The 4 levels of effort used were 40%, 60%, 80%, and 100% MVC. They were paired with one of the following rewards: 1.5 CHF, 2 CHF, 2.5 CHF, 3 CHF, and 5 CHF. Each level of effort appeared with each reward 4 times, resulting in a total of 80 trials (some participants [N = 57] performed an additional number of trials containing lower and higher reward values, in order to estimate reward discounting, see below). Participants could take a self-paced break after 40 trials. All trials were randomized for each participant, such that the same condition could not occur more than 2 times in a row.
Maximum choice time was not restricted. If an effortful option was chosen, participants had to maintain the given effort level for 3.5 s in order to obtain the reward (effective effort was quantified for 2.5 s: 1–3.5 s). The individual adjustment of the effort level based on the calibration trials ensured that the participants were physically capable of maintaining each effort level. To avoid any effects of loss aversion,42,43 participants were given the default reward of 1 CHF if they failed to maintain the required effort level. This approach was chosen, because in line with prospect theory the 1 CHF payout in the low-effort condition can be viewed as the reference point for the decision between the low-effort and the high-effort condition. A payout of 0 CHF in the high-effort condition would thus be perceived as a monetary loss by the participant. Conversely, the effortful option would be viewed as risky because in case of failure to achieve the required effort it could lead to less reward than the no reward option. We aimed to avoid any confounding effects of loss and risk aversion to isolate the effort component as much as possible. Visual feedback was provided about the required level of effort, the duration of the trial, and participants’ level of effort in real time. If the effortless choice was made, participants simply waited for 3.5 s before receiving the 1 CHF reward. Time costs were thus held constant between the effortful and effortless options.
After participants completed all the experimental trials, 5 trials were selected at random for the final payout. Participants then performed another 2 calibration trials at the end of the task, identical to the ones described above, in order to control for the effects of fatigue.
Finally, using a visual-analogue scale, participants answered a series of questions about (a) how satisfying they found the 5 offered rewards used during the experiment; (b) how strenuous they found the 4 effort levels; (c) how advantageous they perceived the different effort-reward combinations presented during the task.
Data Processing
As the reward provided for the effortful option increases within a given effort level, participants are expected to switch from choosing the effortless option to choosing the effortful option, even though the trials are randomized (figure 1A). The task thus allows establishing the amount of reward for which the participant is equally likely to choose the effortless and the effortful option. This amount is formalized as the “indifference point.” Indifference points were obtained for each effort level by fitting a logistic function (Matlab (2018), ezyfit function) to the fraction of effortful choices across all reward levels (figures 1A and 1B). The obtained indifference points allowed calculating the relative subjective value (SV) of reward, ie, how each of the 4 effort levels reduced (discounted) the SV of monetary reward in each participant. For each effort level this was achieved by dividing the default 1 CHF reward by the corresponding indifference point value (figure 1C).
To measure overall effort discounting, following Hartmann et al,27 we used the area under the curve (AUC). For each participant, we computed the AUC of the relative SVs over the 4 effort levels (figure 1C). Smaller AUC represents stronger effort discounting, ie, increasing effort leads to a stronger devaluation of reward and decrease in effortful choices.
During the experiment, indifference points for each effort level were calculated in real time. If, for a given effort level the participant failed to switch by never choosing the effortful option or by always choosing the effortful option (ie, their individual indifference points were outside the reward options provided), additional trials were presented for that effort level. Specifically, when a participant never chose the effortful option at a given effort level even at the usual maximum 5 CHF, the effortful option was presented with a higher than usual reward in 3 additional trials (6 CHF, 7 CHF, and 10 CHF). Conversely, if the participant always chose the effortful option even at the usual minimum of 1.5 CHF, 3 additional trials used smaller than usual rewards (1.20, 1.10, and 1.05 CHF reward) for that effort level. Trials were presented in fixed order: ascending when enticing the participant to choose the effortful option and descending when enticing the participant to choose the effortless option. A participant could receive a maximum number of 12 additional trials (resulting in a total of 92 trials). The additional trials made it possible to calculate indifference points for participants who failed to switch at a given effort level.
Data Analysis and Statistics
First, normal distribution was tested using the Shapiro-Wilk test. To test for differences between 2 groups, we used 2-sided Student t tests (if data were normally distributed) or Mann-Whitney U tests (if data were not normally distributed). For comparisons of more than 2 groups, we performed 1-way ANOVAs with Tukey’s test for post hoc comparisons. General linear models were used to test for the effects of covariates and Tukey’s correction was applied for follow-up of post hoc effects. For correlations between task parameters and symptom domains, Pearson (for normally distributed data) or Spearman (if data were not normally distributed) correlation coefficients were used. All P values <.05 were considered statistically significant. No correlations were calculated for the HC group, based on the low level of psychopathological symptoms. Statistical analyses were computed with SPSS version 25 (IBM Corp., SPSS Inc., Chicago, IL), GraphPad Prism software 8 (GraphPad Software Inc.), and R (R Core Team 2020).
Results
Clinical variables and psychopathological assessment parameters are summarized in table 1.
Table 1.
HC (n = 18) | SZ (n = 42) | MDD (n = 44) | Statistics | |||
---|---|---|---|---|---|---|
Test Statistics | Post Hoc | |||||
Clinical variables | ||||||
Age (years) | 33.00 ± 9.49 | 33.19 ± 10.30 | 36.18 ± 11.06 | F(2, 101) = 1.07 | 1P = .347 | |
Sex (male/female) | 9/9 | 30/12 | 18/26 | χ 2 = 8.28 | 2 P = .016 |
aP = .111 bP = .512 cP = .004 |
Education (years) | 14.25 ± 2.20 | 12.39 ± 3.97 | 14.51 ± 3.16 | F(2, 101) = 4.60 | 1 P = .012 |
aP = .131 bP = .959 cP = .013 |
Inpatients/outpatients | — | 19/23 | 20/24 | χ 2 = 0.75 | 2P = .388 | |
Monthly income (CHF) | 4388.89 ± 2054.80 | 2792.68 ± 1860.70 | 3250.00 ± 2190.10 | F(2, 100) = 3.83 | 1 P = .025 |
aP = .018 bP = .119 cP = .558 |
Handedness (right/left/both) | 15/1/2 | 33/8/1 | 43/0/1 | χ 2 = 13.18 | 2 P = .010 |
aP = .175 bP = .091 cP = .010 |
Number of psychotic episodes | 0 | 4.98 ± 5.14 | 0 | F(2, 101) = 28.91 | 1 P < .001 |
aP < .001
bP = 1.000 cP < .001 |
Number of depressive episodes | 0 | 0.10 ± 0.30 | 3.57 ± 3.16 | F(2, 101) = 36.36 | 1 P < .001 |
aP > .999 bP = 1.000 cP < .001 |
Number of hospitalizations | 0 | 5.26 ± 5.42 | 1.23 ± 1.31 | F(2, 101) = 19.81 | 1 P < .001 |
aP < .001
bP < .001 cP = 1.000 |
Chlorpromazine equivalents (mg/day) | 0 | 477.64 ± 391.36 | 2.28 ± 11.19 | F(2, 101) = 45.59 | 1 P < .001 |
aP < .001
bP > .999 cP < .001 |
BNSS | ||||||
BNSS apathy | 0.78 ± 1.11 | 18.43 ± 9.36 | 20.23 ± 7.48 | F(2, 101) = 43.64 | 1 P < .001 |
aP < .001
bP < .001 P = .529 |
BNSS diminished expression | 0.22 ± 0.43 | 9.64 ± 7.85 | 6.98 ± 6.34 | F(2, 101) = 13.28 | 1 P < .001 |
aP < .001
bP = .001 cP = .143 |
PANSS | ||||||
PANSS positive factor | — | 5.74 ± 2.43 | 4.36 ± 1.06 | U = 559 | 3 P < .001 | |
PANSS negative factor | — | 15.40 ± 6.85 | 13.50 ± 5.20 | U = 801 | 3P = .287 | |
PANSS disorganized factor | — | 5.62 ± 2.25 | 4.36 ± 1.30 | U = 621.5 | 3 P = .007 | |
CDSS (total) | — | 3.07 ± 3.75 | 12.09 ± 5.13 | U = 136 | 3 P < .001 | |
BDI (total) | 1.33 ± 2.63 | 15.00 ± 9.68 | 25.52 ± 12.92 | F(2, 98) = 35.08 | 3 P < .001 |
aP < .001
bP < .001 cP < .001 |
TEPS | ||||||
TEPS anticipatory | 46.89 ± 5.43 | 42.70 ± 8.28 | 36.48 ± 8.84 | F(2,96) = 12.31 | 1 P < .001 |
aP = .177 bP < .001 cP = .003 |
TEPS consummatory | 37.61 ± 7.02 | 36.51 ± 8.00 | 32.70 ± 7.57 | F(2, 96) = 3.77 | 1 P = .026 |
aP = .871 bP = .061 cP = .070 |
Cognition (BNA) | −0.04 ± 0.68 | −0.91 ± 0.69 | −0.12 ± 0.84 | F(2,98) = 13.96 | 1 P < .001 |
aP < .001
bP = .920 cP < .001 |
Global functioning (PSP total) | 96.89 ± 4.78 | 52.24 ± 14.55 | 57.57 ± 16.71 | F(2,101) = 64.18 | 1 P < .001 |
aP < .001
bP < .001 cP = .206 |
Effort task performance | ||||||
MVC (N) | 342.69 ± 100.18 | 309.61 ± 98.93 | 285.71 ± 92.49 | F(2, 101) = 2.30 | 1P = .105 | |
Time to reach MVC (s) | 42.31 ± 12.39 | 38.63 ± 15.52 | 39.42 ± 12.22 | F(2, 101) = 0.46 | 1P = .632 | |
Fatigue (MVC1-MVC2) | −17.37 ± 54.09 | −21.87 ± 75.16 | −6.32 ± 51.59 | F(2,100) = 0.68 | 1P = .508 | |
Final payout (in CHF) | 12.19 ± 2.99 | 12.57 ± 3.29 | 12.22 ± 2.83 | F(2,101) = 0.18 | 1P = .839 | |
Total trial number | 81.33 ± 1.85 | 83.50 ± 4.03 | 83.48 ± 3.93 | F(2,101) = 2.52 | 1P = .086 | |
% successful effortful trials | 69.09 ± 11.39 | 71.46 ± 18.57 | 71.20 ± 14.54 | F(2, 101) = 0.15 | 1P = .860 |
Statistics: 1One-way analysis of variance, 2chi-square test, 3Mann-Whitney U test. Post hoc (Tukey): aHC vs SZ, bHC vs MDD, cSZ vs MDD. Note: BDI, Beck Depression Inventory; BNA, Brief Neurocognitive Assessment; BNSS, Brief Negative Symptom Scale; CDSS, Calgary Depression Scale for Schizophrenia; CHF, Swiss Francs; HC, healthy controls; MDD, major depressive disorder; MVC, maximum voluntary contraction; PANSS, Positive and Negative Syndrome Scale; PSP, Personal and Social Performance Scale; SZ, schizophrenia; TEPS, Temporal Experience of Pleasure Scale. Significant results are indicated in bold. Data are presented as mean ± SD.
Task Performance
We first investigated general effort task performance (table 1). There were no differences between the 3 groups in the overall number of trials completed (F(2,101) = 2.52, P = .086) or number of chosen effortful trials (F(2,101) = 0.87, P = .4). The proportion of chosen and successfully completed effortful trials for each group was as follows: HC: 69.1%, SZ: 71.5%, MDD: 71.2% (F(2, 101) = 0.15, P = .860). When analyzing the proportion of successful effortful trials for each effort and reward condition during the main experiment (ie, without additional trials), there was a significant reward × group interaction (F(2,101) = 8.4, P < .001), with post hoc tests revealing that both patients with SZ and MDD chose significantly more effortful trials in the 1.5 CHF reward condition than the control group (HC vs SZ P < .001, HC vs MDD P = .013, SZ vs MDD P = .15). Thus, both patient groups chose to perform effort for minimal reward more frequently than control participants (figures 2 and 3).
The 3 groups did not differ on MVC before the experiment (F(2, 101) = 2.30, P = .105), time to reach MVC (F(2, 101) = 0.46, P = .632), fatigue (F(2,100) = 0.68, P = .508), or final payout (F(2,101) = 0.18, P = .839). Fatigue was assessed as the difference between the force exerted in the calibration trials before and after the experiment. Additionally, to assess fatigue during the experiment, we binned the 80 trials by 20, and compared the number of chosen effortful trials and the number of effortful trials successfully completed between the 4 bins across groups. No significant differences were observed (all R2 < 0.011, all P > .8).
Group Differences in Effort Discounting
Similar to Hartmann et al27 all groups on average showed effort discounting (ie, the average relative SVs decreased with increasing effort, figure 4). This result was confirmed by an ANOVA on a linear mixed effect model on SV values with group, SV and gender as fixed effects and subject as a random effect. A main effect of SV (F(3,288) = 91.2, P < .001) was observed, but no main effect of group (F(2,96) = 1.3, P = .27), or gender (F(1,96) = 0.1, P = .7).
Importantly, we observed a group × SV interaction (F(6,288) = 2.67, P = .02). Numerically, for the 100% effort condition participants with SZ were more willing to exert effort than the 2 other groups, with post hoc comparisons not reaching significance (SZ vs HC: P = .071; SZ vs MDD: P = .07). Post hoc contrasts also showed that MDD participants significantly reduced SV values with all increases in effort level. This was not the case for SZ and HC participants, whose SV values did not significantly differ between 40% and 60% effort and 80% and 100% effort (for both groups, there was a significant difference between 60% and 80% effort).
Despite the general trend of SV values to drop with increasing effort, individually, not all subjects showed this pattern (ie, main SV effect). In this substantial number of participants (HC = 1, SZ = 10, MDD = 6), SVs for the highest effort level (100%) were higher or equal to those for the lowest effort level (40%).
Moreover, other combinations of nondecreasing SV values (eg, SV(80% effort) > SV(60% effort)) were found for an additional: 2 HC, 13 SZ, and 8 MDD participants. Thus, a high proportion of SZ participants did not perform the task as expected, either because they always chose the effortful option irrespective of reward (resulting in SV values of 1 for all effort levels) or because they discounted reward more for lower effort than higher effort. A chi-square test comparing the total number of such participants within each group (HC = 3/18, SZ = 23/42, MDD = 14/42) showed a significant main effect of group (X2(2) = 8.7, P = .01), the only significant post hoc difference between SZ and HC participants (P = .015).
Association Between Effort Discounting and NS
We investigated the correlation between effort discounting and NS in both patients with SZ and MDD (table 2). We did not observe a significant correlation between BNSS apathy and effort valuation (AUC) in patients with SZ (r(p) = 0.150, P = .344) or MDD (r(s) = 0.238, P = .120). Further, there was no correlation between BNSS diminished expression and AUC in patients with SZ (r(s) = 0.266, P = .088) or MDD (r(s) = 0.226, P = .141), nor between AUC and the Temporal Experience of Pleasure Scale (TEPS) anticipatory (SZ: r(s) = −0.321, P = .053; MDD: r(s) = −0.148, P = .337) or TEPS consummatory subscale (SZ: r(s) = −0.048, P = .776; MDD: r(s) = −0.113, P = .464). AUC did not correlate with depressive symptoms as measured by clinician administered Calgary Depression Scale for Schizophrenia (SZ: r(p) = 0.301, P = .053; MDD: r(s) = 0.113, P = .467) or the participant rated Beck Depression Inventory (BDI) (SZ: r(p) = 0.111, P = .501; MDD: r(s) = 0.092, P = .554).
Table 2.
SZ (n = 42) | MDD (n = 44) | |
---|---|---|
Clinical variables | ||
Number of hospitalizations | r(s) = 0.030, P = .852 | r(s) = 0.238, P = .121 |
Monthly income (CHF) | r(s) = 0.065, P = .687 | r(s) = −0.372, P = .013 |
Chlorpromazine equivalents (mg/day)1 | r(s) = −0.053, P = .741 | r(s) = −0.095, P = .540 |
BNSS | ||
BNSS apathy | r(p) = 0.150, P = .344 | r(s) = 0.238, P = .120 |
BNSS diminished expression | r(s) = 0.266, P = .088 | r(s) = 0.226, P = .141 |
PANSS | ||
PANSS positive factor | r(s) = 0.356, P = .021 | r(s) = −0.099, P = .524 |
PANSS negative factor | r(s) = 0.132, P = .405 | r(s) = 0.247, P = .106 |
PANSS disorganized factor | r(s) = 0.336, P = .030 | r(s) = 0.057, P = .713 |
CDSS (total) | r(p) = 0.301, P = .053 | r(s) = 0.113, P = .467 |
BDI (total) | r(p) = 0.111, P = .501 | r(s) = 0.092, P = .554 |
TEPS | ||
TEPS anticipatory | r(s) = −0.321, P = .053 | r(s) = −0.148, P = .337 |
TEPS consummatory | r(s) = −0.048, P = .776 | r(s) = −0.113, P = .464 |
Cognition (BNA) | r(p) = −0.045, P = .780 | r(s) = −0.038, P = .809 |
Global functioning (PSP total) | r(p) = −0.172, P = .277 | r(s) = −0.051, P = .742 |
Effort task performance | ||
Maximum voluntary contraction (MVC) | r(p) = −0.222, P = .158 | r(s) = −0.027, P = .864 |
Anticipatory reward pleasure (average, VAS) | r(p) = 0.054, P = .734 | r(s) = 0.336, P = .026 |
Perceived effort (average, VAS) | r(s) = −0.102, P = .522 | r(s) = 0.089, P = .565 |
Statistics: Pearson correlation coefficients r(p) were calculated for normally distributed data, Spearman correlation coefficients r(s) were used for nonnormally distributed data. Note: BDI, Beck Depression Inventory; BNA, Brief Neurocognitive Assessment; BNSS, Brief Negative Symptom Scale; CDSS, Calgary Depression Scale for Schizophrenia; CHF, Swiss Francs; HC, healthy controls; MDD, major depressive disorder; MVC, maximum voluntary contraction; PANSS, Positive and Negative Syndrome Scale; PSP, Personal and Social Performance Scale; SZ, schizophrenia; TEPS, Temporal Experience of Pleasure Scale; VAS, Visual Analogue Scale. Significant results are indicated in bold. 1No correlations of chlorpromazine equivalents were calculated in patients with MDD because only 2 participants had values >0.
Association of Covariates With Effort Discounting
Next, we performed correlational analyses between effort valuation (AUC) and other covariates assessed in our study (table 2). We found no correlations between composite cognitive score (as assessed by the BNA) and AUC (SZ: r(p) = −0.045, P = .780); MDD: r(s) = −0.038, P = .809)). Interestingly, there was a positive correlation between the PANSS disorganized factor and AUC in patients with SZ (r(s) = 0.336, P = .030) but not MDD (r(s) = 0.057, P = .713). There was also a positive correlation between the PANSS positive factor and AUC in SZ (r(s) = 0.356, P = .021) but not in MDD (r(s) = −0.099, P = .524).
Taken together these analyses suggest that SZ participants showing higher levels of disorganization and positive symptoms also showed counterintuitive task performance, ie, made more effortful choices, particularly for lower rewards.
No associations of task performance with psychopathology were observed in the MDD group.
Self-report Measures of Monetary Reward Valuation and Perceived Effort
Linear mixed effect models were fit to the subjective valuation scores with group, either effort or reward and their interaction as fixed effects and subject as random effect. The 3 groups did not differ on their subjective evaluation of effort difficulty (F(2,101) = 0.56, P = .57) and displayed a main effect of effort, judging higher levels of effort as more strenuous (F(3,303) = 589.2, P < .001). There was no group × effort interaction (F(6,303) = 1.6, P = .1).
For subjective evaluation of reward a main effect of group was found (F(2,98) = 4.9, P = .009), with participants with MDD judging rewards as more satisfying than participants with SZ (P = .0095). A main effect of reward indicated that all participants increased their appreciation of the reward as its value increased (F(7,686) = 161.3, P < .001). There was no interaction (F(14,686) = 1.6, P = .08).
Finally, when assessing how advantageous every effort-reward combination was rated, we observed a significant group × reward interaction (F(8,1919) = 4.1, P < .0001). Post hoc comparisons showed that the SZ group judged the 1.5 CHF reward level as more advantageous than both HC (P = .003) and MDD (P = .0095) participants, and the 2 CHF reward level as more advantageous than HC (P = .01) participants. This result mirrors the higher number of effortful trials selected by SZ participants for the 1.5 CHF lowest reward condition. Thus, this reward value was judged as more satisfactory by SZ participants and resulted in more effortful choices.
Further Effects of Covariates on Effort Discounting
To further explore the impact of positive and disorganization symptoms, as well as subjective ratings on effort discounting in SZ patients we performed a linear regression to predict SV values. For this, we averaged the ratings for the lowest reward (1.5 CHF) across effort levels. As stated above, it is expected that SV values decrease with increasing effort, and thus SV for 100% effort is expected to be lower than SV for 40% effort. We thus computed a composite SV score by subtracting SV for 100% from SV for 40% effort. An ANOVA on this regression showed that the only significant result obtained was for PANSS disorganization score (P = .03). Thus, disorganization appears to be the most robust predictor of patients’ performance on the task and might account for the unusual performance pattern observed.
Comparison to Hartmann et al
In order to better understand any differences between the participants in the current HC and SZ groups and those in our previous work, we directly compared the 4 groups in a linear mixed effect model with SV, group and study as fixed effects and subjects as random effect.27 An ANOVA on this model yielded a significant effect of SV (F(3,349.67) = 109.1, P < .0001) and study (F(1,421.6) = 14.3, P = .0002), and group × study (F(1,355.8) = 15.7, P < .0001) and SV × study interactions (F(3,349.7) = 4.1, P = .007), figure 5. The group × study interaction showed no difference between HC participants in 2015 and 2021 (P = .86), whereas SZ participants in 2021 had significantly higher SV scores than 2015 SZ participants (P < .0001). The SV × study interaction showed that in 2021 the SVs for 80% and 100% effort levels were significantly higher than in 2015. Taken together these results indicate that the differences between the 2 cohorts stem from reduced effort discounting in the 2021 SZ group.
Analyses Excluding Outliers
We performed similar analyses as described above while excluding “non-discounter” participants—ie, those whose SV 100% values were equal or higher than their SV40% values. These analyses are presented in Supplementary Material.
Discussion
The main finding of the current study was that we did not observe differences in effort discounting between HC participants and patients with SZ or MDD and that there were no associations between effort discounting and NS in patients with SZ or MDD.
Most notably, we did not observe the previously reported association between apathy and effort discounting.27 While there were no obvious methodological differences (eg, the same instructions and a similar dynamometer were used) or variations in sociodemographic or clinical characteristics of the 2 cohorts, the most obvious difference was the high proportion of our SZ participants (and a somewhat lower proportion of MDD participants) who did not show effort discounting, ie, chose the effortful option irrespective of reward or chose to perform more effort for less reward. This is in sharp contrast with the study of Hartmann et al27 where only 1 participant failed to show discounting. This result is surprising, given that the same instructions were delivered to participants in both studies and it was ensured that participants exerted maximal effort during the calibration phase, so that task difficulty was adequately individually adapted. A recent study using an effort-based task (in the form of button presses) in patients with SZ and HC participants has also reported that patients made high-effort choices irrespective of the reward amount they could win (1 euro vs 1 cent).44 It thus appears that in certain circumstances patients with SZ tend to disregard effort to maximize reward. This strategy may appear rational, as indeed, if the high effort can be maintained throughout the task, this should result in maximizing reward. In our task this reward maximization strategy may be successful even when effort cannot consistently be maintained, because participants received the default reward of 1 CHF in failed trials. However, in both our task, and that of Pretus et al,44 reward maximization in patients was not observed: in the former we find no difference between patients and controls in the amount won, and in the latter patients ended up with lesser total gains than controls.
Although a direct comparison with other studies is often difficult, eg, because of differences in task design and high variability in scales used to assess clinical symptoms,45 it is worthwhile noting that several studies have also not observed an association between effort discounting and NS. Docx et al have also used a hand grip-based effort task to investigate effort discounting but have not found an association with motivational symptoms (assessed by the avolition or anhedonia subscale of the Scale for the Assessment of Negative Symptoms).22 These results are in line with 2 large studies that investigated and compared several different effort-based decision-making paradigms and also failed to find an association between a hand grip effort task and total NS (as assessed by the PANSS negative factor) or apathy, as assessed by the motivation and pleasure (MAP) scale of the Clinical Assessment Interview for Negative Symptoms (CAINS).11,21 In summary, while we could not replicate our previous findings, the results of the current study are in line with several other studies, reporting little evidence for an association between NS and physical effort discounting.
Since the majority of physical effort discounting studies have used button-pressing paradigms (in contrast to the present use of a hand grip-based task), it is important to compare the results of the present study with these paradigms. Using a balloon effort task, Gold et al have found a negative association between high-effort choices and total levels of NS.10 While in a subsequent study, the correlation with total NS (PANSS negative factor) could not be replicated, a negative correlation with MAP-CAINS was found.11,21 The same 2 studies have also investigated associations between the “Effort Expenditure for Rewards Task” (EEfRT) and PANSS negative factor/MAP-CAINS and found modest but significant correlations.11,21 However, while several studies have replicated the negative correlations between NS (either total NS and/or apathy) and the EEfRT8 in patients with SZ, several other studies have found no association9,12,13,46,47 or even a positive association.14
Our second aim was to assess effort-based decision making in patients with MDD. Compared to patients with SZ, there are fewer studies that have investigated how effort-based decision making relates to psychopathological symptoms and to our knowledge, no study has so far investigated effort discounting and NS in depressed patients. Some previous studies using the EEfRT and probabilistic reward task have shown MDD patients to make fewer effortful choices than HC.15,16,26 We have not observed this difference with our paradigm. MDD participants discounted reward similar to HC participants, requiring more reward to exert more effort. It should be noted that Yang et al15 only observed a significant difference between first episode MDD patients and HC participants, which might explain the difference compared to our study, in which MDD patients had on average more than 3 depressive episodes. No difference was observed between remitted MDD patients and HC, even though, similar to SZ patients, MDD patients also chose more effortful trials for the lowest reward than HC participants. Overall, however, MDD participants appeared motivated by monetary reward, unlike, eg, the patients in the study of Treadway et al, who did not make more effortful choices for more reward.16 Of note, we observed in patients with MDD but not SZ a significant correlation between effort discounting and monthly income, indicating that socioeconomic status is worth taking into consideration. Previous research using the EEfRT task has also shown that males choose a higher proportion of high effort—high reward options than females. Although not directly comparable, our results showed males required higher rewards to choose the most effortful options (100% MVC) as compared to 80% MVC, while females required the same amount of reward for both. Investigating potential sex differences in effort-based decision-making paradigms in future studies will therefore be important.
As mentioned above, few studies have investigated how effort tasks relate to depression symptomatology. Studies with button-pressing paradigms have shown reduced effort expenditure in patients with MDD compared to HC, but have mostly failed to show a relation to depressive symptomatology (assessed by BDI).15,16 In a grip task paradigm, patients with MDD showed less effort expenditure than HC, but effort discounting did not correlate with depressive symptoms, as measured by the BDI.48 Similarly, we found no correlations between task performance and symptomatology or cognitive ability in MDD patients.
One interesting finding of our study was the positive correlation between AUC and disorganization scores (as measured by the PANSS) in patients with SZ but not MDD. More disorganized patients tended to show less effort discounting (ie, made more effortful choices for less reward). Even though patients in our study did not show higher disorganization scores than the patients in Hartmann et al,27 their level of disorganization was related to their performance on the task. Interestingly, SZ participants made more effortful choices for low rewards and also judged those rewards as more satisfying than HC and MDD participants. In addition, the propensity to evaluate low reward-effort combinations more favorably was correlated with the PANSS disorganization score. Disorganization still correlated with AUC even after non-discounter participants were removed from the analysis (see Supplementary Material). Although compared to other symptom domains symptoms of disorganization have been less investigated, several studies have shown an association with functional outcomes,49,50 impaired quality of life,51 and lack of insight.52
In conclusion, the current study does not support an association between effort discounting and NS in SZ or MDD. Despite these negative results, research on effort-based decision making to develop objective measures of NS remains of interest for 2 main reasons. First, this type of task is suitable for a translational approach as similar tasks can be developed for humans and animals. Second, this type of task could provide an objective complement to more subjective clinical assessments. However, in order to move forward it will be essential to better understand the source of heterogeneity in the existing results. It would be important to better standardize tasks and psychopathological assessment in future studies. Larger sample sizes would be helpful to delineate individual differences in effort-based decision making. Overall, despite the negative results in the present study, further studies directly comparing different effort tasks between patients with SZ and MDD are needed, before definitive conclusions can be drawn.
Supplementary Material
Acknowledgments
This work was supported by a Postdoc. Mobility Fellowship of the Swiss National Science Foundation (F.C.), a Walter and Gertrud Siegenthaler Postdoctoral Fellowship (F.C.), and project support by the Hartmann Mueller Foundation (F.C.) and the Olga Mayenfisch Foundation (M.H.). F.K. was supported by an Early. Postdoc Mobility Fellowship of the Swiss National Science Foundation. S.K. was funded by the Swiss National Science foundation (10001CL_169783) The authors have declared that there are no conflicts of interest in relation to the subject of this study.
References
- 1.Marder SR, Cannon TD. Schizophrenia. N Engl J Med. 2019;381(18):1753–1761. [DOI] [PubMed] [Google Scholar]
- 2.Treadway MT. The neurobiology of motivational deficits in depression—an update on candidate pathomechanisms. Curr Top Behav Neurosci. 2016;27:337–355. [DOI] [PubMed] [Google Scholar]
- 3.Chang WC, Hui CL, Chan SK, Lee EH, Chen EY. Impact of avolition and cognitive impairment on functional outcome in first-episode schizophrenia-spectrum disorder: a prospective one-year follow-up study. Schizophr Res. 2016;170(2–3):318–321. [DOI] [PubMed] [Google Scholar]
- 4.Fervaha G, Foussias G, Agid O, Remington G. Motivational and neurocognitive deficits are central to the prediction of longitudinal functional outcome in schizophrenia. Acta Psychiatr Scand. 2014;130(4):290–299. [DOI] [PubMed] [Google Scholar]
- 5.Husain M, Roiser JP. Neuroscience of apathy and anhedonia: a transdiagnostic approach. Nat Rev Neurosci. 2018;19(8):470–484. [DOI] [PubMed] [Google Scholar]
- 6.Green MF, Horan WP, Barch DM, Gold JM. Effort-based decision making: a novel approach for assessing motivation in schizophrenia. Schizophr Bull. 2015;41(5):1035–1044. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Culbreth AJ, Moran EK, Barch DM. Effort-based decision-making in schizophrenia. Curr Opin Behav Sci. 2018;22:1–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Barch DM, Treadway MT, Schoen N. Effort, anhedonia, and function in schizophrenia: reduced effort allocation predicts amotivation and functional impairment. J Abnorm Psychol. 2014;123(2):387–397. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Fervaha G, Graff-Guerrero A, Zakzanis KK, Foussias G, Agid O, Remington G. Incentive motivation deficits in schizophrenia reflect effort computation impairments during cost-benefit decision-making. J Psychiatr Res. 2013;47(11):1590–1596. [DOI] [PubMed] [Google Scholar]
- 10.Gold JM, Strauss GP, Waltz JA, Robinson BM, Brown JK, Frank MJ. Negative symptoms of schizophrenia are associated with abnormal effort-cost computations. Biol Psychiatry. 2013;74(2):130–136. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Reddy LF, Horan WP, Barch DM, et al. Effort-based decision-making paradigms for clinical trials in schizophrenia: part 1—psychometric characteristics of 5 paradigms. Schizophr Bull. 2015;41(5):1045–1054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Treadway MT, Peterman JS, Zald DH, Park S. Impaired effort allocation in patients with schizophrenia. Schizophr Res. 2015;161(2–3):382–385. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Huang J, Yang XH, Lan Y, et al. Neural substrates of the impaired effort expenditure decision making in schizophrenia. Neuropsychology. 2016;30(6):685–696. [DOI] [PubMed] [Google Scholar]
- 14.McCarthy JM, Treadway MT, Bennett ME, Blanchard JJ. Inefficient effort allocation and negative symptoms in individuals with schizophrenia. Schizophr Res. 2016;170(2–3): 278–284. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Yang XH, Huang J, Zhu CY, et al. Motivational deficits in effort-based decision making in individuals with subsyndromal depression, first-episode and remitted depression patients. Psychiatry Res. 2014;220(3):874–882. [DOI] [PubMed] [Google Scholar]
- 16.Treadway MT, Bossaller NA, Shelton RC, Zald DH. Effort-based decision-making in major depressive disorder: a translational model of motivational anhedonia. J Abnorm Psychol. 2012;121(3):553–558. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Rocca P, Montemagni C, Zappia S, Piterà R, Sigaudo M, Bogetto F. Negative symptoms and everyday functioning in schizophrenia: a cross-sectional study in a real world-setting. Psychiatry Res. 2014;218(3):284–289. [DOI] [PubMed] [Google Scholar]
- 18.Foussias G, Mann S, Zakzanis KK, van Reekum R, Agid O, Remington G. Prediction of longitudinal functional outcomes in schizophrenia: the impact of baseline motivational deficits. Schizophr Res. 2011;132(1):24–27. [DOI] [PubMed] [Google Scholar]
- 19.Kraepelin E. Dementia praecox and paraphrenia. J Nerv Ment Dis. 1921;54(4):384. [Google Scholar]
- 20.Yuen GS, Bhutani S, Lucas BJ, et al. Apathy in late-life depression: common, persistent, and disabling. Am J Geriatr Psychiatry. 2015;23(5):488–494. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Horan WP, Reddy LF, Barch DM, et al. Effort-based decision-making paradigms for clinical trials in schizophrenia: part 2—external validity and correlates. Schizophr Bull. 2015;41(5):1055–1065. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Docx L, de la Asuncion J, Sabbe B, et al. Effort discounting and its association with negative symptoms in schizophrenia. Cogn Neuropsychiatry. 2015;20(2):172–185. [DOI] [PubMed] [Google Scholar]
- 23.Culbreth AJ, Moran EK, Barch DM. Effort-cost decision-making in psychosis and depression: could a similar behavioral deficit arise from disparate psychological and neural mechanisms? Psychol Med. 2018;48(6):889–904. [DOI] [PubMed] [Google Scholar]
- 24.Sherdell L, Waugh CE, Gotlib IH. Anticipatory pleasure predicts motivation for reward in major depression. J Abnorm Psychol. 2012;121(1):51–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Subramaniapillai M, Mansur RB, Zuckerman H, et al. Association between cognitive function and performance on effort based decision making in patients with major depressive disorder treated with Vortioxetine. Compr Psychiatry. 2019;94:152113. [DOI] [PubMed] [Google Scholar]
- 26.Hershenberg R, Satterthwaite TD, Daldal A, et al. Diminished effort on a progressive ratio task in both unipolar and bipolar depression. J Affect Disord. 2016;196:97–100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Hartmann MN, Hager OM, Reimann AV, et al. Apathy but not diminished expression in schizophrenia is associated with discounting of monetary rewards by physical effort. Schizophr Bull. 2015;41(2):503–512. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: DSM-5. 5th ed. xliv, 947. Washington, DC: American Psychiatric Association; 2013. [Google Scholar]
- 29.Lecrubier Y, Weiller E, Herugeta T.. Mini International Neuropsychiatric Interview German Version 5.0.0. München, Germany: Psychiatrischen Universitätsklinik München; 1999. [Google Scholar]
- 30.Cathomas F, Guetter K, Seifritz E, Klaus F, Kaiser S. Quinolinic acid is associated with cognitive deficits in schizophrenia but not major depressive disorder. Sci Rep. 2021;11(1):9992. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Cathomas F, Klaus F, Guetter K, et al. Increased random exploration in schizophrenia is associated with inflammation. NPJ Schizophr. 2021;7(1):6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Kirschner M, Cathomas F, Manoliu A, et al. Shared and dissociable features of apathy and reward system dysfunction in bipolar I disorder and schizophrenia. Psychol Med. 2020;50(6):936–947. [DOI] [PubMed] [Google Scholar]
- 33.Klaus F, Guetter K, Schlegel R, et al. Peripheral biopterin and neopterin in schizophrenia and depression. Psychiatry Res. 2021;297:113745. [DOI] [PubMed] [Google Scholar]
- 34.Woods SW. Chlorpromazine equivalent doses for the newer atypical antipsychotics. J Clin Psychiatry. 2003;64(6): 663–667. [DOI] [PubMed] [Google Scholar]
- 35.Kirkpatrick B, Strauss GP, Nguyen L, et al. The brief negative symptom scale: psychometric properties. Schizophr Bull. 2011;37(2):300–305. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Strauss GP, Esfahlani FZ, Galderisi S, et al. Network analysis reveals the latent structure of negative symptoms in schizophrenia. Schizophr Bull. 2019;45(5):1033–1041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Wallwork RS, Fortgang R, Hashimoto R, Weinberger DR, Dickinson D. Searching for a consensus five-factor model of the Positive and Negative Syndrome Scale for schizophrenia. Schizophr Res. 2012;137(1–3):246–250. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Fervaha G, Agid O, Foussias G, Remington G. Toward a more parsimonious assessment of neurocognition in schizophrenia: a 10-minute assessment tool. J Psychiatr Res. 2014;52:50–56. [DOI] [PubMed] [Google Scholar]
- 39.Nuechterlein KH, Green MF, Kern RS, et al. The MATRICS Consensus Cognitive Battery, part 1: test selection, reliability, and validity. Am J Psychiatry. 2008;165(2):203–213. [DOI] [PubMed] [Google Scholar]
- 40.Fervaha G, Hill C, Agid O, et al. Examination of the validity of the Brief Neurocognitive Assessment (BNA) for schizophrenia. Schizophr Res. 2015;166(1–3):304–309. [DOI] [PubMed] [Google Scholar]
- 41.Juckel G, Schaub D, Fuchs N, et al. Validation of the Personal and Social Performance (PSP) Scale in a German sample of acutely ill patients with schizophrenia. Schizophr Res. 2008;104(1–3):287–293. [DOI] [PubMed] [Google Scholar]
- 42.Klaus F, Chumbley JR, Seifritz E, Kaiser S, Hartmann-Riemer M. Loss aversion and risk aversion in non-clinical negative symptoms and hypomania. Front Psychiatry. 2020;11:574131. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Kahneman D, Tversky A. Prospect theory: an analysis of decision under risk. Econometrica. 1979;47(2):263–291. [Google Scholar]
- 44.Pretus C, Bergé D, Guell X, Pérez V, Vilarroya Ó. Brain activity and connectivity differences in reward value discrimination during effort computation in schizophrenia. Eur Arch Psychiatry Clin Neurosci. 2021;271(4):647–659. [DOI] [PubMed] [Google Scholar]
- 45.Hartmann-Riemer M, Kirschner M, Kaiser S. Effort-based decision-making paradigms as objective measures of apathy in schizophrenia? Curr Opin Behav Sci. 2018;22:70–75. [Google Scholar]
- 46.Fervaha G, Duncan M, Foussias G, Agid O, Faulkner GE, Remington G. Effort-based decision making as an objective paradigm for the assessment of motivational deficits in schizophrenia. Schizophr Res. 2015;168(1–2):483–490. [DOI] [PubMed] [Google Scholar]
- 47.Moran EK, Culbreth AJ, Barch DM. Ecological momentary assessment of negative symptoms in schizophrenia: relationships to effort-based decision making and reinforcement learning. J Abnorm Psychol. 2017;126(1):96–105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Cléry-Melin ML, Schmidt L, Lafargue G, Baup N, Fossati P, Pessiglione M. Why don’t you try harder? An investigation of effort production in major depression. PLoS One. 2011;6(8):e23178. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Ventura J, Thames AD, Wood RC, Guzik LH, Hellemann GS. Disorganization and reality distortion in schizophrenia: a meta-analysis of the relationship between positive symptoms and neurocognitive deficits. Schizophr Res. 2010;121(1–3):1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Rocca P, Galderisi S, Rossi A, et al. ; Members of the Italian Network for Research on Psychoses include . Disorganization and real-world functioning in schizophrenia: results from the multicenter study of the Italian Network for Research on Psychoses. Schizophr Res. 2018;201:105–112. [DOI] [PubMed] [Google Scholar]
- 51.Sigaudo M, Crivelli B, Castagna F, et al. Quality of life in stable schizophrenia: the relative contributions of disorganization and cognitive dysfunction. Schizophr Res. 2014;153(1–3):196–203. [DOI] [PubMed] [Google Scholar]
- 52.Monteiro LC, Silva VA, Louzã MR. Insight, cognitive dysfunction and symptomatology in schizophrenia. Eur Arch Psychiatry Clin Neurosci. 2008;258(7):402–405. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.