EFFORT COST COMPUTATION IN SCHIZOPHRENIA: A COMMENTARY ON THE RECENT LITERATURE

Abstract

The cognitive and affective factors implicated in the motivational impairments seen in many people with schizophrenia remain poorly understood. In the past two years a number of research groups have done studies examining the role of effort-cost computations driven by the hypothesis that over-estimation of the cost of effort involved in volitional behavior might underlie the reduction in goal-directed behavior seen in some people with schizophrenia. The goal of this commentary is to assess the available evidence and the interpretative ambiguities that remain to be addressed by further studies. There is a clear preponderance of evidence suggesting that people with schizophrenia demonstrate altered effort allocation by failing to make high effort response choices in order to maximize reward. The evidence relating altered effort-allocation to the severity of negative symptoms is mixed. It remains for future work to determine the precise mechanisms implicated in altered effort-allocation with two prominent possibilities: 1) that patients overestimate the cost of effort, or, 2) underestimate the value of potential awards. Other mechanisms in need of investigation include the potential contributions of other impairments associated with the illness that actually increase the cost of effort. Further it is possible that accurate value representations fail to invigorate behavior. While questions remain, evidence available to date suggests that the study of cost-benefit decision making may shed new light on the motivational impairments seen in many people with schizophrenia.

Negative symptoms are defined as the absence of normal function: a lack of normal emotional expressivity, a lack of goal-directed behavior, a lack of social engagement. These symptoms are largely unresponsive to standard psychopharmacological treatments and it is common to read that negative symptoms are highly correlated with functional outcome (1). This is hardly surprising given that the same behaviors that are rated as negative symptoms are central features of functional outcome: patients with high levels of avolition and asociality, by definition, have limited instrumental role behavior and limited social contacts – central features of poor outcome. Thus, negative symptoms – particularly in the area of motivation and asociality – are not really a correlate of outcome; such symptoms are outcomes. Given this, increased understanding of the origins of negative symptoms is critical to guide treatment development.

For many years the lack of goal-directed behavior in people with schizophrenia (PSZ) was understood to reflect the impact of anhedonia. Simply put, if goal attainment is not particularly enjoyable - because the hedonic pay-off is blunted - there is little reason to pursue such goals. That causal understanding has been thoroughly challenged by a host of laboratory and experience-sampling studies over the last decade that have repeatedly shown that patients appear to have largely normal in- the-moment hedonic responses to a wide variety of evocative stimuli (2,3). Thus, the puzzle is: why do PSZ fail to pursue goals and potentially rewarding experiences that they seem to truly enjoy? In the framework developed by Berridge and Robinson (4): why do patients, particularly those with high levels of negative symptoms, seem to not want what they like?

In seeking to answer this question, researchers have begun to examine the role of effort-cost computations in decision-making. This is a particularly attractive concept from a translational perspective because there is a very rich basic neuroscience literature in this area demonstrating the critical role of dopamine in wanting (i.e. the willingness to work for rewards, rather than in liking per se; 5). Dopamine appears to play a fundamental role in invigorating and sustaining behaviors that facilitate obtaining a desired reward, in overcoming barriers in time, space, and instrumental requirements that stand between where an animal is at one point in time and where it needs to be to have the opportunity to consume a reward(6). In brief, dopamine-depleted rodents prefer low-effort/low-reward options, instead of high-effort/high-reward ones – an effect that is reversed by amphetamine administration (7). Notably, dopamine blockade does not impact hedonic responses; animals simply cease to be willing to work for, to expend effort to obtain rewards that they “like” (4). Furthermore, it does not impede the physical capability to expend effort: when there is no reward available for the low-effort option, DA- blockade does not impact the selection of the high-effort/high-reward option (8). More detailed investigations have shown that the effective cost of effort is enhanced by pharmacological manipulations that promote excitability of D2-expressing neurons in the striatum, and the effective cost can be reduced by decreasing this excitability (9, 10, 11). These pharmacological observations are complimented by studies showing that over-expression of striatal D2 receptors induced in adulthood, using genetic methods, leads to enhanced willingness to work for higher levels of reward (12). In healthy humans, d-amphetamine administration increases the willingness to work for higher levels of reward, and differences in dopamine release are predictive of how willing an individual is to make high effort choices to obtain higher reward levels (13). This evidence fits with computational models that capture these and a range of other effects by assuming that dopamine – via differential effects on striatal D1 and D2 receptors – modulates the extent to which choices are dictated by prospective gains vs. losses/costs of alternative actions (14). Both human functional imaging and rodent lesion studies suggest that cost/benefit decision making is also critically dependent on the anterior cingulate cortex (ACC) which may serve an integrative function representing actions and their anticipated outcomes—the basis for value-based decision-making (15, 16, 17). The fact that schizophrenia is associated with both dopamine dysfunction, as well as functional abnormalities in the ACC, suggests that effort-based decision making is likely to be altered in schizophrenia (18, 19).

Several recent studies have provided converging results in supporting this hypothesis. In the first such study, from our group (20), 44 patients with schizophrenia (PSZ) and 36 healthy controls (HC) were offered a choice between making 20 speeded alternating button presses to obtain a $1 payoff, or making 100 button presses to earn higher reward levels ($3-7). The probability of receiving the payoff was either 50% or 100%. The critical result was that PSZ were less likely to make the high effort choice at both the highest and 100% certain reward levels compared to HC. The evidence of altered effort allocation was most evident in patients with the highest levels of clinically-rated negative symptoms. This study was followed by Fervaha et al (21), who studied 16 PSZ and 16 HC using the Effort Expenditure for Rewards Task (EEfRT), developed by Treadway (22). In this task, subjects face a series of decisions between an easy task (button pressing with the index finger of the dominant hand for 7 seconds) and a harder task (using the non-dominant hand pinky to do speeded button pressing for 21 seconds), with titration of the number of button presses for each subject to adjust for differences in motor performance. As in our study, PSZ showed reduced willingness to choose the more effortful alternative when both reward magnitude and probability were highest. Notably, higher scores on the Apathy Evaluation Scale (23) correlated with making fewer high-effort choices in the high- reward/probability options. The interpretation of this effect is complex, as it was only seen in an analysis that combined the control and patient group, and was not seen in the patient group alone. Barch et al (24) used a different version of the EEfRT in a sample of 59 PSZ and 39 HC. Patients again showed decreased willingness to make high-effort choices at the highest levels of reward and probability of payoff. In this study, reduced high-effort choices for the highest reward-probability condition were related to ratings of avolition in patients. Importantly, better scores for community and work function were associated with higher rates of hard task choices in the highest probability condition, providing additional evidence that a laboratory-based measure of cost/benefit decision relates to real-world functioning.

Recently, Treadway et al (25) used the EEfRT in a group of 12 PSZ and 15 HC and found the same pattern of results: patients differed from controls at the highest levels of reward probability and magnitude. They also did an analysis where they examined the degree to which each subject's choices was influenced by the expected value on each trial, finding that patients failed to use EV to guide choices, whereas the choices of controls were strongly predicted by EV. Notably, they observed that the choices of patients with lower scores on the Scale for the Assessment of Negative Symptoms (SANS; 26) were more influenced by EV than patients with higher SANS scores.

Thus, the three studies using the EEfRT have all observed the same pattern of results: patients fail to choose high effort alternatives when payoffs are largest and most certain, as we found with a different task (20). In addition to this behavioral signature, it appears that altered effort allocation has a modest association with the severity of negative symptoms.

Confidence in this conclusion is bolstered by the results of two other studies using different methods. Hartmann et al (27) studied 31 PSZ and 20 HC using a novel handgrip-exertion task. In this task, individual exertion thresholds were determined for each subject. Subjects chose between a small amount of money (1 Swiss Franc) that required no exertion, or a larger reward (1.5- 5 Francs) that required squeezing at 40, 60, 80, or 100% of personal maximum pressure for 3.5 seconds. In analyses where the performance of healthy controls was compared to high-apathy and low-apathy patient groups, high-apathy patients showed more severe effort discounting than either controls or low-apathy patients. Consistent with this, apathy ratings from the Brief Negative Symptom Scale (BNSS; 28) were strongly correlated (r = -0.67) with overall effort-discounting scores, whereas diminished expressivity ratings (a separate negative symptom dimension) showed no relationship with discounting. One interpretive advantage of this paradigm is that the duration of trials is identical for the high- and low- effort choices, whereas in the previously discussed paradigms the choice of the high-effort alternative adds a delay: it takes more time to complete more presses, potentially confounding delay discounting (abnormal in PSZ; see 29) in the measurement of effort discounting. In related work, Wolf et al, (30) used a progressive ratio task in a group of 41 PSZ and 37 controls and examined breakpoints – the point where a subject decides they are unwilling to continue with the task as the response demands increase. As hypothesized, PSZ had lower break points: they were less willing to continue the task as response demands increased. Further, breakpoints correlated with the Amotivation scale from the Clinical Assessment Interview for Negative Symptoms (CAINS; 31). The same subjects also did a monetary card guessing task adapted from Delgado et al (32), known to elicit differential ventral striatal activity on winning and losing trials consistent with a prediction error signal. In PSZ, ventral striatal activation correlated with impaired motivation on the progressive ratio task, suggesting a direct link between the processing of unexpected rewards and willingness to expend effort.

The fact that all six of these studies provide evidence that PSZ show altered effort allocation, an effect that is related to negative symptoms in some, but not all, studies, is remarkable and suggests there is a real signal in this body of work. These studies raise the hypothesis that patients overestimate the cost of the effort that will be required to achieve their goals in everyday life. This hypothesis is bolstered by a recent study by Gard et al (33), which used ecological momentary assessment to examine the activities and goals of PSZ and healthy controls over the course of a week, with four surveys administered each day. Gard et al found that PSZ engaged in fewer effortful activities, set less effortful goals, set goals with less long-term benefits, and appeared to misestimate the amount of effort that would be required to achieve a goal. Interestingly, PSZ reported higher levels of anticipatory pleasure for goals relative to controls with similar levels of consummatory pleasure relative to controls. Importantly, higher levels of clinically-rated negative symptoms were associated with having fewer goals that have long-term positive potential. Thus, lack of anticipated or experienced pleasure (i.e., anhedonia) does not appear to be a plausible explanation for the reduction in effortful goal-directed activities. Instead, it appears that PSZ set fewer effort-demanding long-term goals for themselves with 92% of reported goals requiring either no or very little effort, whereas controls reported 69% no/low effort goals. The convergence between laboratory performance measures and self-report on everyday activities strongly suggests that there is an important clinical signal in cost/benefit decision-making.

This apparent unanimity among results appears to be short-lived. Docx et al (34) examined effort-based decision-making in a group of 40 PSZ and 30 controls using a handgrip task similar to that of Hartman (27) and failed to find either an overall effect of diagnosis or a negative symptom effect within the patient group. Further, another large-scale psychometric study examining multiple physical and cognitive effort-based decision-making paradigms has found main effects of diagnostic group across multiple measures suggesting reduced willingness to work harder for higher reward levels in PSZ, with relatively modest correlations between negative symptom severity and willingness to expend effort. Of note: willingness to expend effort was related to self-reported motivation and vocational performance (35).

Two preliminary conclusions appear to be warranted: 1) with one exception, all studies to date have found a main effect of diagnostic group, suggesting that PSZ show a reduced willingness to expend effort to obtain higher levels of reward; 2) this effect may be mediated by negative symptom severity. Given the interest in this area of work from groups around the world, we suspect that other studies will be appearing shortly. However, this may be an opportune time to consider the work to date and offer some thoughts on the interpretation of findings and methodological considerations.

The question remains how best to understand the origins of this apparent alteration in effort- allocation as many different processes may be involved in cost/benefit decision making. One potential confound to consider is that even simple motor tasks may be “harder” for patients than for controls due to motor impairments. This appears unlikely in that most tasks studied to date have very simple motor responses or involve individual difficulty titrations, minimizing but not eliminating, this concern. However, this issue may become more central as investigators move on to study cognitive effort tasks. It should also be noted that all previous studies (of which we are aware) have utilized monetary reinforcers, and the generalizability to other types of rewards has yet to be addressed. It is important to consider that cost/benefit decision making involves at least three separate processes: 1) the estimation or computations of the expected value (EV), or benefit, of an action; 2) the effort required to obtain that reward must be estimated; and 3) the EV of the action must be weighed against the perceived cost of the action. If the value is high enough, it should serve to invigorate action. Thus, overestimation of effort cost, underestimates of reward value, or difficulty translating of value into action selection can all result in altered effort allocation and different patients may demonstrate the same behavioral phenotype for different reasons. There is ample evidence for deficits in these processes from other studies in the literature. For example, we have shown a reduced correlation (relative to controls) between the subjective valuation ratings offered by patients and the amount of effort they expended to either prolong contact with hedonic stimuli or reduce contact with aversive stimuli (36). Thus, a failure to increase responding in high effort conditions, typically attributed to effort aversion, might actually be the result of a blunted representation of the expected reward values that would be obtained, for example a high reward outcome may not be well differentiated from an intermediate reward.

There is other evidence of altered valuation-based decision-making in PSZ and we will only highlight a few of them here. Valuation preferences are typically transitive (i.e., if I like A> B, and B>C, then I would likely prefer A>C). We examined this type of decision making using pictures of puppies, pleasant foods, etc., and found that the choices of PSZ were much less transitive than those of HC (37). If valuations are not made in a parametric, ordered fashion, decisions about effort investment might also occur in a non-orderly fashion. In addition, PSZ show deficits in calculating expected value in gambling type tasks (38). In reinforcement learning tasks, we have found that patients tend to underestimate the reward probability for frequently reinforced stimuli but perform accurately for frequently punished stimuli (39). Further, the degree to which reward probability was underestimated correlated with the severity of negative symptoms. This result is consistent with two earlier experiments from our group (40,41), where we found that the severity of negative symptoms was related to an undervaluation of stimuli that had the highest expected value coupled with intact learning from stimuli that were frequently punished, or intact learning to avoid punishment. Such a devaluation of reward value would be expected to impact cost/benefit decision-making by reducing the motivational incentive value of increasing reward levels.

Thus across multiple decision making and reinforcement learning experiments, PSZ differ from controls in the extent to which they appear to represent the relative positive expected value of stimuli and possible actions and use this information to guide behavior. We speculate that the neural system that codes for reward value may have limited dynamic range in schizophrenia so that as reward value increases, the neural response fails to increment in a linear fashion and ends up being under-additive. The extent of this deficit correlates with the severity of negative symptoms in many, but not all, of the experiments that have examined aspects of this issue. Such altered valuation may well impact choices in effort tasks. In physical cost/benefit decision-making tasks, the differences in effort required are highly salient, whereas the differences in valuation may be relatively subtle. Thus, a failure to adequately represent positive expected value could easily alter effort-cost computations. Note, in most studies, participants are faced with high- versus low-effort response alternatives while the expected values tied to those choices often varies in a much more parametric fashion. Given the possibility that patients have difficulty representing relative expected value, it might be very informative to design tasks where several values are held constant while parametrically varying the amount of effort required to obtain those rewards. This type of design might offer better resolution on the costs of effort by reducing the role of relative valuation. In nearly all paradigms studied to date, reward and effort demand are manipulated simultaneously, decreasing the ability to detect the contribution of these two distinct processes. Raising these interpretive issues should not diminish interest in the study of effort-cost computations as a mechanism implicated in schizophrenia and in negative symptoms. Indeed, there does appear to be a replicable behavioral signal in this area suggesting that this may be a ripe area for additional study. However, the experiments showing alterations in valuation and the ability to translate value into action suggest that there is likely more than one path that leads to altered effort-cost computations.

Another issue that needs to be studied more carefully is the impact of antipsychotic treatment on effort tasks, given the large animal literature implicating acute striatal D2 blockade in reducing motivation to work. As always, it is difficult to assess the impact of core symptoms of SZ versus that of medication in chronically treated patients. The common approach, used in all seven published effort studies including ours, is to determine the effects of antipsychotic doses using standard conversion tables that are based on clinical efficacy rather than direct in-vivo assessment of dopamine blockade (42).

None of the above-mentioned studies found a correlation between antipsychotic dose and willingness to choose the high-effort response option. This post-hoc analytic approach is quite problematic: first, antipsychotic dose and type is not randomly assigned, and second, the conversion tables may not properly assess the D2 receptor affinity per se, which is most relevant given the literature.

For this review, we decided to explore our data set more closely (19). In our sample, we had 16 patients on clozapine monotherapy, a drug with relatively low D2 affinity. We also had 7 patients on monotherapy with either haloperidol or fluphenazine, prototype first-generation antipsychotics with high D2 affinity, and 6 patients on risperidone monotherapy, a drug with both relatively high D2 affinity as well as a broader spectrum of effects. As seen in Figure 1C the clozapine group was receiving the “highest “ daily dose using the conversion tables, but Figure 1A shows that the more relevant factor (if any, see below) appears to be D2 affinity. The clozapine group showed low levels of high-effort choices at the lower reward levels with a clear upward slope at the $5 level, where the payoffs per press actually become advantageous: they are appropriately sensitive to cost/benefit tradeoffs. In contrast, patients on first-generation drugs showed minimal responsiveness to reward level, choosing the greater effort/higher pay-off option less than 50% of the time- a remarkable degree of effort aversion. The risperidone group was eager to choose the high effort alternative in a somewhat undifferentiated fashion, with similar levels of performance from $4-$7.

Figure 1.

Proportion of high effort choices as a function of antipsychotic-type. A. Patients on first- generation drugs show marked indifference to increasing reward levels. B. Haloperidol-equivalent dose across the patient groups. C. Patients on first-generation drugs had much higher negative symptom ratings.

Before concluding that this is an effect of drug type, however, it is important to consider negative symptoms, as seen in Figure 1B. The patients on first-generation drugs have BNSS total scores that are basically twice as high as seen in the other two groups

Thus, what looks like a theoretically interesting effect of drug type is confounded by patient “type”: the patients on first-generation drugs had the highest levels of negative symptoms, and it is impossible to separate cause from effect. The most that can be said about these data is that the conversion tables do not appear to provide a useful signal to use for these types of post-hoc analysis, as the clozapine group has the highest drug dose and relatively well-preserved effort-cost computations. Given the basic neuroscience suggesting a role for dopamine in effort-cost computations, additional study of this question is clearly warranted using more optimal study designs.

To further explore negative-symptom effects in our sample, we removed the seven subjects taking first-generation drugs and used the same negative symptom cut score as in our original paper to form groups of high and low negative symptom patients. As seen in Figure 2A, the effort-based decision making of the two groups is nearly identical, as is their haloperidol daily dose (2B). It is difficult to attribute this lack of difference in decision making to any lack of difference in negative symptom severity. As seen in Figure 2C the BNSS total score in the high negative symptom group is 6-7 times higher than in the low negative symptom group.

Figure 2.

Proportion of high effort choices as a function of negative symptoms with the patients on first- generation drugs removed from the sample. A. Probability of selecting the harder response alternative in the high and low negative symptom groups. B. The groups of high- and low-negative-symptom had very similar haloperidol equivalent doses, whereas the groups differed markedly on negative symptom severity.

These results were truly surprising, as the elimination of a very small number of subjects profoundly altered our original findings. Concerned that our original between-group results (HC vs. SZ) showing altered effort allocation might have been driven by patients on first-generation drugs, we did a comparison of the 36 HC with the 37 PSZ not receiving first-generation monotherapy. We did a group (2 levels: HC, SZ) × value (5 levels: 3, 4, 5, 6, 7) repeated-measures ANOVA, which found a significant within-subjects effect of value [F(4,71) = 46.2, p<0.001] and a main effect of group [F(1,71) = 4.89, p=0.03). In post-hoc tests, significant between-group differences were only observed at the $6 and $7 reward levels (both p's<0.01), consistent with findings from other groups that differences are most likely to be found at the highest reward levels. Thus, relative to controls, PSZ, as a group, show a reduced willingness to expend effort to obtain higher levels of reward that does not appear to be confounded by the use of first-generation antipsychotics.

As noted above, it is difficult to evaluate the validity of such post-hoc analyses suggesting that patients on first-generation antipsychotics had an important impact on our original negative symptom results, given that drug type was not randomly assigned, and other groups have also reported negative- symptom effects on effort-based decision-making including in samples where there was minimal use of first-generation drugs, including in a small subsample of unmedicated patients (23). It is clear that the question of potential antipsychotic effects on cost-benefit decision making could best be addressed in the context of randomized clinical trials comparing different doses of the same drug, or comparing drugs that systematically vary in dopamine receptor affinity.

The study of effort-based decision-making appears to be a new, promising translational approach to investigating motivational deficits common among PSZ. This literature has interpretive limitations: small samples, varied clinical assessment approaches, and a variety of experimental paradigms yielding somewhat different findings. However, this new work compliments studies showing that other aspects of reward-based decision making and reinforcement learning are implicated in negative symptoms including reduced exploration of response alternatives (43), a reduced ability to represent expected value (41), and a reduced ability to learn from positive outcomes (40). These different impairments may account for different aspects of the molar behaviors that are assessed by negative symptom rating scales. Thus, careful work across a range of experimental decision making and reinforcement learning paradigms and a range of clinical measures will be needed to tease apart these factors.

Figure 3.

Proportion of high effort choices in PSZ and HCs with the patients on first-generation drugs removed from the sample.