Commentary: Dopamine-Dependent Loss Aversion during Effort-Based Decision-Making

In a recent study, Chen et al. (2019) found that medicated Parkinson's patients were less loss averse than healthy controls in an effort-based decision-making task. Loss aversion refers to a tendency to weight losses (punishments) more heavily than equivalent gains (rewards) (Kahneman and Tversky, 1979; Tversky and Kahneman, 1992), while effort-based decision making refers to tasks in which responding requires the exertion of physical effort (Kurniawan et al., 2011). Chen et al. found that when decisions were framed in terms of maximizing gains, Parkinson's patients and healthy controls were equally motivated to exert effort to obtain reward. However, when framed in terms of minimizing losses, patients were less motivated to exert effort to avoid punishment. Given that many clinical disorders are characterized by aberrant motivational states (Rahman et al., 2001; Cléry-Melin et al., 2011; Baraduc et al., 2013; Chong et al., 2015) and abnormal sensitivity to rewards and punishments (Kobayakawa et al., 2010; Treadway et al., 2012), studying effort-based decision making can provide important insights into the behavioral effects of such disorders and the mechanisms by which they arise.

In Chen et al., participants performed a typical effort-based decision-making task: On each trial they were shown an amount of points and a level of physical force required to execute the decision (using an individually-calibrated dynamometer). In the reward context, participants could choose to either exert the displayed force to gain the points (obtain reward) or skip the trial and receive nothing. In the punishment context, participants could exert the force to receive nothing (avoid punishment) or skip the trial and lose the points. Trial duration was fixed to avoid confounding temporal discounting with effort discounting. To the resulting choice data, the authors fit several computational models of effort discounting, which compute choice utility based on the size of the reward/punishment attenuated by the effort required to respond (Hartmann et al., 2013). For example, Chen et al.'s (best-fitting) parabolic discounting function is given by:

$$\begin{array}{r}{U}_t=R_t-\alpha E_t^2\end{array}$$ (1)

where U represents choice utility, R is the reward/punishment amount, E is the effort required, α is an effort-discounting parameter, and t indexes trials. The probability of choosing to exert effort on a given trial is then computed using a soft-max choice function that scales choice utility by a choice stochasticity parameter, β:

$$\begin{array}{r}p{(effort)}_t=\frac{1}{1+e^{-{\beta }^{*}{U}_t}}\end{array}$$ (2)

Parameter estimates from these models supported the behavioral results: Patients and controls had similar average effort-discounting parameters in the reward context, but patients had significantly higher effort-discounting parameters in the punishment context, confirming that patients were less motivated to exert effort to avoid punishment.

Although Chen et al. did not explicitly model the latent cognitive processes driving observed choice behavior (nor did they include any neurophysiological measures), the authors speculate that their findings could be due to dopaminergic Parkinson's medication differentially suppressing activity in Basal Ganglia pathways associated with processing punishment (Frank, 2005; Argyelan et al., 2018). This suggestion is consistent with several computational reinforcement learning studies linking reduced Basal Ganglia dopamine activity with impaired learning on punishment-based tasks in medicated Parkinson's patients (Frank et al., 2004; Frank, 2005). However, without additional neurophysiological measures (e.g., fMRI, PET) and an appropriate cognitive model of latent decision processes (e.g., Ratcliff, 1978; Brown and Heathcote, 2008), it is difficult to draw strong conclusions about the latent mechanisms driving the observed loss aversion effects or their neurophysiological basis from Chen et al.'s analyses.

A natural starting point for extending Chen et al.'s modeling to answer such questions would be to augment their soft-max choice rule and utility-based effort-discounting functions with a more comprehensive cognitive process model of decision making, such as a sequential sampling model—the most successful and widely applied class of decision-making models in model-based cognitive neuroscience (for reviews, Mulder et al., 2014; Forstmann et al., 2016). Sequential sampling models treat decision making as a process of accumulating samples of evidence¹ from stimuli until a threshold amount is reached, triggering a response (Figure 1).

Figure 1.

Illustration of a sequential sampling model of effort-based decision making. Within a trial, evidence about the relative value of each choice option (exert effort vs. skip trial) is accumulated over time with mean rate v until a threshold is reached, which triggers the corresponding response (at the time labeled response onset). Response time is the time it takes to reach a threshold (decision time) plus an intercept term representing non-decision processes (e.g., perceptual encoding and motor response time). Note that other linking functions between drift rate and choice utility are possible. For an introduction to applying such models to experimental data, Voss et al. (2013).

Crucially, sequential sampling models can be thought of as generalizing soft-max into the time domain, which allows them to simultaneously explain how choices and response times arise from a common set of latent cognitive processes. This provides closer contact between model and data, additional constraint on theory, and more robust inferences. As noted in prior work (Tuerlinckx and De Boeck, 2005; Bogacz et al., 2006; Miletić et al., 2020), soft-max (Equation 2) is formally equivalent to the following sequential sampling choice function, which describes the probability that evidence first reaches the upper threshold (corresponding to an “exert effort” response in Figure 1) as:

$$\begin{array}{r}{p(effort)}_t=\frac{1}{1+e^{-\frac{2v_{t}A}{s^2}}}\end{array}$$ (3)

where v is the mean rate of information processing (drift rate), A is the response threshold, s is the standard deviation of the evidence accumulation process, and t indexes trials. Equating drift rate with choice utility² (Equation 1) and substituting β for 2A/s² yields the original soft-max choice function used by Chen et al. (Equation 2). Integrating Chen's models with Equation 3 would thus yield a standard sequential sampling model with choice utility-based drift rates and an additional effort-discounting parameter. Sequential sampling models explain behavior in terms of psychologically interpretable parameters (e.g., information processing speed, response caution, motor response/encoding time, choice bias), so a combined model would simultaneously decompose observed decision-making behavior into component cognitive processes and quantify effort discounting within a single theoretical framework (for a similar approach to decomposing loss aversion in risk-based decision making using a standard sequential sampling model, Clay et al., 2017).

Such an approach would offer clear benefits for the field of effort-based decision making, improving measurement and facilitating theory development by providing a more detailed characterization of group and individual differences in loss aversion. In particular, this approach could expand on Chen et al.'s results by explaining the relationship between effort discounting and additional Parkinson's-related cognitive and motor deficits known to affect response latency (e.g., longer motor response times, less efficient information processing, impaired cognitive control over thresholds, O'Callaghan et al., 2017; Servant et al., 2018). Patients prone to effort discounting may have difficulty integrating information about effort into the decision process (a drift rate effect) or set more impulsive evidence criteria (a threshold effect) compared with healthy controls. Testing these competing accounts would further our understanding of Parkinson's disease and better titrate individual differences in cognitive and motor processes. Explicit mechanisms to capture additional phenomena relevant to effort-based decision making, such as learning, urgency, and fatigue effects, can also be instantiated within the same sequential sampling framework (Milosavljevic et al., 2010; Miletić et al., 2020), and linking cognitive processes with neurophysiological measures (e.g., by treating model parameters as covariates to neural activity or constructing a joint model, Turner et al., 2017, 2019) would be especially informative regarding Chen et al.'s broader questions about the role of dopamine and Basal Ganglia activity in explaining Parkinson's-related differences in decision making. Overall, moving toward a cognitive process model of effort-based decision making promises a finer-grained mechanistic understanding of aberrant motivational states in neuropsychiatric disorders and more detailed insight into the sources of group and individual differences in effort-based decision making.

Author Contributions

The author confirms being the sole contributor of this work and has approved it for publication.

Conflict of Interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.