Influence of Depression Symptoms on History Independent Reward and Punishment Processing

Abstract

Prior research indicates that depressed individuals are less responsive to rewards and more sensitive to punishments than non-depressed individuals. This study examines decision-making under reward maximizing or punishment minimizing conditions among adults with low (n = 47) or high (n = 48) depression symptoms. We utilized a history-independent decision making task where learning is experience-based and participants’ goal was to enhance immediate payoff. Results indicated a significant interaction between incentive condition (reward maximizing, punishment minimizing) and depression group. Within the low depression group, better performance was observed for reward maximization than punishment minimization. In contrast, within the high depression group, better performance was observed for punishment minimization than reward maximization. Further, the high depression group outperformed the low depression symptom group in the punishment minimization condition, but no depression group differences were observed in the reward maximization condition. Computational modeling indicated that the high depression group was more likely to choose options with the highest expected reward, particularly in the punishment condition. Thus, decision-making is improved for people with elevated depression symptoms when minimizing punishment relative to maximizing rewards.

1. Introduction

Depression is a common and impairing condition that predicts a number of negative outcomes, such as future suicide attempts, interpersonal problems, unemployment, and substance abuse (Kessler and Walters, 1998; Kessler et al., 2003). The World Health Organization estimates that 121 million people are currently suffering from depression and it is the leading cause of disability worldwide among people 5 years of age and older. Given its high prevalence, it is perhaps not surprising that the annual economic cost of major depressive disorder (MDD) in the U.S. alone is over $70 billion in medical expenditures, lost productivity, and other costs (Greenberg et al., 1993; Philip et al., 2003).

Depression is consistently associated with biased processing of negative information (Gotlib and Joormann, 2010). Depressed individuals focus on negative self-referent thoughts and exhibit enhanced effortful recall of negative-valence material (Mathews and MacLeod, 2005). Similarly, depressed individuals are hypersensitive to negative feedback and punishment (Eshel and Roiser, 2010). For instance, depressed people are more likely than controls to revert back to a previously learned rule following non-contingent negative feedback (Murphy et al., 2003). This effect appears to be specific to unipolar depression, as bipolar participants experiencing a depressive episode do not display sensitivity to non-contingent punishment whereas people with unipolar depression do show this bias (Taylor Tavares et al., 2008). Depressed people also display greater electrophysiological response to errors than controls (Holmes and Pizzagalli, 2008). These findings are consistent with the suggestion that depressed individuals possess bio-behavioral motivation systems (Fowles, 1994) that increase sensitivity to punishment and generate negative affect (Kasch et al., 2002). Given this sensitivity to negative outcomes, decision-making may be enhanced in depression when the goal is to minimize punishment.

Depression is also associated with decreased sensitivity to rewarding stimuli (Berenbaum and Oltmanns, 1992; Henriques et al., 1994; Gotlib and Joormann, 2010). For example, depressed individuals exhibit attenuated responses to pleasant drinks (Berenbaum and Oltmanns, 1992) and monetary rewards (Henriques et al., 1994). Depressed individuals also are more inconsistent in their decision-making when trying to delay receipt of rewards (Takahashi et al., 2008) and tend to be more conservative in their decision-making, even when the likelihood of receiving a reward is high (Murphy et al., 2001). Further, on a probabilistic reward learning task, depressed individuals displayed significantly reduced reward responsiveness compared to healthy controls. Trial-by-trial analyses indicated that depressed individuals were less likely than controls to rely on past reinforcement history to guide current decision-making, particularly in the absence of an immediate reward (Pizzagalli et al., 2008; Gradin et al., 2011). Poor responsiveness to rewards predicts a more protracted course of depression (Kasch et al., 2002).

Taken together these data suggest that depression is associated with intact or increased sensitivity to punishment and reduced sensitivity to reward. Although these findings are important, and motivated the current work, what is lacking is an integrated empirical examination of the effects of incentives (reward vs. punishment) and depression symptoms on decision-making. In other words, few studies have examined reward and punishment processing within the same subjects using decision-making tasks that are directly comparable. Most prior studies have used between subjects designs and/or only studied punishment or reward processing in isolation. By studying reward and punishment processing within the same subjects using tasks that are directly comparable, we will be able to assess the relative performance of depressed and non-depressed individuals across incentive conditions. This should yield a more comprehensive test of incentive processing in depression than many prior studies.

Another important feature of this study is the focus on history independent decision-making. History independent decision-making refers to circumstances where rewards for current decisions are independent of the choices that were made in the past (Worthy et al., 2011; Worthy and Maddox, 2012). That is, the level of reward received for any given trial does not depend on the level of reward received for prior decisions. For example, the probability of winning money by selecting red on the roulette wheel is independent of whether red was picked previously. When current rewards do not depend upon previous choices, decision-making is history independent. This is in contrast to history-dependent tasks, where the level of reward for a given trial is in part dictated by past decisions.

However, it is important to note that, unlike playing roulette, participants still need to learn from past decisions to perform well on history-independent decision-making tasks. These are decision-making problems for which the gains and losses are unknown and one must learn about them from experience. Over time, participants can learn which choices provide large rewards or most effectively minimize losses. So, even though the rewards are distributed in a history independent fashion, learning about the nature of the decision-making environment can lead to enhanced performance. Such decision-making relies on information processing strategies that involve learning the values associated with each choice directly, and do not require developing a complex mental model of the environment. Learning simply involves choosing the option that maximizes immediate payoff. This is in contrast to other paradigms where decisions are made under risk—the probabilities of gain or loss for each option are known but they differ in risk (Kahneman and Tversky, 1979; Murphy et al., 2001). Decision-making on the history independent task is dependent on experience—that is, people learn which options have the highest reward (or minimize losses) over time.

Depressed individuals may be particularly good at history independent tasks. Depression is typically associated with performance deficits in tasks that tap effortful, reflective information processing. For instance, depressed individuals display difficulties with effortful problem-solving (Elderkin-Thompson et al., 2006), planning (Rogers et al., 2004), and cognitive flexibility (Butters et al., 2004). Depressed individuals also have memory deficits (Burt et al., 1995), particularly in free recall tasks and other tasks that require controlled aspects of recognition (Hertel, 1998; Dalgleish et al., 2007). With sufficient external support, however, cognitive performance can be improved (Hertel and Rude, 1991). In contrast, performance remains intact when optimal performance relies on automatic, reflexive information processing (Hartlage et al., 1993). Depressed individuals have sufficient cognitive resources for non-cognitively demanding tasks, but performance suffers when required to engage or control cognitive resources (Hertel, 1994).

Taken together, these data suggest that depression is associated with intact or nearly intact performance in tasks that require automatic, reflexive processing, such as history independent decision-making tasks. Further, increased sensitivity to punishment should facilitate learning the task options that produce the least amount of punishment (i.e., smallest point loss). Decision-making performance may therefore be enhanced in depression when the goal is to minimize punishments in a history independent decision-making task. Testing this hypothesis is the main goal of this study.

A secondary goal is to examine how depression affects reward processing on a history-independent decision-making task. On the one hand, the relatively effortless processing required to perform a history independent task should not be disrupted in depression (Hartlage et al., 1993). However, as noted above, depression is associated with reduced responding to rewards. Thus, an exploratory aim is to determine whether depression disrupts reward processing on a history-independent decision-making task.

To achieve these aims, participants completed two history-independent decision-making tasks that are identical except in one version participants are instructed to minimize punishments and in the other version they are instructed to maximize rewards. Thus, we can directly examine how depression influences decision-making performance within each incentive condition. This allows for a rigorous and comprehensive test of whether depression interacts with incentive structure (reward vs. punishment) to predict decision-making performance. Finally, computational modeling was used to assess the cognitive processes that give rise to any observed group differences in decision-making performance.

2. Methods

2.1 Participants

Participants were 95 adults recruited from the Austin, Texas community (see Table 1 for demographic information). On average, the sample was 19.69 years old, female, and educated¹. Participants were recruited using flyers posted in the community and with ads posted on the Web. Participants received $10 for their participation in the 50-minute study. Inclusion criteria included normal or corrected-to-normal vision and fluency in the English language.

Table 1.

Demographic characteristics of study participants.

	Low Depression	High Depression
n	47	48
Age (years)	20.23 (3.28)	19.17 (1.41)
Gender (m/f)	19/28	18/30
CES-D	8.72 (4.06)	26.21 (8.81)

2.2 Depression Classification

At the beginning of the experimental session, each participant was administered the Center for Epidemiological Studies Depression Scale (CES-D; Radloff, 1977). Following convention (Weissman et al., 1977), participants who obtained a score of 15 or less were classified as having low depression symptoms, and participants who obtained a 16 or greater were classified as having high depression symptoms. CES-D scores of 16 or greater reflect moderate or greater symptoms of depression (Radloff, 1977). A cut-point of 16 on the CES-D has very good sensitivity and specificity for the prediction of current major depressive disorder (Beekman et al., 1997). Consistent with prior research (e.g., Pizzagalli et al., 2005), CES-D total score was dichotomized to facilitate comparisons of participant groups with versus without elevated depressive symptoms.

2.3 Decision-Making Task

The four-option history-independent decision-making task is displayed in Figure 1. On each trial, participants selected one of the four options and received points (in the reward-maximizing condition) or lost points (in the punishment-minimization condition). Each task included a total of 80 trials. The points gained for each option on each of the 80 trials in the reward-maximizing condition are shown in Figure 2. Two of the four options in the task are “A” options (denoted by the solid line in Figure 2) and each provides the same reward for a given trial. The remaining two options are “B” options (denoted by the broken line in Figure 2) and each provides the same reward for a given trial. During the first 50 trials of the task, the B options always provide a larger reward than the A options. Thus, if one of the B options is selected on each of the first 50 trials immediate payoffs will be maximized. During the final 30 trials, the reward contingencies reverse and now the A options always provide a larger reward than the B options. If one of the A options is selected on each of the final 30 trials then immediate payoffs will be maximized. The location of the A options and the B options on the computer screen was randomized for each participant.

Figure 1.

Screen shot from two trials of the task in the reward-maximizing and punishment-minimizing conditions from Experiment 1.

Figure 2.

Reward structure associated with the history-independent decision-making task.

The punishment-minimizing version of the task was derived directly from the reward-maximizing version by subtracting 11 points from the rewards given for each option on each trial in Figure 2. During the first 50 trials of the punishment-minimization task, the B options always provide a smaller loss of points than the A options. Thus, if one of the B options is selected on each of the first 50 trials losses will be minimized. During the final 30 trials, the reward contingencies reverse and now the A options always provide a smallest loss compared to the B options. Importantly, because points received for the reward-maximization and punishment-minimization conditions are perfectly correlated across trials, performance on these tasks is directly comparable. These tasks have been used to assess reward and punishment processing in prior research with healthy (Worthy et al., 2007b; Worthy and Maddox, 2012) and elderly populations (Worthy et al., 2011).

3. Procedure

Each participant completed the reward-maximization and punishment-minimization versions of the task in the same session in a counterbalanced order². Participants were informed that they would make 80 choices and that they would receive points for each choice (in the reward-maximizing condition) or would lose points for each choice (in the punishment-minimizing condition). No information was provided regarding the nature of the rewards or punishments and participants were told only to maximize rewards or minimize losses. To enhance engagement with the task, all participants were given the same performance goal in the form of a point total to reach or cumulative point loss to avoid. The goal was attained if the participant selected the option with the best immediate reward (or smallest loss) on approximately 90% of the trials. Progress towards this goal was indicated on screen and updated after each trial (see Figure 2). The Internal Review Board at the University of Texas approved all study procedures.

4. Results

Total Points

Since the punishment-minimizing condition was constructed by subtracting 11 points from the outcome in each trial of the reward-maximizing condition, we added 11 points to each punishment-minimizing outcome to allow direct comparisons across the reward and punishment versions of the task. We then conducted a 2 (incentive condition: punishment-minimization, reward-maximization) × 2 (depression symptoms: low, high) repeated measures analysis of variance with total points as the dependent variable, task condition as a repeated measures factor and depression symptom group as a between subjects factor. Analyses indicated a significant main effect for depression group, F(1, 93) = 5.95, p = 0.02, partial η² = 0.06, a main effect for task condition, F(1, 93) = 5.72, p = 0.02, partial η² = 0.06, and a significant depression group × task condition interaction, F(1, 93) = 14.42, p < 0.001, partial η² = 0.13.

Follow-up analyses first examined simple effects for incentive condition within each depression group. Within the low depression group, total points were significantly higher for reward than punishment, F(1, 46) = 10.39, p = 0.002, partial η² = 0.18 (see Figure 3). In contrast, within the high depression group, total points were significantly higher for punishment than reward, F(1, 47) = 5.18, p = 0.03, partial η² = 0.09. Within the reward condition, there was no difference in total points between depression groups, F(1, 93) = 0.02 p = 0.88, partial η² = 0.00. However, within the punishment condition, the high depression group had significantly more total points than the low depression group, F(1, 93) = 11.83 p < 0.001, partial η² = 0.11 (Figure 3). People with elevated depression appear to be better able to process information and solve the task at a higher performance level than people with few depression symptoms when minimizing punishment rather than maximizing reward. In contrast, relative to punishment processing, decision-making is enhanced for the low depression group when maximizing rewards. However, it is important to note that depression groups did not differ in reward maximization, so the key finding is that punishment processing is enhanced among people with high depression symptoms.³

Figure 3.

Total points obtained for the reward-maximizing and punishment-minimizing versions of the choice-independent task. Standard error bars are included.

4.1 Computational Modeling

To better understand the strategies participants used to make decisions in the task we fit two learning models that have been extensively used to model decision-making behavior: A heuristic-based Win-Stay-Lose-Shift (WSLS) model and a Softmax Reinforcement Learning (RL) model (Worthy et al., in press; Sutton and Barto, 1998; Worthy et al., 2007a; Frank and Kong, 2008; Steyvers et al., 2009; Lee et al., 2011; Otto et al., 2011; Worthy and Maddox, 2012). WSLS models were originally developed for simple prediction tasks where the participant chooses an option and receives a reward with a certain probability, p, or does not receive a reward with a probability, (1-p). It assumes that participants will “stay” by picking the same option on the next trial if they are rewarded (a “win” trial), or “shift” by selecting another option on the next trial if they are not rewarded (a “lose” trial).

In the tasks used in the present experiment participants select from among four options on each trial and gain or lose between 1–10 points. We have developed a WSLS model for these tasks that assumes that participants compare the reward received on the present trial to the reward received on the previous trial (Worthy and Maddox, 2012). The trial is a “win” trial if the reward on the present trial is equal to or greater than the reward received on the previous trial, and the trial is a “loss” trial if the reward on the present trial is less than the reward received on the previous trial.

The WSLS model has two free parameters. The first parameter represents the probability of staying with the same option on the next trial if the reward received on the current trial is equal to or greater than the reward received on the previous trial:

$$P(a_i,t\mid {\mathit{choice}}_{t-1}=a_i\&r(t-1)\ge r(t-2))=P(\mathit{stay}\mid \mathit{win}).$$ (1)

In Equation 1 r represents the reward received on a given trial. The probability of switching to another option following a win trial is 1−P(stay|win). To determine a probability of selecting each of the other three options we divide this probability by three, so that the probabilities for selecting each option sum to one.

The second parameter represents the probability of shifting to the other option on the next trial if the reward received on the current trial is less than the reward received on the previous trial:

$$P\left(a_j,t\mid {\mathit{choice}}_{t-1}=a_i\&r(t-1)<r(t-2)\right)=P(\mathit{shift}\mid \mathit{loss}).$$ (2)

This probability is divided by three and assigned to each of the other three options. The probability of staying with an option following a “loss” is 1−P(shift|loss). Importantly, this model assumes a simple, heuristic-based strategy that requires the reward received on the previous trial to be maintained in working memory (e.g., Otto et al., 2011).

The Softmax RL model operates by developing and updating expected reward values (EV) for each option, a_j, on each trial, t. These EVs are denoted here and elsewhere as EV(a_j, t). The EVs for each option are used to determine the model’s probability for selecting each option. Action selection probabilities for each option are computed via a Softmax decision rule:

$$P(a_i,t)=\frac{e^{[\theta ·EV(a_i,t)]}}{{\sum }_{j=1}^2{e}^{[\theta ·EV(a_j,t)]}}$$ (3)

Here θ is an exploitation parameter that determines the degree to which the option with the highest EV is chosen. As θ approaches infinity the highest valued option is chosen more often, and as θ approaches 0 all options are chosen equally often.

The Softmax model assumes that the EV for the option chosen on each trial, denoted as option i, is updated on each trial using the following equation:

$$EV(a_i,t+1)=EV(a_i,t)+\alpha ·[r(t)-EV(a_i,t)]$$ (4)

This model assumes that the EVs for each option are updated only when that option is selected, and are based only on the reward received immediately after making a choice. Learning is primarily mediated by a prediction error between the reward received and the EV for the chosen option (the bracketed portion of Equation 4). The prediction is positive if the reward received is larger than expected and negative if the reward received is less than expected.

Learning is modulated by a learning rate, or recency parameter (α), 0 ≤ α ≤ 1 that weighs the degree to which participants update the EVs for each option based on the most recently received rewards. As α approaches 1 greater weight is given to the most recent rewards in updating EVs, indicative of more active updating of EVs on each trial, and as α approaches 0 rewards are given less weight in updating EVs. When α = 0 no learning takes place, and EVs are not updated throughout the experiment from their initial starting points, Q(a_i, t_o).

In addition to the WSLS and Softmax learning models we also fit a Baseline model that assumed random responding. This model has three free parameters representing the probability of selecting options 1–3 on any given trial. The probability of selecting the fourth option is one minus the sum of the probabilities for selecting options 1–3. This model has been used in a number of experiments to ensure that the learning models adequately capture that the data and that participants are not simply behaving randomly (Worthy et al., in press; Yechiam and Busemeyer, 2005; Worthy and Maddox, 2012).

4.2 Model-Based Predictions

The Softmax model has been used in a number of studies, primarily when the rewards in the environment are history-independent, as in the current experiment (e.g., Worthy et al., 2007a; Sutton and Barto, 1998; Otto et al., 2011; Daw et al., 2006; Yechiam and Busemeyer, 2005), while WSLS models have been shown to provide a better fit to data from tasks where rewards are history-dependent (Worthy et al., in press). Based on this, we predicted that data from participants in each condition would be best fit by the Softmax model.

The Softmax model’s exploitation parameter (θ) provides an index of the degree to which participants exploit the options with the highest EV on each trial, and this parameter has been positively correlated with performance on in history-independent tasks (Worthy et al., 2007a; Worthy and Maddox, 2012). We predicted that depressed individuals increased sensitivity to punishments (i.e., loss of points) would facilitate the identification of the correct strategy more quickly and make them less susceptible to altering this strategy in response to random perturbations in trial by trial rewards. Thus, we expected the high depression group to have a higher exploitation parameter for the punishment-minimization task than the low depression group, but that there would be no difference between high and low depression groups for the reward-maximization task.

Model-Based Results

We fit each participant’s data individually with the WSLS, Softmax, and Baseline models detailed above. The models were fit based on their ability to predict each decision a participant made on each trial by maximizing negative log-likelihood. We used Akaike weights to compare the relative fit of each model (Akaike, 1974; Wagenmakers and Farrell, 2004). Akaike weights are derived from Akaike’s Information Criterion (AIC) which is used to compare models with different numbers of free parameters. AIC penalizes models with more free parameters. For each model, i, AIC is defined as:

$${\mathit{AIC}}_i=-2{\mathit{logL}}_i+2V_i$$ (5)

where L_i is the maximum likelihood for model i, and V_i is the number of free parameters in the model. Smaller AIC values indicate a better fit to the data. We first computed AIC values for the Softmax, WSLS, and Baseline models for each participant’s data. Akaike weights were then calculated to obtain a continuous measure of goodness-of-fit. A difference score is computed by subtracting the AIC of the best fitting model for each data set from the AIC of each model for the same data set:

$${\mathrm{\Delta }}_i(\mathit{AIC})={\mathit{AIC}}_i-\mathit{minAIC}$$ (6)

From the differences in AIC we then computed the relative likelihood, L, of each model, i, with the transform:

$$L(M_i\mid \mathit{data})\propto exp\left\{-\frac{1}{2}{\mathrm{\Delta }}_{\text{i}}(\text{AIC})\right\}$$ (7)

Finally, the relative model likelihoods are normalized by dividing the likelihood for each model by the sum of the likelihoods for all models. This yields Akaike weights:

$$w_i(\mathit{AIC})=\frac{exp\left\{-\frac{1}{2}{\mathrm{\Delta }}_{\text{i}}(\text{AIC})\right\}}{exp\left\{-\frac{1}{2}{\mathrm{\Delta }}_{\text{k}}(\text{AIC})\right\}}$$ (8)

These weights can be interpreted as the probability that the model is the best model given the data set and the set of candidate models (Wagenmakers and Farrell, 2004).

We computed the Akaike weights for each model for each participant. Table 2 shows the average Akaike weights for participants in each condition. Akaike weights were consistently highest for the Softmax model, indicating best fit, with virtually no evidence for the Baseline model providing the best fit. This confirms that participants were not responding randomly.

Table 2.

Akaike Weights for Each Computational Model

	WSLS	Softmax	Baseline
Reward-Maximization
Non-Depressed	0.28 (0.06)	0.70 (0.06)	0.02 (0.02)
Depressed	0.29 (0.06)	0.71 (0.06)	0.00 (0.00)
Punishment-Minimization
Non-Depressed	0.38 (0.07)	0.64 (0.06)	0.07 (0.04)
Depressed	0.31 (0.06)	0.64 (0.07)	0.02 (0.02)

Note: Higher Akaike weights indicate a better fit to the data. Standard errors of the mean are listed in parentheses. WSLS refers to the Win-Stay-Lose-Shift computational model.

Having established that the Softmax model fit the data the best, we next examined the estimated learning rate and exploitation parameter values for participants in each condition. Figure 4a shows the average estimated learning parameter values for participants in each condition. A 2 (depression: low depressed, high depressed) × 2 (incentive: reward-maximization, punishment-minimization) repeated measures ANOVA revealed a marginally significant effect of depression, F(1, 93) = 2.81, p < 0.10, partial η² = 0.029. Depressed participants’ data were best fit by higher learning rate parameter values than data from non-depressed participants (non-depressed = 0.73, depressed = 0.81). There was also a marginally significant effect of incentive F(1, 93) = 3.46, p < 0.10, partial η² = 0.036. Learning rates were higher for the reward maximizing condition than the punishment minimizing condition (reward maximizing = 0.80, punishment minimizing = 0.74). The interaction between depression and incentive was non-significant, F(1,93<1) = 0.01, p = 0.92. Thus, the high depression group used recently received rewards to update their expected reward value for a particular choice to a greater extent than low depression individuals, and participants updated more based on recent outcomes when maximizing rewards than when minimizing punishments. However, these results should be interpreted with caution, as effects were only marginally significant.

Figure 4.

Best fitting learning rate and exploitation parameter values from the Softmax model. Standard error bars are included.

Figure 4b shows the average estimated exploitation parameter values. A 2 (depression: depressed vs. non-depressed) × 2 (incentive: reward-maximizing vs. punishment-minimizing) ANOVA revealed s significant main effect of depression, F(1, 93) = 4.92, p < 0.05, partial η² = 0.05. Depressed participants’ data were best fit by higher exploitation parameter values relative to non-depressed participants’ data (non-depressed = 0.53, depressed = 0.62). There was no effect of incentive, F(1,93) = 2.36, p = 0.12; however the interaction between depression and incentive was significant, F(1,93) = 6.77, p < 0.05. To examine the locus of the depression X incentive interaction we performed comparisons within each incentive condition. These comparisons revealed that the main effect of depression was driven by differences in estimated exploitation parameter values within the punishment-minimization condition. There was no effect of depression within the reward-maximization condition, F(1,93) = 0.31, p = 0.58, but there was a large effect of depression within punishment-minimization condition, F(1, 93) = 11.02, p < 0.001, partial η² = 0.106.

5. Discussion

This study examined the association between depression symptoms and decision-making under punishment minimization and reward maximization conditions. We found that decision-making performance was enhanced for people with elevated depression symptoms when trying to minimize punishment. The high depression group performed better under punishment minimization than reward maximization condition and they also performed better than the low depression group under punishment minimization. In contrast, the low depression group performed better in the reward maximization condition compared to the punishment minimization condition. These data suggest that people with high levels of depression symptoms can make good decisions, particularly when trying to minimize losses rather than maximize rewards.

Computational modeling was performed to better understand the processes that give rise to enhanced decision-making under punishment minimization in depression. These analyses found that decision-making was best fit by a Softmax Reinforcement Learning model. This model asserts that learning occurs when there is a discrepancy between the expected reward and reward actually received. These models examine how frequently the option with the highest expected value is chosen (i.e., degree of exploitation) and whether participants update the expected values for current options based on the most recently received rewards (i.e., learning rate).

Analyses indicated that depressed individuals, in general, had higher exploitation and learning rate. That is, depressed individuals were more likely to choose an option with the highest expected reward and were more likely to alter expected values based on recently received rewards. These effects were particularly pronounced under punishment minimization conditions. These data suggest that depressed individuals’ can engage in effective decision-making, particularly when they are trying to prevent loss compared to increasing rewards. We speculate why this may be the case.

One possible explanation for the enhanced decision-making performance in depression, particularly under punishment minimization, is that depressed individuals are more sensitive to punishing stimuli (Berenbaum and Oltmanns, 1992; Gotlib and Joormann, 2010), perhaps due to alterations in dopaminergic and serotonergic neurotransmission (Takahashi, 2011). Thus, in an effort to avoid or minimize loss, depressed individuals may more readily learn which options were likely to provide the least punishment (i.e., produce the smallest point decrease). Results from computational modeling were consistent with this possibility.

Further, history-independent tasks, such as the one used in the current study, are not cognitively effortful. This may be optimal for depressed people, as automatic cognitive processes are not usually disrupted in depression (Beevers, 2005; Carver et al., 2008). Thus, the punishment-minimizing, history-independent decision-making task may have been the ideal circumstance to observe enhanced decision-making in depression. Future work should examine whether reward and punishment processing is disrupted in more effortful decision-making tasks. One possibility is that reward processing may be particularly compromised in depression when the decision-making task requires high cognitive effort (Pizzagalli et al., 2008). Punishment processing may also be hampered in cognitively effortful decision-making tasks, although possibly to a lesser degree given depressed individual’s sensitivity to punishment.

Computational modeling revealed that the high depression symptom group performed particularly well on the punishment processing task because they were more likely to choose an option with the highest expected value and were more likely to alter expected values based on recently received rewards. Future work should attempt to draw a more direct link between the greater memory for recent actions in depressed individuals, suggested by the higher learning rate values, and the greater ruminative tendencies often found in depressed individuals (Nolen-Hoeksema, 2000; Watkins and Teasdale, 2001). It is possible that the tendency for depressed individuals to ruminate extends beyond autobiographical memories and is symptomatic of a more general tendency to focus on negative past events. If so, this task may be one for which the tendency to focus on recent past events helps performance.

Nevertheless, we believe this study nicely highlights the benefits of performing computational modeling for behavioral data, as it allowed us to identify putative cognitive processes that give rise to the depression-related behavioral differences. Although computational modeling is a powerful approach for the study of cognitive processes, it has been used sparingly to study cognitive biases associated with psychopathology and disordered emotion(Armony et al., 1997). Increased use of computational modeling will be an important future direction for this area of research and may provide important new insights into the cognitive processes that give rise to behavior thought to maintain the disorder (c,f. Siegle et al., 2004).

Findings from this study suggest that depressed individuals have the cognitive processing capabilities to perform well in decision-making tasks (c.f. Hertel and Rude, 1991), but it may depend upon the nature of the optimal decision rule and whether the goal is reward-maximization or punishment-minimization. Although depressed individuals are thought to have deficits in executive functioning, particularly set shifting (Austin et al., 2001), the context in which executive functioning occurs is very important. Cognitive deficits may be most obvious when effortful reward-maximizing is measured, but absent when the goal is to minimize punishment and cognitive effort is minimal.

Finally, this work may also have intriguing implications for how to improve decision-making among depressed individuals. One possibility is that depressed individuals may improve decision-making when it is framed in terms of avoiding loss rather than maximizing gains. This is consistent with the idea that people make better decisions when they pursue goals in a manner that fits their regulatory orientation—also known as good regulatory fit (Higgins, 2000). Higgins (2000) posits that when regulatory fit is in place, people will be more motivated to pursue a goal, they will feel better about goal pursuit, and they will value their decision. There is some evidence to suggest that depressed individuals are more likely to adopt a preventative regulatory style that emphasizes the avoidance of loss (Miller and Markman, 2007). Thus, framing decisions in terms of minimizing punishment may lead to better regulatory fit and potentially better decision-making outcomes for depressed individuals. However, prior behavioral research suggests that minimizing punishment by avoiding situations that involve decision-making could perpetuate a depressed mood state by encouraging withdrawal from potential positive reinforcement (e.g., social interactions)(Kanter et al., 2010). Thus, although decision-making may be enhanced when framed in terms of punishment-minimization, it remains important for depressed individuals not to withdraw and instead to engage in healthy decision-making situations.

A second implication of this work is that depressed individuals can perform well when decision-making tasks are relatively simple. In the current study, the task did not require participants to learn complex decision-making rules to perform well on this task. This suggests that depressed individuals may improve their decision-making performance when tasks are relatively straightforward and do not require substantial cognitive effort. Consistent with this possibility, a major component of problem-solving therapy, which has been shown to be effective for treating depression (Nezu et al., 1989), is to break down complex problems into its simpler constituent parts. It may be that depressed individuals are able to make more optimal decisions when cognitive demands are low. Better decision-making, in turn, may lead to depression improvement.

Several limitations of the current study should be noted. First, we did not complete diagnostic interviews. Thus, we do not have information regarding past psychopathology, use of psychoactive substances or medications, or neurological diseases. In addition, because we did not screen for major depression, it is unknown whether clinically depressed individuals would perform similarly on the decision-making tasks used in this study. All depressed participants exceeded a cut-point on the CES-D commonly used to screen for major depressive disorder (Radloff, 1977); however, it is likely that many participants would not have met criteria for a major depressive episode. Second, we did not measure IQ, collect educational status, or assess other factors that might influence decision-making. Third, we do not know if findings will generalize to other tasks, including other decision-making tasks (e.g., lottery tasks utilized in behavioral economics; (Kahneman and Tversky, 1979)). Determining the boundary conditions for these associations will require further testing. Finally, consistent with a vast decision-making literature, tasks in the current studies used points as rewards and punishments. It will be important to determine whether use of actual incentives and punishments (e.g., monetary gains and losses) produce similar results.

Despite these limitations, this work provides important new insight into decision-making in depression. Depressed individuals (relative to people with few depression symptoms) excel at decision-making when the task involves punishment-minimization. Computational modeling suggested that depressed individuals were more likely to choose an option with the highest expected value and were more likely to alter expected values based on recently received rewards. Thus, people with depression symptoms do not have global decision-making deficits; rather, they show enhanced decision-making on cognitively simple tasks that emphasize minimizing loss.