Table 1. .
Reinforcement learning model parameters that could be altered in anhedonia
| Construct | Description | Computational instantiation | Evidence implicating | Evidence exonerating | Missing evidence |
|---|---|---|---|---|---|
| Value-guided behavior | Capacity of value representations to guide choice | Value (Equation 1) | Most studies report broadly intact acquisition | ||
| Feedback insensitivity | “Blunted” response to feedback, both positive and negative | Reduced learning rate (Equation 2) | Chase, Frank et al. (2010), Steele et al. (2007) | Rothkirch, Tonn, Kohler, & Sterzer (2017) | |
| Enhanced punishment sensitivity | Relatively enhanced response to negative feedback | Enhanced learning rate if outcome is aversive | Beevers et al. (2013), Herzallah et al. (2013), Maddox et al. (2012), Murphy, Michael, Robbins, & Sahakian (2003), Taylor Tavares et al. (2008) | Cavanagh, Bismark, Frank, & Allen (2011), Chase, Frank et al. (2010), Whitmer, Frank, & Gotlib (2012) | |
| Reduced reward sensitivity | Relatively reduced response to positive feedback | Reduced learning rate if outcome is appetitive | Beevers et al. (2013), DelDonno et al. (2015), Herzallah et al. (2013), Kunisato et al. (2012), Maddox et al. (2012), O. J. Robinson et al. (2012), Treadway, Bossaller, Shelton, & Zald (2012) | Cavanagh et al. (2011), Chase, Frank et al. (2010), Chase, Michael, Bullmore, Sahakian, & Robbins (2010), Whitmer et al. (2012) | |
| Pavlovian bias | Influence of reward- or punishment-predictive stimuli on behavior | See Equation 3 | Bylsma, Morris, & Rottenberg (2008), Huys, Golzer et al. (2016), Radke, Guths, Andre, Muller, & de Bruijn (2014); see Mkrtchian, Aylward, Dayan, Roiser, & Robinson (2017) for anxiety | ||
| Temperature | Stochastic choice | Temperature (Equation 4) | Huys et al. (2012), Huys et al. (2013), Kunisato et al. (2012); for indirect evidence, see Blanco, Otto, Maddox, Beevers, & Love (2013), Clery-Melin et al. (2011); for trend level, see Chase et al. (2017) | Chung et al. (2017), Rothkirch et al. (2017) | |
| Reduced outcome magnitude sensitivity | Linear or nonlinear scaling of utility across increasing expected value | [Outcome*sensitivity] or [Outcome^sensitivity] | Indirect evidence: Herzallah et al. (2013), Treadway et al. (2012) | ||
| Effort costs | Suppression of responding by effort | [Outcome value–effort cost] | Hershenberg et al. (2016), Treadway et al. (2012), Yang et al. (2016), Yang et al. (2014) | No simple increase in effort costs: Clery-Melin et al. (2011), Sherdell, Waugh, & Gotlib (2012) | |
| Working memory/“model-based” learning | Rapid adaptation of behavior in response to feedback | Various approaches, e.g., control choice in terms of previous outcome (Myers et al., 2016) | N/A | N/A | Little direct examination in MDD |
| Uncertainty-modulated learning | Increases or decreases in learning rate in response to uncertainty | Modulation of learning rate (e.g., Equation 2) by stimulus/outcome uncertainty | N/A | N/A | Little direct examination in MDD (but see Browning et al., 2015, on anxiety) |
Note. Here we define indirect evidence as suggestive that the construct might be significant, but this was not assessed directly via a modeling or other analytic strategy. To complete this table, combinations of the following terms were used in systematic searches: reward, model-based learning, Pavlovian, exploration, decision, choice, punishment learning, with anhedonia or major depression. The goal of the table is to provide an overview of salient exemplars of existing data from studies incorporating depressed, dysphoric, or euthymic individuals, which may be particularly relevant for the constructs listed.