Reward-based reinforcement learning is altered among individuals with a history of major depressive disorder and psychomotor retardation symptoms

Abstract

Reward-based reinforcement learning impairments are common in major depressive disorder, but it is unclear which aspects of reward-based reinforcement learning are disrupted in remitted major depression (rMDD). Given that the neurobiological substrates that implement reward-based RL are also strongly implicated in psychomotor retardation (PmR), the present study sought to test whether reward-based reinforcement learning is altered in rMDD individuals with a history of PmR. Three groups of individuals (1) rMDD with past PmR (PmR+, N = 34), (2) rMDD without past PmR (PmR−, N = 44), and (3) healthy controls (N = 90) completed a reward-based reinforcement learning task. Computational modeling was applied to test for group differences in model-derived parameters – specifically, learning rates and reward sensitivity. Compared to controls, rMDD PmR + exhibited lower learning rates, but not reduced reward sensitivity. By contrast, rMDD PmR− did not significantly differ from controls on either of the model-derived parameters. Follow-up analyses indicated that the results were not due to current psychopathology symptoms. Results indicate that a history of PmR predicts altered reward-based reinforcement learning in rMDD. Abnormal reward-related reinforcement learning may reflect a scar of past depressive episodes that contained psychomotor symptoms, or a trait-like deficit that preceded these episodes.

1. Introduction

Major Depressive Disorder (MDD) is a highly impairing and prevalent psychiatric disorder, where at least half of individuals experience multiple depressive episodes in their lifetime (Kessler and Walters, 1998; Kessler et al., 1997). Indeed, the single best predictor of future depression is past depression (Lewinsohn et al., 1989; Tram and Cole, 2006). Identifying processes that are disrupted in individuals with remitted depression could reveal key mechanisms of depression relapse.

One potentially important process through which depression develops, maintains, and recurs is via aberrant reward-based reinforcement learning (RL). RL refers to the ability to learn which actions are more likely to lead to rewarding outcomes. Reward-based RL is impaired in current MDD (Kumar et al., 2008; Pizzagalli et al., 2005, 2008) and has been shown to predict treatment response (Vrieze et al., 2013; Whitton et al., 2020). Specifically, individuals with current MDD often fail to exhibit a bias toward more rewarding stimuli relative to controls. Computational modeling, which is increasingly being utilized to identify specific cognitive processes that are impaired in various psychopathologies (Adams et al., 2016; Huys et al., 2016), indicates that reward-based RL deficits are, in part, related to reduced reward sensitivity (for a review, see Chen et al., 2015; Huys et al., 2013).

In contrast with current MDD, reward-based RL has been studied to a lesser extent in remitted MDD (rMDD) and even fewer studies have applied computational modeling to reward-based RL in rMDD. Similar to current MDD, there is emerging evidence of reward-based RL deficits among individuals with rMDD (Pechtel et al., 2013; Whitton et al., 2016). Although a history of MDD itself may predict impaired reward-based RL, MDD is a very heterogenous disorder (Merikangas et al., 1994; Østergaard et al., 2011), and a history of specific symptoms may be especially detrimental for reward-based RL in rMDD.

One depression symptom that may be particularly critical for reward-based RL is psychomotor retardation (PmR), which broadly describes reduced speed and amplitude in gross and fine motor activity (Schrijvers et al., 2009; Sobin and Sackeim, 1997). Although PmR has been understudied in MDD, studies of PmR are increasingly recognizing its clinical significance (Shankman et al., 2020; Walther et al., 2019). Motor symptoms not only correspond to more severe MDD (Calugi et al., 2011; Ulbricht et al., 2016) and poorer treatment response (Janzing et al., 2020; Taylor et al., 2006), but also to greater MDD recurrence (Gorwood et al., 2014). Thus, PmR may be a critical factor that relates to the development and/or recurrence of MDD.

PmR may be critical for reward-based RL due to overlapping neurocircuitry. Broadly, reward-based RL is implemented in the basal ganglia, including the dorsal striatum and ventral striatum, and regions of the prefrontal cortex (PFC; Galvan et al., 2005; O’Doherty, 2011; Pujara et al., 2016). Notably, PmR in MDD is also related to structural and functional alterations of many of these regions and related circuits (Bracht et al., 2012; Martinot et al., 2001; Naismith et al., 2002). Additionally, reward-based RL is supported by dopaminergic functioning (Samson et al., 2010), which is impaired individuals with PmR (Martinot et al., 2001). Taken together, a history of PmR, which may be indicative of trait-like corticostriatal and/or dopaminergic dysfunction, may relate to key deficits in rMDD, including reward-based RL.

The primary goal of the present study was to identify whether reward-based RL is altered among individuals with remitted MDD with a history of PmR symptoms (rMDD PmR+) relative to (a) healthy controls and (b) rMDD without a history of PmR symptoms (rMDD PmR−). To assess reward-based RL, participants completed the Probabilistic Reward Task (PRT), and computational modeling was applied to participants’ trial-wise choice behavior. It was hypothesized that individuals with rMDD would exhibit a reduced response bias (indicated by traditional PRT measures), as well as computational evidence of impaired reward-based RL relative to controls. Additionally, it was expected that differences in reward-based RL deficits would emerge for rMDD PmR + relative to rMDD PmR−, but not rMDD PmR-relative to controls. Given the specific links between dopamine and (a) psychomotor functioning and (b) learning rates in previous research (e.g., Martinot et al., 2001; Pizzagalli et al., 2020), it was anticipated that reward-based RL differences for the rMDD PmR + would most likely emerge for the learning rate. Finally, it was expected that group differences in reward-based RL would not be accounted for by current clinical symptoms, including anhedonia.

2. Methods

2.1. Participants

Young adults (N = 504) were recruited as part of a larger family study of transdiagnostic mechanisms of internalizing psychopathology from mental health clinics and the local Chicago, Illinois community. Primary eligibility criteria included (1) age 18-30 years old and (2) having a biological sibling age 18–30 who was also interested in participating (for full method details, see Gorka et al., 2016; Weinberg et al., 2015). Participants were oversampled during recruitment for severe internalizing psychopathology symptoms using the Depression, Anxiety, and Stress Scale (Lovibond and Lovibond, 1995; see Supplement for additional details).

Individuals were selected for the present study based on lifetime MDD and PmR symptoms using the Structured Clinical Interview for DSM-5 (SCID-5; First et al., 2015). Participants were included in one of three groups: (1) rMDD PmR+, if they met criteria for past but not current MDD and had PmR symptoms during their past major depressive episode, (2) rMDD PmR−, if they met criteria for past but not current MDD and did not have PmR symptoms during their past depressive episode, and (3) controls, if they had no lifetime history of any psychiatric disorder or PmR. Individuals who met criteria for current MDD and/or had current PmR and/or agitation (which broadly describes increases in purposeless motor activity; Sobin and Sackeim, 1997), on the SCID-5 were excluded from the study. Participants were also excluded if they did not have viable PRT data (Pizzagalli et al., 2005; Whitton et al., 2016; see Supplement). The final sample included 34 participants in the rMDD PmR + group, 44 in the rMDD PmR-group, and 90 in the control group (total N = 168).

2.2. Assessments and measures

2.2.1. Clinical diagnoses and psychomotor symptoms

Diagnoses and psychomotor symptoms were assessed via the SCID-5 (see Supplement for additional details). In contrast with traditional SCID assessment procedures, interviewers assessed all symptoms of MDD for all participants, even if the cardinal symptom(s) were not endorsed. Thus, lifetime psychomotor symptoms were characterized for all participants, allowing for the exclusion of individuals who did not meet criteria for MDD but endorsed lifetime psychomotor agitation (across groups) and/or lifetime PmR in the rMDD PmR− and control groups.

2.2.2. Inventory of depression and Anxiety Symptoms (IDAS-II)

The IDAS-II (Watson et al., 2012) is a 99-item questionnaire that measures current symptoms of depression and anxiety. In the present study, General Depression (Well-Being items removed), Well-Being (reverse scored to capture anhedonia), Panic, and Social-Anxiety were included as covariates of current internalizing symptoms. The observed internal consistencies were in the good-excellent range (General Depression: α = 0.91, Well-Being: α = 0.86, Panic: α = 0.80, and Social Anxiety: α = 0.86).

2.2.3. Personality Inventory for DSM-5 (PID-5)

The PID-5 (Krueger et al., 2012) is a 220-item measure of pathological personality traits and has strong psychometric properties (Thomas et al., 2013; Katz et al., 2018). The PID-5’s Anhedonia facet was included as a covariate to account for the trait-like tendency for individuals to experience anhedonia. The observed internal consistency was in the excellent range (α = 0.93).

2.2.4. Psychomotor speed

Condition 5 (Motor Speed) from the Delis-Kaplan Executive Function System Trail Making Test (Delis et al., 2001) assessed current psychomotor speed. During this condition participants connect empty dots while tracing over a dotted line as quickly as possible. Reaction time was used as the dependent measure.

2.2.5. Probabilistic Reward Task (PRT)

The PRT (Pizzagalli et al., 2005) is a well-validated probe of reward-based RL. The task included 200 trials across two blocks (i.e., 100 trials per block) and was administered in E-prime using code based on Pizzagalli et al. (2005). The task administration followed the instructions that were outlined by Pizzagalli and Whitton (2015, unpublished manual). During this task, participants are first presented with a jittered fixation cross, followed by two dots (“eyes”) and vertical line (“nose”). This is followed by either a short (10 mm) or long (11 mm) horizontal line (“mouth”; see Fig. 1). Participants must select the identity of each cue by pressing a corresponding keyboard key (‘z’ or ‘/‘). The cues represent a “lean” and a “rich” stimulus that are rewarded probabilistically (counterbalanced across participants), with the rich stimulus 3x more likely to deliver a reward following a correct response than the lean stimulus. During the feedback phase, participants either receive feedback indicating that they were correct and won 20 cents, or they receive no feedback.

Fig. 1.

Schematic representation of a trial from the Probabilistic Reward Task.

Traditional PRT measures that were calculated included response bias and discrimination (Pizzagalli et al., 2005; Whitton et al., 2016; Pizzagalli and Whitton, 2015, unpublished manual; see Supplement for the response bias and discrimination formulas). In addition to looking at overall response bias and discrimination, differences in the two PRT measures between blocks (Block 2-Block 1) were computed for each participant (Δresponse bias and Δdiscrimination).

2.2.6. Computational modeling of reinforcement learning

Several models that build on the standard Rescorla-Wagner (RW) model have been applied to the PRT task in previous research (Huys et al., 2013). These models were fit to participants’ behavior, including the ‘Action’, ‘Belief’, and ‘Stimulus-Action’ models. All models took the following general form:

$$Q_{t+1,c}=Q_{t,c}+\alpha \times {\delta }_t,$$

where $Q_{t,c}$ is the quality, or value estimation, of cue c on trial t, α is the learning rate (which takes on the same value across all trials), and ${\delta }_t$ is the prediction error (PE; $\rho \times {\mathit{\text{outcome}}}_t-Q_{t,c}$) on trial t. To capture reward-sensitivity, parameter ρ was included in the PE term, which scales the value of the outcome, with $0\le \rho \le 1$ and values closer to 1 reflecting greater reward sensitivity.

The probability of choosing key 1 (‘z’) or 2 (’/‘) was linked to participants’ expectations of reward using a softmax function. Specifically, value expectations $\left(Q_{t,c}\right)$ were transformed using a softmax that captures the difference between weighted $\left(W_{\text{t}}\right)$ value expectations for each action (choosing the ‘z’ key and choosing the ‘/’ key), with larger differences leading to an increased likelihood of selecting one action over the other:

$$p\left(a_t\mid s_t\right)=1/1+\exp \left[-\left(W_t\left(a_t,s_t\right)-W_t\left({\overline{a}}_t,s_t\right)\right)\right],$$

where at reflects the selected action and ${\overline{a}}_{\text{t}}$ reflects the unselected action for the presented stimulus s_t on trial t. The weighted value estimates that are included in the softmax function for each action $W_{\text{t}}\left(a_{\text{t}},{\text{s}}_{\text{t}}\right)$ and $W_{\text{t}}({\overline{a}}_{\text{t}},{\text{s}}_{\text{t}})$ reflect a weighted combination of value estimates for each stimulus:

$$W_t\left(a_t,s_t\right)=\varsigma \times Q_t\left(a_t,s_t\right)+(1-\varsigma )\times Q_t\left(a_t,{\overline{s}}_t\right),$$

where $\overline{s}\text{t}$ reflects the stimulus that was not presented on trial t. The parameter that scales the value estimates, ζ, reflects the degree to which a learner is certain of the identity of the stimulus on screen, with $0\le \zeta \le 1$ and values closer to 1 reflecting greater certainty.

For the Stimulus-Action model, $\zeta =1$ (i.e., a learner is certain which stimulus is the long versus short cue), thus nullifying the second term from the model, $(1-\zeta )\times Q_{\text{t}}\left(a_{\text{t}},{\overline{s}}_{\text{t}}\right)=0$. For the Action model, $\zeta =0.5$ (i.e., a learner is uncertain which stimulus is the long versus short cue) and each term, $W_{\text{t}}\left(a_{\text{t}},{\text{s}}_{\text{t}}\right)$ and $W_{\text{t}}\left(a_{\text{t}},{\overline{s}}_{\text{t}}\right)$, is weighed equally. For the Belief model, ζ is a free parameter that is estimated for each learner, which captures individual differences in certainty about stimulus identity.

Models were fit to participants’ choice behavior using maximum likelihood estimation and Akaike Information Criterion (AIC) values were used to identify the best-fitting model.

2.2.7. Analytical approach

Analyses examined the effect of remitted MDD with a history of PmR on reinforcement learning using linear mixed effects models. Specifically, we tested whether remitted MDD with a history of PmR symptoms (rMDD PmR+) differed from controls or rMDD without a history of PmR symptoms (rMDD PmR−) on RL using dummy coding with rMDD PmR + as the reference group. We also examined whether behavior or modeling parameters differed for rMDD PmR-relative to controls.

Analyses assessed for group differences in (1) traditional PRT behavioral measures and (2) model-estimated (a) learning rates, α, and (b) reward sensitivity, ρ (from the best-fitting model). Because these parameters are typically correlated (Huys et al., 2013), follow up tests examined unique effects for the parameters. Age and gender were included as covariates, and family was included as a random effects factor to account for sibling relatedness (i.e., familiality). Follow-up analyses examined whether the inclusion of psychopathology covariates altered the results.

3. Results

3.1. Sample characteristics

As shown in Table 1, there were significant no group differences in age, F(2,165) = 2.35, p = .098 or self-identified gender, χ2(2) = 1.76, p = .415, and the sample was quite diverse. Means and standard deviations for clinical measures and psychomotor speed are also reported in Table 1. Both rMDD groups endorsed more current symptoms of general depression, social anxiety, panic, and anhedonia than controls, but critically, the two rMDD groups did not differ on any of the clinical measures. Psychomotor speed did not significantly differ across groups, F(2,165) = 2.85, p = .061.

Table 1.

Demographic, cognitive, and clinical characteristics.

	rMDD PmR+n = 34	rMDD PmR− n = 44	Controls n = 90
Age	22.4 (3.3)	23.3 (3.6)	22.0 (3.2)
Gender	77% female	68% female	64% female
Ethnicity
White	38%	63%	38%
Black/African American	24%	8%	7%
Hispanic	32%	17%	19%
Asian	6%	6%	26%
Middle Eastern	–	–	8%
Mixed Race	–	6%	2%
IDAS Subscales
Depression (Well-Being items removed)a	36.2 (11.3)	35.1 (12.0)	25.0 (7.0)a,b
Well-Beinga	25.2 (5.8)	25.3 (7.4)	27.32 (6.4)
Social Anxietya	11.4 (5.4)	10.4 (4.5)	7.1 (1.9)a,b
Panica	11.9 (4.6)	10.3 (2.6)	9.0 (2.4)a,b
PID-5 Facets
Anhedoniab	0.73 (.51)	0.71 (.69)	0.36 (.38)a,b
Current Subthreshold Psychomotor Symptoms
Psychomotor Retardation	n = 5 (15%)	n = 0	n = 0
Psychomotor Agitation	n = 3 (9%)	n = 1 (2%)	n = 0
Lifetime Subthreshold Psychomotor Symptoms
Psychomotor Retardation	-	n = 0	n = 0
Psychomotor Agitation	n = 0	n = 9 (21%)	n = 1 (1%)
Motor Speed (D-KEFS Trail Making Test, Condition 5)c	26.2 (12.6)	22.2 (6.5)	22.0 (8.3)

Note. IDAS=Inventory of Depression and Anxiety Symptoms; PID-5 = Personality Inventory for DSM-5.

^a,bScores significantly differed for controls relative to the rMDD groups.

^arMDD PmR+: n = 34, rMDD PmR−: n = 43, Controls: n = 89.

^brMDD PmR+: n = 34, rMDD PmR−: n = 43, Controls: n = 90.

^crMDD PmR+: n = 34, rMDD PmR−: n = 42, Controls: n = 85.

Given evidence that smoking status is associated with RL performance and related parameters (Baker et al., 2020; Reynolds, 2006), we examined whether current smoking status varied across groups. There were very few current smokers across all groups (i.e., those who smoked one or more cigarettes daily; see Table 2), and, importantly the ratio of current smokers to non-smokers within each group did not differ between groups, χ2(2) = 1.40, p = .496. Additionally, because some psychiatric medications have been shown to relate to altered RL (e.g., Kumar et al., 2008), we tested for group differences in current psychiatric medication use. As shown in Table 2, the only group difference in medication usage was for the rMDD groups relative to controls, χ2(2) = 10.72, p = .005; both rMDD groups had higher rates of current SSRI use, but SSRI rates did not differ between the rMDD groups, χ2(1) = 0.04, p = .847.

Table 2.

Rates of current psychiatric medication use and smoking status across groups.

	rMDD PmR+	rMDD PmR−	Controls
Medication
Anxiolytics	3% (n = 1)	2% (n = 1)	0%
SSRI	12% (n = 4)	9% (n = 4)	0%a,b
SNRI	0%	2% (n = 1)	0%
NDRI	3% (n = 1)	2% (n = 1)	0%
MAOI	0%	0%	0%
Sedatives	0%	0%	0%
Hypnotics	0%	0%	0%
# of Medications	0: 88% (n = 30)	0: 89% (n = 39)	0: 100% (n = 90)
	1: 6% (n = 2)	1: 9% (n = 4)
	2: 6% (n = 2)	2: 2% (n = 1)
Currently smoke at least 1 cigarette daily	3% (n = 1)	9% (n = 4)	4% (n = 4)

^arMDD PmR + vs. Controls: t(165) = 2.80, B = .22, p = .006.

^brMDD PmR-vs. Controls: t(165) = 2.37, B = .19, p .019.

3.2. Traditional PRT measures

Means and standard deviations for traditional PRT measures are provided in Table 3. Although accuracy for the rich stimulus, response bias, and discrimination measures were all numerically lower and reaction times were longer for the lean and rich stimuli for the rMDD PmR + group, no significant group differences emerged for the traditional measures.

Table 3.

Traditional measures from probabilistic reward task.

	rMDD PmR+	rMDD PmR−	Controls	Group Comparison
Accuracy	.76 (.06)	.77 (.09)	.77 (.07)	F(2,163) = 0.04, p = .957
Lean	.71 (.09)	.72 (.11)	.71 (.12)	F(2,163) = 0.24, p = .787
Rich	.83 (.06)	.85 (.06)	.85 (.07)	F(2,163) = 0.80, p = .451
Reaction Time
Rich	433.63 (77.4)	418.88 (101.9)	421.76 (82.3)	F(2,163) = 0.19, p = .827
Lean	473.61 (91.1)	446.64 (107.0)	458.33 (105.6)	F(2,163) = 0.40, p = .671
Response Bias	.15 (.15)	.17 (.16)	.21 (.20)	F(2,163) = 1.37, p = .258
ΔResponse Bias	.05 (.15)	.04 (.30)	.06 (.23)	F(2,163) = 0.07, p = .937
Discrimination	.55 (.14)	.58 (.21)	.59 (.20)	F(2,163) = 0.57, p = .564
ΔDiscrimination	.03 (.19)	.03 (.18)	.02 (.18)	F(2,163) = 0.30, p = .742
Reward Ratio (Rich vs. Lean)
Block 1	29.4/9.8	29.4/9.9	29.4/10	F(2,163) = 1.67, p = .191
Block 2	29.6/9.7	29.7/9.9	29.8/9.8	F(2,163) = 0.90, p = .411
Valid Trials (%)	98 (.03)	98 (.01)	99 (1.0)	F(2,163) = 0.51, p = .601

3.3. Model fit

As shown in Fig. 2A, the Action model provided the best fit based on AIC across all participants (log-likelihood values are provided in the Supplement). For all groups, the Action model provided a better fit than the Belief and Stimulus-Action models (see Fig. 2B). Not only did the Action model provide a better fit relative to other models for all groups, the model fit did not significantly differ between groups, F(2,165) = 2.38, p = .096.

Fig. 2.

A. Summed Akaike information criterion (AIC) values for the Stimulus-Action, Action, and Belief Models across participants. AIC values indicate that participants’ behavior was better captured by the Action Model than the Stimulus-Action and Belief Models. B. Average AIC values for the Action, Stimulus-Action, and Belief Models for each group. AIC values indicate that for each group, participants’ behavior was better captured by the Action Model than the Stimulus-Action and Belief Models.

3.4. Learning curves

Value estimates, which are a function of model parameters and participants’ choices, were recovered using the best-fitting (Action) model. Trial-wise changes in value estimates across the task (i.e., learning curves) are depicted for each group in Fig. 3.

Fig. 3.

Model-derived trial by trial value expectations (V) averaged across the remitted major depressive disorder (rMDD) and control groups.

3.5. Learning rate and reward sensitivity

Across all groups, the learning rate was correlated with accuracy for the rich, r(166) = 0.18, p = .023 and lean stimuli, r(166) = 0.16, p = .030, but reward sensitivity did reach the level of significance for either stimulus, r(166) = 0.14, p = .076 and r(166) = 0.13, p = .094 (note: the parameters were transformed due to non-Gaussianity and scaled such that larger values are more positive). When included as predictors in the same model, neither parameter accounted for a unique portion of variance in accuracy, rich: B = 0.14, p = .103 and B = 0.06, p = .559, and lean: B = 0.16, p = .089 and B = 0.05, p = .661.

For the learning rate (α), there was a significant effect of Group, F (2,163) = 4.19, p = .017, which was driven by higher learning rates for controls relative to the rMDD PmR + group, t(163) = 2.85, B = 0.27, p = .005. Although numerically lower for rMDD PmR-relative to rMDD PmR+, learning rates did not significantly differ for rMDD groups, t (163) = 1. 43, B = 0.14, p = .154. Additionally, learning rates did not differ for the rMDD PmR-group relative to controls, t(163) = −1.40, B = −0.11, p = .164. Follow-up tests revealed that the difference in learning rates between rMDD PmR+ and controls remained after including psychopathology covariates (e.g., current symptoms of depression, anhedonia, social anxiety, and panic), as well as current psychiatric medication use (see Table 4).

Table 4.

Effect of group (PmR + versus controls) on learning rate for controlling for psychopathology and current psychiatric medication use covariates.

DV: α	PmR + vs. Controls	Covariate
1.	B=.28, p=.005	B = −.13, p = .096
2.	B=.27, p=.009	B = .01, p = .911
3.	B=.29, p=.008	B = .05, p = .536
4.	B=.24, p=.029	B = −.05, p = .589
5.	B=.28, p=.009	B = .02, p = .769
6.	B=.28, p=.005	B=−.02, p=.971
7.	B=.28, p=.006	B=.04, p=.634

¹ = Inventory of Depressive and Anxiety Symptoms (IDAS), Well-being (reverse-scored).

² = Personality Inventory for the DSM-5, Anhedonia.

³ = IDAS, Depression.

⁴ = IDAS, Social Anxiety.

⁵ = IDAS, Panic.

⁶ = Current SSRI medication use.

⁷ = Any current psychiatric medication use.

For reward sensitivity (ρ) there was no main effect of Group, F (2,163) = 1.45, p = .238. To identify whether the difference in learning rates between the rMDD PmR+ and control groups remained after controlling for reward sensitivity (which was significantly correlated with learning rate, r[166] = −0.34), the reward sensitivity parameter was included as a covariate. The difference in learning rates between controls and the rMDD PmR + group remained significant after controlling for reward sensitivity, B = .22, p = .023.

3.6. Alternative model fitting

To identify whether the model that provided the best fit to the data varied as a function of estimation method, model fit was also examined using Expectation-Maximization (Huys et al., 2013; emfit toolbox, available at https://github.com/mpc-ucl/emfit). This estimation method also identified the Action model as the best-fitting model (see the Supplement). Multivariate regression analyses were conducted to test for group differences in the reward sensitivity and learning rate parameters. Similar to the results for reward sensitivity and learning rate tests above, the learning rate parameter differed for rMDD PmR + relative to Controls (p = .023), but not the rMDD PmR-group (p = .281). Additionally, controls and the rMDD PmR-groups did not differ in the learning rate (p = .734). The reward sensitivity parameter did not significantly differ for any of the groups.

4. Discussion

Psychomotor retardation symptom history predicted altered reward-based RL in individuals with rMDD. Whereas group differences did not reach significance for traditional PRT measures, differences emerged using more specific parameters that were identified with computational modeling. These findings add to a growing literature of atypical reward-based RL in individuals with rMDD (Pechtel et al., 2013; Whitton et al., 2016), and, furthermore, highlight a potentially important role of psychomotor retardation in reward-based RL in MDD.

Despite comparable computational model fit across groups, indicating that participants completed the task using similar learning strategies, individuals with rMDD with a history of PmR exhibited lower learning rates compared to controls. Learning rates reflect the degree to which individuals revise their expectations based on the error in their predictions. Although higher learning rates are not inherently “better,” since persistently large updates can disrupt the convergence between a learner’s internal expectations and actual stimulus-outcome probabilities, there was a trend for higher learning rates to relate to greater accuracy. This suggests that, at least in the context of this implicit learning task in which signal detection was difficult, relatively higher learning rates were beneficial (see Pizzagalli et al., 2020 for a similar finding for learning rate and PRT accuracy). In contrast with learning rates, reward sensitivity, which reflects how much individuals weigh reward-related outcomes in their expectations, did not differ between the rMDD PmR+ and control groups. None of the results changed after controlling for current symptoms of psychopathology, suggesting that effects were not due to residual current symptoms.

In contrast with rMDD with a history of PmR, rMDD individuals without a history of PmR did not differ from controls on any of the traditional or computational task measures, including learning rates. Although neither learning rates nor reward sensitivity significantly differed between the rMDD groups, the learning curves show that individuals with rMDD with a history of PmR exhibited slower initial learning of action-outcome pairings relative to rMDD without a history of PmR and controls. Taken together, results indicate that individuals with rMDD with a history of PmR, rather than rMDD broadly, exhibit altered reward-based RL.

Results are in line with previous research that identified reward-based RL is disrupted in individuals with a history of MDD, even after accounting for symptoms of perceived stress and anhedonia (Petchel et al., 2013), the latter of which correlates with the reward sensitivity parameter (Huys et al., 2013; Whitton et al., 2020). Although current MDD and anhedonia have related to reduced sensitivity to rewards in previous work (Huys et al., 2013), a history of MDD has been found to only weakly relate to atypical model-derived reward sensitivity (Huys et al., 2013). That reward sensitivity is not the primary contributor to reward-based RL deficits in rMDD is further supported by present results. While this is not the first study to identify a role of other reward-based RL processes in MDD (Chen et al., 2015), to our knowledge it is the first to identify a primary role of altered learning rates in reward-based RL in individuals with rMDD who have experienced PmR.

Despite not having clinically elevated current PmR symptoms or significantly slower psychomotor speed (as indicated by D-KEFS motor speed performance and RT during the PRT), a history of PmR during a past episode of MDD predicted current alterations in reward-based RL. This suggests that difficulties in rMDD with reward-based RL are not merely the result of psychomotor slowing. As discussed above, there is evidence that corticostriatal circuitry and dopaminergic functioning that supports reward-based RL are disrupted in PmR (Bracht et al., 2012; Naismith et al., 2002). Although indirect evidence of these links, administration of a low dose dopaminergic D₂ agonist (which has antagonistic effects on behavior) has been found to disrupt learning rates, but not reward sensitivity, during reward-based RL (Huys et al., 2013; Pizzagalli et al., 2008); a pattern that emerged in our study. These findings are particularly relevant for precision medicine, which aims to identify specific prevention efforts and treatments for specific individuals (Fernandes et al., 2017). For example, computationally-derived PRT parameters could to help identify which individuals may benefit from enhancing dopaminergic functioning to prevent MDD relapse.

There are several limitations that should be noted. First, psychomotor symptom history was based on a retrospective clinical assessment via the SCID. Importantly though, the psychomotor retardation is one of the symptoms of lifetime MDD with the strongest test-retest reliability (k = .72; Kaiser et al., 2020). This suggests that specific symptoms of lifetime MDD, including psychomotor retardation, are reported consistently over time. Another limitation is that this study is unable to clarify whether abnormal reward-based RL processing reflects a scar of past MDD episodes that contained psychomotor symptoms or a trait-like deficit that preceded episodes. In the future, it will be important to conduct prospective studies that include fine-grained assessments of psychomotor symptoms, including laboratory-based and other instrumental measures (e.g., actigraphy, an ecologically sensitive approach to assessing movement in daily life; Shankman et al., 2020). Finally, while the research questions and groupings were determined prior to implementing the study, participants were included from a larger study that was did not initially recruit participants specifically for rMDD and PmR history.

It is also important to note that our results did not replicate a previously established finding of an altered response bias for rMDD (Pechtel et al., 2013; Whitton et al., 2016). Our results are also in slight contrast with Huys et al. (2013), who found that although both the Action and Belief models provided a good fit to performance, the Belief model slightly outperformed the Action model. By contrast, we found that the Action model provided a better fit, which reflects the weighing of previous actions (e.g., pressing key 1 or key 2) to a greater degree than beliefs about stimulus identity. This suggests that participants generally did not have a high degree of certainty about stimulus identity during cue presentation. It is plausible that task design features may have played a role in these differences. First, Huys et al. (2013) included data from six studies and all of the studies included 3 blocks of 100 trials, which was also the number of blocks included in Pechtel et al. (2013), whereas our study included 2 blocks of 100 trials (i.e., 1/3 fewer trials). Second, for all but one study in Huys et al. (2013), the difference between the mouth size of the two cues was slightly larger (13 mm vs. 11.5 mm) than the one used in the present study (11 mm vs. 10 mm). Indeed, the one study that used the same mouth sizes as our task (Bogdan and Pizzagalli, 2006) exhibited the poorest fit to the Belief model. In the future, researchers could consider assessing the role of psychomotor symptom history in rMDD using different combinations of these task features to clarify mechanisms of altered RL.

Reward-based RL is a critical cognitive process that is impaired in current and remitted depression. In the present study, there was evidence of less efficient experience-dependent reward-based RL among individuals with rMDD who have a history of psychomotor slowing. In daily life, these difficulties could contribute to and/or perpetuate disengagement from activities and interpersonal relationships. Targeting (a) residual PmR symptoms in individuals with a history of MDD and/or (b) reward-based RL in individuals with a history of PmR may help to prevent relapse into future depressive episodes.

Data availability statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.