Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 May 1.
Published in final edited form as: Psychol Med. 2014 Oct 16;45(7):1413–1424. doi: 10.1017/S0033291714002517

Corticostriatothalamic Reward Prediction Error Signals and Executive Control in Late-Life Depression

Alexandre Y Dombrovski 1, Katalin Szanto 1, Luke Clark 2, Howard J Aizenstein 1, Henry W Chase 1, Charles F Reynolds 3rd 1, Greg J Siegle 1
PMCID: PMC4380546  NIHMSID: NIHMS667524  PMID: 25319564

Abstract

Background

Altered corticostriatothalamic encoding of reinforcement is a core feature of depression. Here we examine reinforcement learning in late-life depression in the theoretical framework of the vascular depression hypothesis. This hypothesis attributes the co-occurrence of late-life depression and poor executive control to prefrontal/cingulate disconnection by vascular lesions.

Methods

Our fMRI study compared 31 patients with major depression aged 60+ to 16 controls. Using a computational model, we estimated neural and behavioral responses to reinforcement in an uncertain, changing environment (probabilistic reversal learning).

Results

Poor executive control and depression each explained distinct variance in corticostriatothalamic response to unexpected rewards. Depression, but not poor executive control, predicted disrupted functional connectivity between the striatum and prefrontal cortex. White-matter hyperintensities predicted diminished corticostriatothalamic responses to reinforcement, but did not mediate effects of depression or executive control. In two independent samples, poor executive control predicted a failure to persist with rewarded actions, an effect distinct from depressive oversensitivity to punishment. The findings were unchanged in a subsample of participants with vascular disease. Results were robust to effects of confounders including psychiatric comorbidities, physical illness, depressive severity, and psychotropic exposure.

Conclusions

Contrary to the predictions of the vascular depression hypothesis, altered encoding of rewards in late-life depression is dissociable from impaired contingency learning associated with poor executive control. Functional connectivity and behavioral analyses point to a disruption of ascending mesostriatocortical reward signals in late-life depression and a failure of cortical contingency encoding in elderly with poor executive control.

Keywords: prefrontal cortex, executive control, aged, reward, reinforcement, depression

Introduction

One cardinal feature of depression is weakened or distorted behavioral effectiveness of reinforcement (reviewed in (Eshel and Roiser, 2010)). Learning theory suggests that the problem may lie in representing unexpected rewards that normally drive appetitive behavior (Bush and Mosteller, 1951, Rescorla and Wagner, 1972). A reward’s behavioral effectiveness depends not only on its magnitude, but also on the amount of surprise, since surprise results in learning. This surprise is quantified by the reward prediction error—the difference between the actually received and expected reward. In every episode of learning (trial, t), expected reward is updated from its previous value by the prediction error (Eq. 1, simplified for illustration):

ExpectedRewardt=ExpectedRewardt-1+α∗RewardPredictionErrortRewardPredictionErrort=ActualRewardt-ExpectedRewardt-1 Eq. 1

Reward expectancy, in turn, controls approach behavior. Variants of this simple learning rule underlie all reinforcement learning models (Sutton and Barto, 1998). Equation 1 helps understand the asymptotic learning curves observed in operant conditioning (Bush and Mosteller, 1951, Skinner, 1938). Early in learning, a rat rewarded for lever presses has low reward expectancy. Early rewards generate large prediction errors, driving a rapid increase in the response rate. Late in learning, reward expectancy becomes high, prediction errors wane, and the response rate stabilizes.

Neural reward prediction error signals are thought to originate in the mesostriatal dopamine pathway (Schultz et al., 1997). They are believed to shape cognitive processes and behavior through synaptic modification of corticostriatothalamic circuits [see (Glimcher, 2011), (Chase et al., 2015) for a theoretical review and meta-analysis of human imaging studies]. Specifically, trial-by-trial learning signals are thought to train sustained, multi-trial prefrontal representations of the reward contingency (Histed et al., 2009, Pasupathy and Miller, 2005). In human imaging, the degree of co-variation between model-estimated reward prediction errors and BOLD is typically assumed to reflect individual differences in corticostriatothalamic prediction error signals (Dombrovski et al., 2013, Gradin et al., 2011, Kumar et al., 2008). In depression, corticostriatothalamic circuits display altered responses to rewards and punishments in general (Eshel and Roiser, 2010) and diminished reward prediction error signals in particular (Gradin et al., 2011, Kumar et al., 2008). Using fMRI, we have recently extended these observations to late-life depression (Dombrovski et al., 2013). We also observed that, in depressed individuals, oversensitivity to punishments (Murphy et al., 2003) was inversely related to the strength of corticostriatothalamic reward prediction error signals (Dombrovski et al., 2013).

These observations reveal certain neural correlates of altered reward-guided behavior in depression, yet its specific neural mechanisms remain unclear. One particularly influential account of behavioral disturbance in depression emphasizes failures of top-down control, broadly linked to prefrontal dysfunction (Robbins, 2007). Executive control is impaired in depression (Snyder, 2013), and the co-occurrence of depression and poor executive control forms the basis for the vascular depression hypothesis (Alexopoulos et al., 1997, Taylor et al., 2013). It attributes depression and poor executive control in old age to prefrontal and cingulate disconnection from subcortical structures by ischemic white matter lesions (Aizenstein et al., 2011, Alexopoulos, 2002, Alexopoulos et al., 2012, Sneed et al., 2007). This hypothesis is supported by findings of poor treatment response in patients with poor executive control and white matter hyperintensities (WMH)(Baldwin et al., 2004, Barch et al., 2012, Sneed et al., 2007). Can the vascular depression hypothesis explain alterations in reward-guided behavior and reward signals seen in late-life depression? Our fMRI study of reward-guided behavior tested the prediction that both depression and poor executive control are related to a disconnection of the prefrontal and cingulate cortices from the striatum and thalamus.

To establish that we possess the appropriate index of corticostriatothalamic integrity for testing the predictions of the vascular depression hypothesis, we assessed whether individual variation in prediction error signals scaled with executive control ability in patients with late-life depression. We were then able to test whether depression and executive control accounted for shared variance in corticostriatothalamic integrity (indexed by prediction error signals), as would be the case if they were both caused by disconnection. Then, to obtain more direct evidence of functional striato-cortical disconnection, we tested the effects of depression and poor executive control on striato-cortical functional connectivity. Next, to examine the role of vascular lesions, we tested whether (i) WMH disrupted corticostriatothalamic responses and striato-cortical functional connectivity and (ii) whether they explained the relationship of these indices with depression and executive control. All analyses were repeated in a subsample with vascular conditions. Finally, we tested whether late-life depression and poor executive control had similar or distinct effects on reward- and punishment-guided behavior in two non-overlapping samples.

Methods

Participants and diagnosis of depression (Table 1)

Table 1.

Demographic, clinical and cognitive characteristics

Characteristics Non-psychiatric controls (n = 16) Depressed (n = 31) F χ2 P value
Male, No. (%) 6 (38) 12 (40) - 1.13 0.57
Age, mean (SD), y 71 (8.0) 66 (5.4) 5.91 0.02
White, mean (SD), y 14 (88) 22 (73) - 4.38 0.36
Education, mean (SD), y 13.8 (1.9) 14.5 (3.0) 0.92 - 0.34
Premorbid IQ estimate,1 mean (SD) 103 (9) 105 (16) 0.31 - 0.58
Dementia rating scale, mean (SD) 138 (3.0) 136 (5.8) 2.03 - 0.16
Executive interview, mean (SD) 7 (4) 8 (4) 0.16 - 0.70
Physical illness burden,2 mean (SD) 6.5 (2.5) 9.3 (3.8) 6.41 - 0.02
Hamilton depression scale,3 mean (SD) 2.9 (3.5) 12.3 (7.3) 23.0 - <0.001
Beck hopelessness scale, mean (SD) 0.8 (0.9) 7.2 (6.2) 17.2 - <0.001
Antidepressant exposure,4 mean (SD) - - 3.3 (1.9) - - -
Lifetime substance use, No. - - 8 - - - -
Lifetime anxiety, No. - - 14 - - - -
Open in a new tab
1

Wechsler Test of Adult Reading, standard score

2

Cumulative Illness Rating Scale, adapted for geriatrics

3

Hamilton Rating Scale for Depression

4

Antidepressant Treatment History Form, cumulative score

Between January 1 of 2008 and August 31 of 2011, we recruited 47 participants aged 60 and older: 31 with major depressive disorder (MDD) and 16 psychiatrically healthy controls. MDD was diagnosed by Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition Axis I Disorders(2000, First M, 1995) (SCID/DSM-IV). Participants with MDD had symptoms of varying severity (Table 1; examined in Sensitivity analyses). To exclude individuals with dementia, all were required to have a MMSE score of ≥24. Our reported sample is exclusive of elderly with sensory disorders that precluded cognitive testing, limited English, mental retardation, delirium, neurologic disorders including strokes, bipolar disorder, schizophrenia, schizoaffective disorder, exposure to electroconvulsive therapy in the previous 6 months, and conditions precluding an fMRI assessment. This is a subsample of participants described in our earlier report (Dombrovski et al., 2013), exclusive of 6 participants missing assessments of executive control.

All participants provided written informed consent. The University of Pittsburgh Institutional Review Board approved the study.

Assessments

Cognitive and clinical characterization

Current global cognitive function was assessed with the Dementia Rating Scale (DRS) (Mattis, 1988). Depression severity was measured with the 17-item Hamilton Rating Scale for Depression (HRSD) (Hamilton, 1960). Burden of physical illness was assessed with the Cumulative Illness Rating Scale adapted for Geriatrics (CIRS-G) (Miller et al., 1992). We measured antidepressant exposure in the current episode of depression using the Antidepressant Treatment History Form (Sackeim, 2001), based on antidepressant trial duration, dose, and use of augmenting agents. We additionally assessed exposure to sedatives/hypnotics, anticholinergics, and opioids.

Executive control

We assessed executive control with the Executive Interview (EXIT; range 0–50) (Royall et al., 1992). The 25 items are administered in rapid succession with minimal instructions to elicit automatic behaviors. Items include modifications of well-known “frontal lobe” tests (number/letter sequencing, Stroop, fluency, go/no-go, and Luria’s hand sequences). Higher EXIT scores indicate greater impairment. Crucially, unlike in the probabilistic reversal learning task, no external reinforcement is given.

Probabilistic reversal learning fMRI task (Fig. 1)

Fig. 1.

Fig. 1

Open in a new tab

Probabilistic Learning Task. (A) On each trial, the participant chooses between two stimuli. Following the choice, feedback is displayed: green frame for reward (as shown), red frame for reward omission. A fixation screen follows. Scanning is synchronized with trial onset. (B) One stimulus initially has a high probability of reward and the other, a low probability of reward. This contingency is reversed unbeknownst to the participant every 25 trials, a total of 12 reversals. (C)The reinforcement learning model estimates expected reward and reward prediction error for each trial (shown for 50 trials), using the participant’s reinforcement history and choices. [Simplified learning rule provided for illustration; the model is described in full in Dombrovski et al., JAMA Psychiatry, 2013.]

On each of 300 trials, participants choose between two stimuli using button presses. In the first 25-trial block, one stimulus has a high probability of reward when chosen (varied between 80% and 87%), and the other, a low probability of reward (13%–20%). Thus, even when correctly choosing the high-reward probability stimulus, participants receive occasional misleading or ‘probabilistic’ negative feedback. After every 25 trials, this contingency is reversed, such that the high-reward probability stimulus is assigned a low reward probability and vice versa. There are a total of 12 reversals. On this task, one needs to first learn the identity of the best stimulus and then trade off ‘staying’ with the previously reinforced stimulus despite occasional misleading feedback and ‘switching’ when a true reversal occurs. The tendency to ‘stay’ too long after reversal while ignoring negative feedback leads to perseverative errors. Conversely, the tendency to ‘switch’ after a single misleading punishment results in lose-switches (also known as probabilistic switches), previously linked to depression (Murphy et al., 2003). Spontaneous or win-switch errors occur when the participant abandons a choice rewarded on a preceding trial.

Imaging

Thirty-two 3mm slices were acquired parallel to the AC-PC line using a reverse EPI pulse sequence (one of two 3T Siemens Trio magnets, T2*-weighted images depicting BOLD contrast [activity and connectivity indices did not vary between scanners: t(51)<0.73, p>.47]; TR=1670ms, TE=29ms, FOV=20cm, flip=75), yielding 954 whole-brain volumes per participant. Following slice-time correction, motion correction (AFNI 3dVolReg) and linear detrending to eliminate scanner drift, data were converted to percent-change, temporally smoothed, cross-registered to the Colin-27 Montreal Neurological Institute template (AIR’s 32-parameter non-linear warp), and spatially smoothed (6mm FWHM).

Estimation of prediction error signals using reinforcement learning

We estimated prediction error signals from participants’ reinforcement history and behavior on the probabilistic reversal learning task, using a modified Rescorla-Wagner reinforcement learning (RL) model (Dombrovski et al., 2010, Dombrovski et al., 2013)(Fig. 1).

Components of BOLD response tracking prediction errors

We estimated the empirical HRF in the control group using the independent switch vs. stay contrast in the vlPFC. It was then convolved with the positive prediction error estimates from the RL model for each subject. Voxelwise BOLD signal was regressed on these estimates using AFNI’s 3dDeconvolve. We used AFNI’s 3dTtest to map prediction error responses in the group of healthy controls. We applied a voxelwise threshold of p<.001 and a whole-brain cluster-size threshold (Cox, 1996, Forman et al., 1995). The resulting thresholded group map served as a functional mask for extracting mean responses to positive prediction error in each region for each participant.

White-matter hyperintensities (WMH)

T2-weighted FLAIR data (TR=9002ms;TEeffective=56ms;TI=2200ms; N[excitations]=1) were obtained on 27/47 participants, demographically and clinically similar to the rest of the group (p<.19). WMH volumes were estimated with an automated localization and segmentation method (Wu et al., 2006) and normalized to each participant’s overall brain volume (Aizenstein et al., 2011).

Replication of behavioral effects in a second sample

We used data from our earlier behavioral study of late-life depression (Dombrovski et al., 2010), which used identical recruitment/assessment procedures and an 80-trial probabilistic reversal learning task with a single reversal. To ensure independence, we excluded participants who subsequently took part in the imaging protocol, leaving 39 with major depression and 12 non-psychiatric controls. The groups were similar in race (white: controls: 12/12, depressed: 32/39) and education (controls: mean[SD]=15.6[2.7]years, depressed: 14.1[3.0]), but the depressed participants were older (controls: 65.7[5.3]years, depressed: 70.9[8.1]years, p=.045) and more likely to be female (controls: 2/12, depressed: 23/39). Thus, in our analysis, we co-varied for age and gender as well as history of suicide attempts, which was previously shown to predict task performance (Dombrovski et al., 2010).

Statistical analysis

We performed five separate families of tests: (1) magnitude of neural responses to unpredicted rewards—positive prediction errors—and effects of executive control and depression on these responses; (2) corticostriatal functional connectivity modulated by unexpected rewards for each participant and examined the effects of depression and executive control on connectivity indices; (3) effects of WMH on corticostriatal responses to unpredicted rewards; (4) effects of executive control and depression on behavior in the present study; (5) replication of behavioral findings using data from our earlier study.

1. Effects of cognitive control and depression on BOLD response tracking prediction errors

Applying positive prediction error group maps from non-psychiatric controls as a functional mask, we extracted BOLD responses to positive prediction error for each subject and region. Responses to positive prediction error were strongly inter-correlated across regions of the corticostriatothalamic circuit, revealing a single underlying factor (Dombrovski et al., 2013). Thus, to reduce dimensionality and control type I error, we examined summary activations across the entire network responsive to prediction error instead of activations in single regions. We followed up by examining separate clusters, to check whether the effects of executive dysfunction were spatially uniform. Our corticostriatothalamic network mask was defined in healthy controls, and to preempt concerns about circularity, we first examined the effect of executive control on prediction error signals in the independent group of depressed patients. After independently establishing this effect, we examined effects of depression and executive control in the combined group. We included age, gender, and education as covariates in our main analyses in order to de-noise the relationship between the independent (executive dysfunction, depression) and dependent variables (BOLD). To ensure that this relationship was not an artifact of partialing out common variance between independent variables and covariates (Miller and Chapman, 2001), we verified that the results were similar without covariates.

2. Striatocortical functional connectivity in the context of unexpected rewards

We aimed to test the effects of poor executive control and depression on the training of sustained prefrontal representations by trial-by-trial striatal signals (Pasupathy and Miller, 2005). We used the framework of generalized psychophysiological interaction analysis (McLaren et al., 2012) to estimate striatocortical functional connectivity specifically associated with unexpected rewards as opposed to any surprising outcomes. We defined the striatal seed as the region responsive to positive prediction errors in the entire sample within the Talairach Daemon striatum (pvoxelwise<.001). Using AFNI, we extracted the time course for the striatal seed and deconvolved the HRF from it. We then computed an interaction between the z-scored “neural” striatal time course and the positive and negative prediction (PE+ and PE−) error regressors. The single-subject model, implemented using AFNI’s 3dDeconvolve included the PE+*striatum and PE−*striatum interactions, as well as the following nuisance regressors: main effects of PE+ and PE−, striatal seed time course, motor action (left or right), trial stages (choice, feedback, wait), and the six motion regressors. We then computed a map of the difference between the regression coefficients for the PE+*striatum and PE−*striatum interactions, which was taken to the group analyses.

3. Effects of WMH

We examined effects of WMH burden on BOLD responses and functional connectivity. We also tested whether WMH burden explained the effects of depression and executive control, as the vascular depression hypothesis would predict.

4. Effects of cognitive control and depression on behavior

We used linear models with EXIT scores and depression group status as independent variables, age, gender and education as covariates, and the following behavioral indices as dependent variables: lose-switches, spontaneous switches, and perseverative errors. When necessary, we tested the robustness of results to deviations from normality using a generalized linear model.

5. Replication of behavioral effects

We tested the effect of executive control on reversal learning performance, including depression, gender, and history of suicide attempt as factors and age and education as covariates.

Results

Executive control and neural response to unpredicted rewards (Fig. 2)

Fig. 2.

Fig. 2

Open in a new tab

Corticostriatothalamic responses to positive prediction error and executive control. (a) Positive prediction error functional network masks were independently derived in controls. (b) In depressed participants, weak corticostratothalamic response to prediction errors was related to poor executive control as captured by the Executive Interview.

In depressed patients, poor executive control predicted blunted responses to unpredicted rewards in the network independently identified in controls, encompassing a frontoparietal circuit, a cinguloopercular circuit, dorsomedial prefrontal cortex, thalamus, and striatum (F[1,26]=13.4, p=.001, ηp2=.34, controlling for age, gender, and education). In the combined group of patients and controls, lower neural response to unexpected rewards in this network was predicted by poorer executive control (F[1,41]=15.7, p<.001, ηp2=.29) and depression (F[1,42]=17.4, p<.001, ηp2=.31), controlling for age (ηp2=.02), gender (ηp2=.08), and education (ηp2<.01). Contrary to the prediction of the vascular hypothesis, effects of executive control and depression were not shared, but rather were simply additive (model including depression [ηp2=.34], age, gender, education: R2=.35; model including executive control [ηp2=.29], depression [ηp2=.31], age, gender, education: R2=.57). Results were similar without covariates (Fig. 1). The disruptive effects of depression and poor executive control on responses to unpredicted rewards were not anatomically selective, with no significant differences between prefrontal, thalamo-striatal, paralimbic (precuneus), parietal, and temporal regions (p>.33, ηp2 ≤.11).

Effects on striatocortical connectivity modulated by unpredicted rewards (Fig. 3)

Fig. 3.

Fig. 3

Open in a new tab

Corticostriatal functional connectivity in the context of unexpected rewards. (a) Positive vs. negative prediction errors differentially enhanced functional connectivity between the striatum and left operculoinsular, right ventrolateral prefrontal cortex, bilateral superior temporal gyrus, and bilateral thalamus. (b) In a region-of-interest analysis, this cortico-thalamic network was less strongly modulated in depressed patients compared to controls. There were no effects of executive control. (c) An independent whole- brain contrast mapped the effect of depression to the left operculoinsular cortex. There were no significant effects of executive control.

In healthy controls, unpredicted rewards, contrasted with unpredicted punishments, triggered an increase in functional connectivity between striatum and left operculoinsular cortex, right vlPFC, bilateral superior temporal gyrus (STG), bilateral thalamus, and posterior cingulate cortex (Fig. 2a). In a ROI analysis, functional connectivity in this circuit was less strongly modulated by unpredicted rewards in patients with depression than in controls (F[1,42]=10.4, p=.002, ηp2=.20), but was not affected by poor executive control (p=.85, ηp2<.01; Fig. 2b). Since the ROIs were identified in controls, we also performed an unbiased whole-brain group contrast. Similarly, an effect of depression was seen for striatal connectivity with the left operculoinsular cortex (Fig. 2c), with no significant effects of poor executive control.

Effects of depression and executive control in a subsample with vascular conditions

To test the predictions of the vascular depression hypothesis in a group at highest risk for neurovascular injury, we limited the sample to those with clinically significant vascular disease. These 37 participants (25 depressed, 12 controls; 22 female; mean[SD] age: 68[7]) scored ≥2 on the Cumulative Illness Rating Scale adapted for Geriatrics, Vascular subscale [examples: one symptom of athersclerotic disease (angina, claudication, bruit, amaurosis fugax, absent pedal pulses) or need for daily antihypertensives or aortic aneurysm < 4 cm (Miller et al., 1992)]. The results remained qualitatively unchanged. Weaker corticostriatothalamic response to unexpected rewards was predicted by poorer executive control (F[1,30]=20.6, p<.001, ηp2=.41) and depression (F[1,30]=14.3, p<.001, ηp2=.32; age: ηp2=.04, gender: ηp2=.08, education: ηp2=.08). Further, striato-cortical functional connectivity was weaker in patients with depression than in controls (F[1,30]=5.8, p=.022, ηp2=.16), but was unaffected by poor executive control (p=.61, ηp2<.01).

WMH

WMH burden predicted blunted corticostriatal responses to unpredicted rewards (F[1,22]=8.7, p=.007, ηp2=.28), after controlling for predictors reported above. WMH burden failed to explain the effects of depression (effect of depression controlling for WMH: ηp2=.51, without WMH: ηp2=.40) or poor executive control (controlling for WMH: ηp2=.35, without WMH: ηp2=.40). WMH burden explained no significant variance in striatocortical connectivity (p>.29, ηp2<.05). Finally, in the sub-sample of participants with vascular disease, white-matter hyperintensity (WMH) burden predicted a weaker neuro-hemodynamic response to unexpected rewards (F[1,19]=5.5, p=.030, ηp2=.23), but did not explain the effects of executive control (F[1,19]=10.0, p=.005, ηp2=.35) or depression (F[1,19]=11.0, p=.004, ηp2=.37).

WMH location

We verified that WMH located in tracts terminating in prefrontal and cingulate cortex (cingulum bundle, superior longitudinal fasciculus, uncinate fasciculus, and anterior thalamic radiation) exerted similar effects on corticostriatothalamic prediction error signals as the total WMH burden. WMH in each of these tracts had disruptive effects on prediction error signals, but did not account for effects of depression and executive dysfunction, which remained undiminished (data available upon request).

Behavioral signatures of poor executive control and depression

Individuals with poor executive control failed to persist with rewarded actions. Meanwhile, participants with depression over-reacted to single misleading punishments. Specifically, poor executive control predicted spontaneous switches (EXIT: F[1,41]=10.1, p=.003, ηp2=.20; depression: F[1,41]=0.5, p=.50, ηp2=.01; controlling for age, gender, and education). Meanwhile, depression predicted lose-switches (depression: F[1,41]=6.5, p=.014, ηp2=.14; EXIT: F[1,41]=1.8, p=.19, ηp2=.04; controlling for age, gender, and education). Neither executive control nor depression affected perseverative errors (p>.31, ηp2<.03). A repeated-measures analysis confirmed a double dissociation of these effects: poor executive control was selectively associated with spontaneous switches (EXIT*error type: F[1,41]=7.9, p=.007, ηp2=.16) and depression was selectively associated with lose-switches (depression*error type: F[1,41]=4.2, p=.048, ηp2=.09, both controlling for age, gender, and education). Only executive control (F[1,40]=4.4, p=.043, ηp2=.10) but not depression (F[1,41]=1.8, p=.18, ηp2=.04) explained significant variance in the overall number of correct responses. Finally, WMH burden did not explain any additional variance in behavioral indices (p>.24, ηp2 <.06).

Replication of behavioral effects

In the replication sample, participants with poor cognitive control similarly failed to persist with rewarded actions on a single-reversal task. Poor executive control predicted spontaneous switches during acquisition (F[1,40]=6.5, p=.014, ηp2=.14; controlling for the effects of age, gender, education, and history of depression and attempted suicide), but not lose-switches or perseverative errors (p>.39, ηp2<.02). There were not enough spontaneous switches in the post-reversal phase for analysis. As we have reported, depression was not reliably related to lose-switches in that study (Dombrovski et al., 2010). As in the fMRI study, poor executive control selectively predicted the encoding of rewards (spontaneous switches) rather than sensitivity to single misleading punishments (lose-switches; EXIT*error type: F[1,47]=6.2, p=.016, ηp2=.12).

Sensitivity analyses

Comorbid conditions

Depressed participants with lifetime history of substance use disorders tended to display blunted corticostriatothalamic positive prediction error responses (F[1,20]=4.1, p=.058, ηp2=.17). This effect was additive and did not modify that of executive dysfunction (F[1,20]=7.4, p=.013, ηp2=.27). Lifetime history of anxiety disorders had no significant impact (F[1,20]=0.2, p=.69; all controlling for depression, age, gender, and education).

Burden of physical illness

While depressed participants suffered from a higher burden of physical illness (Table 1), it had no effect on corticostriatothalamic positive prediction error responses (F[1,39]=0.3, p=.57, ηp2<.01), while the effects of executive control (F[1,39]=21.9, p<.001, ηp2=.36) and depression remained unchanged (F[1,39]=22.0, p<.001, ηp2=.36).

Age at first episode of depression

One might argue that prefrontal disconnection associated with poor executive control better accounts for cases of late-onset depression. However, later age of onset was not associated with poorer executive control in our sample (r=.05, p=.78) and did not modify the effects of executive control on prediction error signals (EXIT: F[1,24]=13.0, p=.001, ηp2=.35; age at first episode: p=.81, ηp2<.01) or on spontaneous switch errors (EXIT: F[1,24]=5.1, p=.034, ηp2=.17; age at first episode: p=.72, ηp2<.01). At the same time, patients with earlier onset of depression demonstrated a greater sensitivity to single misleading punishments indexed by lose-switches, while there were still no significant effects of executive control (age at first episode: F[1,24]=5.0, p=.035, ηp2=.17; EXIT: p=.27, ηp2=.05).

Medication exposure

The relationship between executive dysfunction and corticostriatothalamic positive prediction error responses was unaffected by exposure to opioids, sedatives or anticholinergics (EXIT: F[1,23]=12.1, p=.002, ηp2=.35; exposure to any medication class: p>.58, ηp2<.02). The same was true for the cumulative strength of antidepressant exposure measured by the Antidepressant Treatment History Form (EXIT: F[1,17]=9.0, p=.008, ηp2=.35; ATHF: p=.73, ηp2<.01).

Potential experimental and analytic confounds

It was important to verify that the relationship between executive control and corticostriatothalamic prediction error signals was not driven by participants who were not on task or whose behavior could not be fit with the reinforcement learning model. Neither the median response time nor its variation coefficient had an impact on corticostriatothalamic positive prediction error signals (p>.12, ηp2<.06) and did not diminish the effect of executive control. Participants with poor executive control had significantly poorer fits of the reinforcement learning model (F[1,40]=7.8, p=.008, ηp2=.16). However, controlling for model fits (F[1,39]=1.0, p=.33, ηp2=.03) did not diminish the effect of executive control (F[1,39]=13.7, p=.001, ηp2=.26).

Distribution of spontaneous switch errors

The distribution of spontaneous switch errors was zero-inflated. However, an analysis using a generalized linear model with a negative binomial log-link confirmed that the relationship between poor executive control and spontaneous switch errors was robust to this violation of normality (EXIT: Wald χ2=8.1, p=.005). Distributions of other dependent variables did not significantly deviate from the Gaussian.

Conclusions

We tested a prediction of the vascular depression hypothesis that prefrontal/cingulate disconnection from subcortical nuclei would account for neural correlates of depression and poor executive control during reward-guided behavior. Overall, this prediction was not confirmed. First, poor executive control was associated with disrupted corticostriatothalamic responses to unpredicted rewards independently of the effects of depression. Second, depression, but not poor executive control, was associated with disrupted striatocortical differential connectivity to unpredicted rewards vs. unpredicted punishments. Third, while white-matter hyperintensities predicted blunted corticostriatothalamic responses, they failed to account for the effects of depression or executive control. Fourth, poor executive control and depression had distinct behavioral signatures. Poor executive control predicted spontaneous switches, presumably reflecting a basic disruption in contingency encoding. Depression predicted oversensitivity to misleading punishments, presumably because preceding rewards were not robustly encoded. Below, we briefly explicate the neurobiological framework for interpreting the results. We then discuss implications for the vascular depression hypothesis and alternative models of depression.

Corticostriatothalamic mechanisms of reward learning and the vascular depression hypothesis

Reward prediction errors were theoretically predicted (Montague et al., 1996) and empirically shown (Schultz et al., 1997) to originate in the mesostriatal dopaminergic pathway. Prediction errors are thought to influence behavior through synaptic modification of corticostriatothalamic circuits. Prediction error signals during Pavlovian learning were first described in the human ventral striatum (O’Doherty et al., 2003), which forms reentrant loops with the medial orbitofrontal and anterior cingulate cortices (Middleton and Strick, 2000) classically implicated in reward-guided behavior. During the learning of action-reward (instrumental) contingencies, prediction error signals are found in the dorsal striatum connected to associative prefrontal cortices (O’Doherty et al., 2004).

In this study, we used the strength of instrumental prediction error signals in the striatum, thalamus, cinguloopercular and frontoparietal networks as an indicator of their functional integrity. Based on what we know about the corticostriatothalamic reward systems from lesion, electrophysiological, and imaging studies, our functional connectivity and behavioral findings suggest that the two syndromes are paralleled by disruptions at different levels of corticostriatothalamic circuits. Alterations in reward signals associated with executive dysfunction, not surprisingly, fit the pattern of prefrontal dysfunction and, possibly, disconnection. The behavior of individuals with impaired executive control bears a close resemblance to that of patients with bilateral prefrontal lesions (Hornak et al., 2004) and, interestingly, to that of primates with lesions to white-matter prefrontal connections (Rudebeck et al., 2013). Our findings parallel those of a study showing that fluid intelligence (a proxy for executive control) was correlated with striatal prediction error signals independently of dopamine synthesis capacity (Schlagenhauf et al., 2012). Both executive control and fluid intelligence depend on the functional integrity and intact outputs of the frontoparietal network (Cole et al., 2012). Meanwhile, in depression, it is the ascending transmission of mesostriatal reward prediction errors that appears to be affected. This is indicated by diminished functional striatocortical connectivity in response to positive vs. negative prediction errors. These data are consistent with a disruption of the mesostriatal output through the pallidum and thalamus to the associative prefrontal neocortex. This interpretation is in agreement with electrophysiological evidence that, as contingencies are encoded, striatal responses to reinforcement emerge and asymptote before the prefrontal responses, which mediate behavioral improvement (Pasupathy and Miller, 2005). Thus, reduced functional striatocortical connectivity in depression presumably reflects disrupted training of tonic prefrontal representations by phasic striatal signals. Supporting this idea, the oversensitivity of depressed patients to misleading punishments resembles the behavior of individuals with a genetically reduced expression of striatal D2 receptors, which conceivably attenuates mesostriatal prediction error signals (Jocham et al., 2009). The notion that ascending dopaminergic signals are deficient in depression is further supported by the recent finding of an increased burden of brainstem Lewy bodies and tangles and lower density of ventral tegmental area dopaminergic neurons in older people with depressive symptoms (Wilson et al., 2013).

The high co-occurrence of depression and poor cognitive control nevertheless demands an alternative explanation. We would argue that impaired executive control prevents people from finding and exploiting reward contingencies in their environment. This inability to learn reward contingencies is depressogenic: a potentially controllable environment becomes uncontrollable, akin to that found in learned helplessness experiments (Seligman and Maier, 1967). Support for this account is provided by evidence that cognitive decline rather than lesion location predicts depressive symptoms after a stroke (Nys et al., 2006) and that cardiovascular risk factors predict depressive symptoms only if executive control is impaired (Mast et al., 2004).

Limitations

The cross-sectional, case-control design of our study precludes causal inferences. It also raises questions about possible confounders, although many were ruled out in our sensitivity analyses. The fact that many patients were partially recovered from depression can be seen as a limitation. Another limitation is the availability of FLAIR data on only a subsample (58%), albeit large enough to detect a strong relationship between the WMH burden and BOLD response. The EXIT is a broad screening measure of executive control, and it is possible that its specific domains would show a stronger overlap with effects of depression. However, a recent meta-analysis demonstrated a broad, non-selective impairment across executive function domains in depression (Snyder, 2013). Further, our results are strengthened by absence of external reinforcement on the EXIT. Finally, a modest sample size warrants caution.

In summary, in a sample of older adults, depression and impaired executive control were linked to dissociable disruptions in the corticostriatothalamic encoding of reinforcement. Neither of these disruptions was explained by WMH. We conclude that ascending striatocortical reward signals appear to be disrupted in late-life depression, while deficits associated with poor executive control more closely resemble effects of lateral prefrontal lesions or prefrontal disconnection.

Acknowledgments

The authors would like to thank Amanda Collier, B.S., for her work on data collection and processing, Jan Kalkus, B.S., for help with data processing and manuscript preparation, and Swathi Gujral, B.S., Natalie Truty, B.S., and Cori Shollenberger, B.S., for their hard work on recruitment and assessments. Meryl A. Butters, Ph.D., supervised neuropsychological assessments and shared data collected in her study. Ariel G. Gildengers, M.D., also shared data collected in his study. They would also like to thank George Alexopoulos, MD, for his thoughtful comments on the manuscript.

Grant support: NIMH (K23MH086620, K23MH070471, R01MH085651, R01MH080240, R01MH084921) and the American Foundation for Suicide Prevention.

Dr. Reynolds reports receiving pharmaceutical support for NIH-sponsored research studies from Bristol-Myers Squibb, Forest, Pfizer, and Lilly; receiving grants from the National Institute of Mental Health, National Institute on Aging, National Center for Minority Health Disparities, National Heart Lung and Blood Institute, Center for Medicare and Medicaid Services (CMS), Patient Centered Outcomes Research Institute (PCORI), the Commonwealth of Pennsylvania, the John A Hartford Foundation, National Palliative Care Research Center (NPCRC), Clinical and Translational Science Institute (CTSI), and the American Foundation for Suicide Prevention; and serving on the American Association for Geriatric Psychiatry editorial review board. He has received an honorarium as a speaker from MedScape/WEB MD. He is the co-inventor (Licensed Intellectual Property) of Psychometric analysis of the Pittsburgh Sleep Quality Index (PSQI) PRO10050447 (PI: Buysse) and is supported by the National Institutes of Health through Grant Numbers P60 MD000207; P30 MH090333; UL1RR024153, UL1TR000005; and the UPMC Endowment in Geriatric Psychiatry.

Footnotes

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helinski Declaration of 1975, revised in 2008.

None of the other authors have any conflicts of interest to report.

References

  1. Aizenstein HJ, Andreescu C, Edelman KL, Cochran JL, Price J, Butters MA, Karp J, Patel M, Reynolds CF., 3rd Fmri correlates of white matter hyperintensities in late-life depression. American Journal of Psychiatry. 2011;168:1075–82. doi: 10.1176/appi.ajp.2011.10060853. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Alexopoulos G, Kiosses DN, Choi SJ, Murphy CF, Lim KO. Frontal white matter microstructure and treatment response in late-life depression: A preliminary study. American Journal of Psychiatry. 2002;159:1929–1932. doi: 10.1176/appi.ajp.159.11.1929. [DOI] [PubMed] [Google Scholar]
  3. Alexopoulos GS, Hoptman MJ, Kanellopoulos D, Murphy CF, Lim KO, Gunning FM. Functional connectivity in the cognitive control network and the default mode network in late-life depression. Journal of Affective Disorders. 2012;139:56–65. doi: 10.1016/j.jad.2011.12.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Alexopoulos GS, Meyers BS, Young RC, Campbell S, Silbersweig D, Charlson M. ‘Vascular depression’ hypothesis. Archives of General Psychiatry. 1997;54:915–22. doi: 10.1001/archpsyc.1997.01830220033006. [DOI] [PubMed] [Google Scholar]
  5. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4. American Psychiatric Association; Washington, DC: 2000. Dsm-iv. [Google Scholar]
  6. Baldwin R, Jeffries S, Jackson A, Sutcliffe C, Thacker N, Scott M, Burns A. Treatment response in late-onset depression: Relationship to neuropsychological, neuroradiological and vascular risk factors. Psychological Medicine. 2004;34:125–36. doi: 10.1017/s0033291703008870. [DOI] [PubMed] [Google Scholar]
  7. Barch DM, D’Angelo G, Pieper C, Wilkins CH, Welsh-Bohmer K, Taylor W, Garcia KS, Gersing K, Doraiswamy PM, Sheline YI. Cognitive improvement following treatment in late-life depression: Relationship to vascular risk and age of onset. American Journal of Geriatric Psychiatry. 2012 doi: 10.1097/JGP.0b013e318246b6cb. Publish Ahead of Print. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bush RR, Mosteller F. A mathematical model for simple learning. Psychological Review. 1951;58:313–323. doi: 10.1037/h0054388. [DOI] [PubMed] [Google Scholar]
  9. Chase HW, Kumar P, Eickhoff SB, Dombrovski AY. Reinforcement learning models and their neural correlates: An activation likelihood estimation meta-analysis. Cognitive, Affective, & Behavioral Neuroscience. 2015:1–25. doi: 10.3758/s13415-015-0338-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cole MW, Yarkoni T, Repovs G, Anticevic A, Braver TS. Global connectivity of prefrontal cortex predicts cognitive control and intelligence. Journal of Neuroscience. 2012;32:8988–99. doi: 10.1523/JNEUROSCI.0536-12.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Cox RW. Afni: Software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research. 1996;29:162–73. doi: 10.1006/cbmr.1996.0014. [DOI] [PubMed] [Google Scholar]
  12. Dombrovski AY, Clark L, Siegle GJ, Butters MA, Ichikawa N, Sahakian BJ, Szanto K. Reward/punishment reversal learning in older suicide attempters. American Journal of Psychiatry. 2010;167:699–707. doi: 10.1176/appi.ajp.2009.09030407. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Dombrovski AY, Szanto K, Clark L, Reynolds CF, Siegle GJ. Reward signals, attempted suicide, and impulsivity in late-life depression. JAMA Psychiatry. 2013 doi: 10.1001/jamapsychiatry.2013.75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Eshel N, Roiser JP. Reward and punishment processing in depression. Biological Psychiatry. 2010;68:118–124. doi: 10.1016/j.biopsych.2010.01.027. [DOI] [PubMed] [Google Scholar]
  15. First MSR, Gibbon M, Williams JBW. Structured clinical interview for DSM-IV Axis I Disorders - Patient edition (SCID-I/P) 1995. Version 2.0 ed. [Google Scholar]
  16. Forman SD, Cohen JD, Fitzgerald M, Eddy WF, Mintun MA, Noll DC. Improved assessment of significant activation in functional magnetic resonance imaging (fmri): Use of a cluster-size threshold. Magnetic Resonance in Medicine. 1995;33:636–47. doi: 10.1002/mrm.1910330508. [DOI] [PubMed] [Google Scholar]
  17. Glimcher PW. Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis. Proceedings of the National Academy of Sciences of the United States of America. 2011;108(Suppl 3):15647–54. doi: 10.1073/pnas.1014269108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Gradin VB, Kumar P, Waiter G, Ahearn T, Stickle C, Milders M, Reid I, Hall J, Steele JD. Expected value and prediction error abnormalities in depression and schizophrenia. Brain. 2011;134:1751–64. doi: 10.1093/brain/awr059. [DOI] [PubMed] [Google Scholar]
  19. Hamilton M. A rating scale for depression. Journal of Neurology, Neurosurgery & Psychiatry. 1960;23:56–62. doi: 10.1136/jnnp.23.1.56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Histed MH, Pasupathy A, Miller EK. Learning substrates in the primate prefrontal cortex and striatum: Sustained activity related to successful actions. Neuron. 2009;63:244–53. doi: 10.1016/j.neuron.2009.06.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hornak J, O’Doherty J, Bramham J, Rolls ET, Morris RG, Bullock PR, Polkey CE. Reward-related reversal learning after surgical excisions in orbito-frontal or dorsolateral prefrontal cortex in humans. Journal of Cognitive Neuroscience. 2004;16:463–78. doi: 10.1162/089892904322926791. [DOI] [PubMed] [Google Scholar]
  22. Jocham G, Klein TA, Neumann J, von Cramon DY, Reuter M, Ullsperger M. Dopamine drd2 polymorphism alters reversal learning and associated neural activity. Journal of Neuroscience. 2009;29:3695–704. doi: 10.1523/JNEUROSCI.5195-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kumar P, Waiter G, Ahearn T, Milders M, Reid I, Steele JD. Abnormal temporal difference reward-learning signals in major depression. Brain. 2008;131:2084–93. doi: 10.1093/brain/awn136. [DOI] [PubMed] [Google Scholar]
  24. Mast BT, Yochim B, MacNeill SE, Lichtenberg PA. Risk factors for geriatric depression: The importance of executive functioning within the vascular depression hypothesis. Journals of Gerontology Series A: Biological Sciences and Medical Sciences. 2004;59:1290–4. doi: 10.1093/gerona/59.12.1290. [DOI] [PubMed] [Google Scholar]
  25. Mattis S. Psychological assessment resources. Odessa, FL: 1988. Dementia rating scale (drs): Professional manual. [Google Scholar]
  26. McLaren DG, Ries ML, Xu G, Johnson SC. A generalized form of context-dependent psychophysiological interactions (gppi): A comparison to standard approaches. Neuroimage. 2012;61:1277–86. doi: 10.1016/j.neuroimage.2012.03.068. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Middleton FA, Strick PL. Basal ganglia and cerebellar loops: Motor and cognitive circuits. Brain Research: Brain Research Review. 2000;31:236–50. doi: 10.1016/s0165-0173(99)00040-5. [DOI] [PubMed] [Google Scholar]
  28. Miller GA, Chapman JP. Misunderstanding analysis of covariance. Journal of abnormal psychology. 2001;110:40. doi: 10.1037//0021-843x.110.1.40. [DOI] [PubMed] [Google Scholar]
  29. Miller MD, Paradis CF, Houck PR, Mazumdar S, Stack JA, Rifai AH, Mulsant B, Reynolds CF., 3rd Rating chronic medical illness burden in geropsychiatric practice and research: Application of the cumulative illness rating scale. Psychiatry Research. 1992;41:237–48. doi: 10.1016/0165-1781(92)90005-n. [DOI] [PubMed] [Google Scholar]
  30. Montague PR, Dayan P, Sejnowski TJ. A framework for mesencephalic dopamine systems based on predictive hebbian learning. Journal of Neuroscience. 1996;16:1936–47. doi: 10.1523/JNEUROSCI.16-05-01936.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Murphy FC, Michael A, Robbins TW, Sahakian BJ. Neuropsychological impairment in patients with major depressive disorder: The effects of feedback on task performance. Psychological Medicine. 2003;33:455–67. doi: 10.1017/s0033291702007018. [DOI] [PubMed] [Google Scholar]
  32. Nys GM, van Zandvoort MJ, van der Worp HB, de Haan EH, de Kort PL, Jansen BP, Kappelle LJ. Early cognitive impairment predicts long-term depressive symptoms and quality of life after stroke. Journal of Neurological Science. 2006;247:149–56. doi: 10.1016/j.jns.2006.04.005. [DOI] [PubMed] [Google Scholar]
  33. O’Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science. 2004;304:452–4. doi: 10.1126/science.1094285. [DOI] [PubMed] [Google Scholar]
  34. O’Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ. Temporal difference models and reward-related learning in the human brain. Neuron. 2003;38:329–37. doi: 10.1016/s0896-6273(03)00169-7. [DOI] [PubMed] [Google Scholar]
  35. Pasupathy A, Miller EK. Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature. 2005;433:873–6. doi: 10.1038/nature03287. [DOI] [PubMed] [Google Scholar]
  36. Rescorla RA, Wagner AR. A theory of pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF, editors. Classical conditioning ii. Appleton-Century-Crofts; New York, London: 1972. pp. 64–99. [Google Scholar]
  37. Robbins TW. Shifting and stopping: Fronto-striatal substrates, neurochemical modulation and clinical implications. Philosophical Transactions of the Royal Society B: Biological Sciences. 2007;362:917–32. doi: 10.1098/rstb.2007.2097. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Royall DR, Mahurin RK, Gray KF. Bedside assessment of executive cognitive impairment: The executive interview. Journal of American Geriatrics Society. 1992;40:1221–6. doi: 10.1111/j.1532-5415.1992.tb03646.x. [DOI] [PubMed] [Google Scholar]
  39. Rudebeck PH, Saunders RC, Prescott AT, Chau LS, Murray EA. Prefrontal mechanisms of behavioral flexibility, emotion regulation and value updating. Nature Neuroscience. 2013;16:1140–1145. doi: 10.1038/nn.3440. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Sackeim HA. The definition and meaning of treatment-resistant depression. Journal of Clinical Psychiatry. 2001;62(Suppl 16):10–7. [PubMed] [Google Scholar]
  41. Schlagenhauf F, Rapp MA, Huys QJ, Beck A, Wustenberg T, Deserno L, Buchholz HG, Kalbitzer J, Buchert R, Bauer M, Kienast T, Cumming P, Plotkin M, Kumakura Y, Grace AA, Dolan RJ, Heinz A. Ventral striatal prediction error signaling is associated with dopamine synthesis capacity and fluid intelligence. Human Brain Mapping. 2012 doi: 10.1002/hbm.22000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. doi: 10.1126/science.275.5306.1593. [DOI] [PubMed] [Google Scholar]
  43. Seligman ME, Maier SF. Failure to escape traumatic shock. Journal of Experimental Psychology. 1967;74:1–9. doi: 10.1037/h0024514. [DOI] [PubMed] [Google Scholar]
  44. Skinner BF. The behavior of organisms; an experimental analysis. D. Appleton-Century company; New York, London: 1938. [Google Scholar]
  45. Sneed JR, Roose SP, Keilp JG, Krishnan KR, Alexopoulos GS, Sackeim HA. Response inhibition predicts poor antidepressant treatment response in very old depressed patients. American Journal of Geriatric Psychiatry. 2007;15:553–63. doi: 10.1097/JGP.0b013e3180302513. [DOI] [PubMed] [Google Scholar]
  46. Snyder HR. Major depressive disorder is associated with broad impairments on neuropsychological measures of executive function: A meta-analysis and review. Psychological Bulletin. 2013;139:81–132. doi: 10.1037/a0028727. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Sutton RS, Barto AG. Reinforcement learning: An introduction. MIT Press; Cambridge, MA: 1998. [Google Scholar]
  48. Taylor WD, Aizenstein HJ, Alexopoulos GS. The vascular depression hypothesis: Mechanisms linking vascular disease with depression. Molecular Psychiatry. 2013 doi: 10.1038/mp.2013.20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Wilson RS, Nag S, Boyle PA, et al. Brainstem aminergic nuclei and late-life depressive symptoms. JAMA Psychiatry. 2013;70:1320–1328. doi: 10.1001/jamapsychiatry.2013.2224. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Wu M, Rosano C, Butters M, Whyte E, Nable M, Crooks R, Meltzer CC, Reynolds CF, 3rd, Aizenstein HJ. A fully automated method for quantifying and localizing white matter hyperintensities on mr images. Psychiatry Research. 2006;148:133–42. doi: 10.1016/j.pscychresns.2006.09.003. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES