Levodopa Enhances Reward Learning but Impairs Reversal Learning in Parkinson's Disease Patients

A commentary on Differential influence of levodopa on reward-based learning in Parkinson's disease. by Graef, S., Biele, G., Krugel, L. K., Marzinzik, F., Wahl, M., Wotka, J., Klostermann, F., and Heekeren, H. R. (2010). Front. Hum. Neurosci. 4:169. doi: 10.3389/fnhum.2010.00169

Parkinson's disease (PD) and associated dopaminergic medications, such as levodopa and dopamine agonists, have differential effects on cognition. Dopamine medications might either enhance or impair cognition in PD patients depending on type of cognitive task and medication used. In a recent neuropsychological study, Graef and colleagues have tested the effects of PD and levodopa monotherapy on feedback and reversal learning. In feedback learning, subjects learn to select stimuli, based on either rewarding (positive) or punishing (negative) feedback to their responses. Thus, reward learning involves learning to select responses that lead to reward, while punishment learning involves learning to avoid responses that lead to negative outcome. In reversal learning tasks, subjects initially learn to associate different stimuli with different responses, and subsequently learn to associate the same stimuli with the opposite responses (i.e., reversal). Graef et al. have found that levodopa enhances reward learning but impairs reversal learning in PD patients. This finding is consistent with earlier reports that dopamine medications enhances reward learning (Frank et al., 2004; Moustafa et al., 2008a; Bodi et al., 2009; Palminteri et al., 2009) but impairs reversal learning (Swainson et al., 2000; Cools et al., 2001) in PD patients.

One notable aspect of the current study is Graef et al. have tested PD patients on levodopa only. This is contrasted from prior studies that have recruited PD patients on multiple therapies including levodopa and dopamine agonists (Cools et al., 2001; Frank et al., 2004; Moustafa et al., 2008b; Bodi et al., 2009), thus confounding dissociable effects of levodopa vs. dopamine agonists on cognition. Furthermore, the Graef et al. findings are in agreement with other studies showing that the administration of levodopa enhances feedback learning in rats (Pavlis et al., 2006), healthy subjects (Knecht et al., 2004; Pessiglione et al., 2006; Floel et al., 2008; Pleger et al., 2009), stroke patients (Scheidtmann et al., 2001; Rosser et al., 2008), and PD patients (Beeler et al., 2010; de Vries et al., 2010).

Interestingly, Graef et al. have also found that the administration of different doses of levodopa to PD patients does not affect reward learning, that is, levodopa has no dose effect on reward learning. This is in line with the fact that levodopa is a dopamine precursor, taken up by dopamine neurons and converted into dopamine, and thus largely produced in natural conditions. Accordingly, an increase of doses of levodopa may not significantly alter presynaptic dopamine levels in the basal ganglia, and thus may not impact cognition, as reported in the Graef study. This is, however, contrasted with the effects of different doses of dopamine agonists on cognition. For example, studies have found that in healthy subjects, a low-dose (1.25 mg) of the dopamine agonist bromocriptine has no effect or impairs working memory (Gibbs and D'Esposito, 2005) while a high-dose (2.5 mg) of bromocriptine enhances working memory (Luciana et al., 1992; Luciana and Collins, 1997; McDowell, et al., 1998). It is possible that a larger dose of dopamine agonists further increase postsynaptic dopamine in the basal ganglia and prefrontal cortex (Moustafa and Gluck, 2011), and thus explain how different doses of dopamine agonists are associated with dissociable cognitive function.

Another notable finding in the Graef et al. study is in addition to reward learning, levodopa also ameliorates depression and motor performance (as measured by the Beck Depression Inventory and UPDRS scales) but has no effect on Mini-Mental State Exam (MMSE). This is in line with findings that depression and motor performance are related to basal ganglia dopamine function (Walter et al., 2010), while MMSE scores are associated with medial temporal lobe function (Ikeda et al., 2008; Ding et al., 2009), which is thought to be intact in PD patients.

One unresolved issue in the literature is the confounding results regarding the effects of dopamine medications on feedback learning and reversal learning. With regards to feedback learning, unlike Graef et al. results, other studies have found that the administration of dopamine medications to PD patients impairs or has no effect on feedback learning (Czernecki et al., 2002; Shohamy et al., 2006; Jahanshahi et al., 2009). Similarly, studies have shown that the administration of dopamine medications to PD patients either enhances or has no effect on reversal learning (Czernecki et al., 2002; Rutledge et al., 2009), unlike what Graef et al. have found. It is important to note that unlike the Graef study, other studies have tested PD patients on multiple therapies, including levodopa and dopamine agonists, which might explain the different results in these studies. Along the same lines, Feigin et al. (2003) have found that the administration of levodopa slightly (but not significantly) impairs sequence learning in PD patients. Sequence learning is a feedback learning paradigm in which subjects learn to make a “sequence” of motor responses, based on corrective feedback. Feigin et al. have tested PD patients who were on both levodopa and dopamine agonists, but were only on levodopa during the time of testing. Differences between the Graef et al. and Feigin et al. results could be due to long-term effects of the intake of dopamine agonists on cognition in the Feigin et al. study.

How would dopamine replacement therapies enhance reward learning but impair reversal learning? There are two theories that explain Graef et al. results. Frank (2005) argue that the basal ganglia direct pathway (along with dopamine D1 receptors) is required for reward learning, while the indirect pathways (along with dopamine D2 receptors) is essential for punishment learning, which involves learning to avoid responses that lead to negative feedback. Frank further argues that dopamine medications increase dopamine levels in the basal ganglia and enhance reward learning via D1 receptors, but impair punishment learning via D2 receptors. According to Frank, following reversal of reward contingencies in the reversal learning task, subjects are more likely to receive more negative feedback, and thus an inability to learn from negative feedback will lead to reversal learning impairment in medicated PD patients. An alternative theory is proposed by Cools et al. (2001), which argue that while the ventral striatum is intact in PD patients, dorsal striatum is dysfunctional. Cools et al. further argue that reward learning is mediated by the dorsal striatum while punishment learning by the ventral striatum. According to Cools, dopamine medications ameliorate the function of the dorsal striatum, but overdose the ventral striatum and orbitofrontal cortex loop, and thus enhance reward learning but impair reversal learning. Moustafa and Gluck (under review) have recently built a computational model showing how overdosing cortical areas impair reversal learning in medicated PD patients, as reported by Graef et al.

Overall, Graef and colleagues have shown that the administration of levodopa monotherapy has both enhancing and deleterious effects on cognition in PD patients. Future research should investigate the dissociable effects of levodopa vs. dopamine agonists on cognition in PD patients. To my knowledge, studying differential effects of dopamine replacement therapies in PD patients has been only conducted using working memory tasks (Brusa et al., 2003; Costa et al., 2003), but not with reward and reversal learning paradigms.