Abstract
Risk evaluation is a critical component of decision making. Risk tolerance is relevant in both daily decisions and pathological disorders such as attention-deficit hyperactivity disorder (ADHD), where impulsivity is a cardinal symptom. Methylphenidate, a commonly prescribed drug in ADHD, improves attention but has mixed reports on risk-based decision making. Using a double-blinded placebo protocol, we studied the risk attitudes of ADHD patients and age-matched healthy volunteers while performing the 2-step sequential learning task and examined the effect of methylphenidate on their choices. We then applied a novel computational analysis using the hierarchical drift–diffusion model to extract parameters such as threshold (‘a’—amount of evidence accumulated before making a decision), drift rate (‘v’—information processing speed) and response bias (‘z’ apriori bias towards a specific choice) focusing specifically on risky choice preference. Critically, we show that ADHD patients on placebo have an apriori bias towards risky choices compared to controls. Furthermore, methylphenidate enhanced preference towards risky choices (higher apriori bias) in both groups but had a significantly greater effect in the patient population independent of clinical scores. Thus, methylphenidate appears to shift tolerance towards risky uncertain choices possibly mediated by prefrontal dopaminergic and noradrenergic modulation. We emphasise the utility of computational models in detecting underlying processes. Our findings have implications for subtle yet differential effects of methylphenidate on ADHD compared to healthy population.
Subject terms: Predictive markers, Human behaviour, ADHD
Introduction
Evaluation of risk is one of the core constructs critical to decision making. Risk-taking can involve the evaluation of explicit risk where the likelihoods of benefit (reward) or harm (loss) are known; or ambiguity, where the likelihoods are unknown1. Attention-deficit hyperactivity disorder (ADHD) is a neurodevelopmental disorder characterised by inattention, impulsivity, and decision-making deficits2. Higher risk-taking behaviours are a common feature of ADHD, which include substance abuse3 and gambling behaviours4.
A meta-regression analysis comparing multiple studies on risk-taking behaviour including multiple implicit (Iowa Gambling Task (IGT), Balloon Analogue risk task (BART)) and explicit forms (Game of dice, Card Playing Task, Cambridge Gambling Task (CGT) & Probabilistic discounting task)) showed greater risk-taking in ADHD5. Methylphenidate (MPH), a commonly prescribed medication has a differential influence on ADHD patients relative to healthy controls based on the sub-type of impulsivity (Table 1). These mixed differential effects call out for novel approaches to understand mechanistic differences and identify behavioural markers associated with the pathophysiology.
Table 1.
Dimension | Effect on controls | Effect on ADHD |
---|---|---|
Response Inhibition | Enhanced correct Go responses39,40 associated with fronto-parietal activity40 in a Go-Nogo task, Improved response inhibition in a modified stop signal task41 | Improved response inhibition on stop signal task to that of non-medicated controls13,42 |
Delayed reward | Decreased delay discounting23 | Decreased delayed discounting in ADHD children24 |
Risk-taking | No difference in number of gambles or response time in controls43. Increase in gambles despite an increase in loss38 | Mixed results in betting behaviour with both decreased5 and no effect reported5,44. Improved IGT score in inattention sub-type44 |
Reflection Impulsivity | No effect on Information Sampling Task in both controls and ADHD5. In controls, MPH differentially modulated the performance, depending on baseline impulsivity45 | |
Waiting Impulsivity | Increased premature responding in a 4 choice serial time task in controls45 |
Recently, computational psychiatry has sought to explain how these behaviours are underpinned by altered computational processes during decision making6–8. Drift–diffusion models (DDM) assess and extract parameters corresponding to the underlying neural processes of decision making, by posing it as an evidence accumulation problem9. The three main parameters i.e., decision threshold (a) or boundary separation, an index of the amount of evidence accumulated prior to a decision; drift rate (v), an index of the speed of evidence accumulation; and response bias (z), the apriori tendency or bias towards one choice or another, represent different processes in choice reaction time paradigms.
Computational modelling in ADHD using DDM predominantly focused on either understanding the relationship between its parameters with proposed pathophysiological mechanisms such as reduced glutamatergic cortical drive; dopamine transfer deficit theory (or the deficit in transfer of dopaminergic responses to unconditioned rewards) and moderate brain arousal theory10 or task-related attention deficits, such as drift rate being an index of attention rather than measure of information processing10. Here we focus on the concept of risk or uncertainty tolerance with higher real world and clinical implications.
In this study, we exploit the capacity of a hierarchical modelling framework to decode the neural mechanisms of decision making specific to risk attitudes. We emphasise the role of response bias (z) currently underexplored in the ADHD literature, in risk-based decision making. Response bias is suggested to reflect the ‘apriori’ bias of an individual towards a specific choice, correlating with activity of fronto-parietal and fronto-basal ganglia networks11,12. Using hierarchical drift–diffusion modelling (HDDM), we previously analysed the two-step sequential learning task in compulsive disorder patients (obsessive compulsive disorder and alcohol dependence), in the context of uncertainty and conflict irrespective of choice preference7. Using the same task, we now apply a novel analysis, focusing on choice preference (risk attitudes) in ADHD and Healthy volunteers (HV) and compare the effects of MPH on each of the groups.
Methods
Participants and Protocol
A total of 49 participants (25-ADHD patients and 24-HV, a standard for behavioural studies with medication challenges between groups8,13,14) were recruited from a specialist clinic at Sussex Partnership NHS Foundation Trust and HV via classified advertisements and university mailing lists. The patient assessment included semi-structured interviews using the Diagnostic Interview for ADHD in adults (DIVA), completion of the Conner’s self-report Adult ADHD questionnaire, and review of school reports wherever possible. All patients had confirmed DSM-IV diagnoses of ADHD (See8 for complete details of selection/exclusion criteria). Local and national ethical approvals were obtained from Brighton and Sussex Medical School (14/014/HAR; 12/131/HAR) and the East of England (Hertfordshire) National Research Ethics Committee (reference: 12/EE/0256) and written informed consent was obtained. The demographics and clinical measures are reported in Table 2.
Table 2.
Parameter | ADHD | HV | p-value | |
---|---|---|---|---|
Age | 33.48 ± 9.8 | 31.04 ± 8.24 | ns | |
FSIQ | 108.77 ± 6.63 | 108.85 ± 6.83 | ns | |
BDI | 14.36 ± 8.7 | 6.63 ± 6.9 | 0.001 | |
ADHD Index | 24.16 ± 5.72 | 8.96 ± 5.03 | <0.001 | |
STAI Trait | 54.28 ± 11.44 | 37.7 ± 10.88 | <0.001 | |
Medication duration (months) | 32 ± 40.8 | Not applicable | ||
DSM—Inattentive | 18.96 ± 4.8 | 7.16 ± 4.4 | <0.001 | |
DSM—hyperactivity/Impulsivity | 18.04 ± 6 | 4.7 ± 3.3 | <0.001 | |
Gender—Male (Female) | 15 (10) | 16 (8) | Χ2(1) = 0.234, p = ns |
Behvaioural measures and Reinforcement Learning estimates | ||||
---|---|---|---|---|
Measure | Controls-Placebo | Controls-MPH | ADHD-Placebo | ADHD-MPH |
Percentage of risky choices | 0.52 ± 0.06 | 0.53 ± 0.06 | 0.53 ± 0.06 | 0.54 ± 0.07 |
Response time of risky choice (ms) | 841 ± 163.7 | 804.2 ± 121 | 840 ± 222.5 | 833.1 ± 220.6 |
Response time of Non-risky choice (ms) | 822.4 ± 147.6 | 800.7 ± 137.1 | 830 ± 214 | 840 ± 222.5 |
β1-Randomness Stage1 | 4.1 ± 3.7 | 5.7 ± 4.01 | 4.3 ± 2.6 | 4.45 ± 3.1 |
β2-Randomness Stage2 | 3.2 ± 1.68 | 3.7 ± 1.92 | 2.7 ± 1.76 | 3.48 ± 2.13 |
η1-Learning rate Stage1 | 0.52 ± 0.3 | 0.52 ± 0.27 | 0.46 ± 0.33 | 0.44 ± 0.35 |
η2-Learning rate Stage2 | 0.41 ± 0.25 | 0.43 ± 0.28 | 0.45 ± 0.31 | 0.42 ± 0.25 |
ps-Perseveration | 0.13 ± 0.2 | 0.14 ± 0.17 | 0.1 ± 0.1 | 0.14 ± 0.18 |
w-Model free - model based | 0.36 ± 0.26 | 0.36 ± 0.29 | 0.29 ± 0.23 | 0.24 ± 0.22 |
MPH methylphenidate, ADHD attention-deficit hyperactivity disorder, HV healthy volunteers, FSIQ Full Scale Intelligence Quotient, ADHD Index, BDI Beck Depression Inventory, STAI State-Trait Anxiety Inventory measures of healthy controls and patient group.
Exclusion criteria included any neurological or psychiatric history past or current, except for anxiety and/or unipolar depressive disorder currently in remission, history of significant head injury, and current drug or alcohol abuse. Each patient was managed on a stable regimen of methylphenidate (minimum 18 mg) or dexamphetamine (minimum 10 mg) for at least 2 months before study enrolment. For healthy controls, a history of serious cardiovascular conditions including cardiomyopathy, coronary artery disease, ventricular arrhythmia or hypertension, heart failure, current or recent use of monoamine oxidase inhibitors, anticoagulants, anticonvulsants or antipsychotics or a diagnosis of glaucoma were on exclusion criteria. Moreover, ADHD participants were routinely screened for these potential contra-indications to stimulant medication at clinical assessment.
The experimental design was a double-blinded placebo-controlled study with all participants completing two sessions of testing separated by a minimum of 1 week. ADHD patients were required to refrain from their regular medication on the test as well as 2 days before the day of testing. A.S who conducted the experiment and the entire participant testing was blind to treatment allocation. N.A.H (a qualified doctor) was aware of treatment allocation for safety reasons but was not involved in actual testing. During the first session, the participant in a randomised manner received either MPH or placebo in sealed envelopes. The alternate treatment was allocated in the second session. Patients with ADHD were given their normal morning medication dose or an inactive placebo whereas HV received 20 mg of MPH or placebo.
The sequential learning task
The widely tested and published sequential learning task7,15,16 consists of two stages; In stage 1, participants are presented with a stimulus pair (Fig. 1a). Upon selection, depending on the participant’s choice and stage 1 transition probability (P = 0.70 or 0.30), the second stage stimuli pair were then presented. The choice of a stimulus at stage 2 led to a reward (£1 or no reward). The reward probabilities of stage 2 stimuli were dynamic, based on a slow random Gaussian walk (P = 0.2–0.75, Fig. 1b). Subjects were allowed a decision time of 2 s at each stage, a 1.5 s transition time between stages, and observed the outcome for 1 s. Participants underwent a computerised self-paced training lasting 15–20 min and completed all 134 trials.
Risk in the sequential learning task
In this study, the risk associated with a stage 2 choice was calculated based on their variance in reward probability (Fig. 1c, calculated using Eq. 1)17. This formulation comes from a theoretical framework of risk/uncertainty17 supported by experimental human imaging studies, where dopaminergic, striatal, and midbrain neural activity resemble the inverted U-curve while coding for risk17,18. This definition also controls for the confounding effect of the expected value of the choice from risk.
1 |
where is the risk variable, is the reward probability of the transitioned stimulus ‘x’ at stage 2 for trial i and subject j. The riskiest choice was the stimulus whose reward probability was at chance level (P = 0.5), i.e., associated with the greatest variance in outcomes and the least risky choice was associated with either P = 0.25 or P = 0.75 with a greater likelihood of either winning or not winning and hence greater certainty (or lower uncertainty).
Using Eq. 1, we calculated the risk associated with the two choices (Fig. 1c) for each trial and labelled the option with the highest variance as the risky one (Fig. 1d). This information was then compared with the subject’s actual choice in a given trial, to classify it as either a risky or a non-risky choice. This was repeated for all the trials and subjects.
Hierarchical drift–diffusion model (HDDM)
HDDM falls under the class of sequential sampling methods which utilise Bayesian methods to estimate the DDM parameters such as the threshold (a) and the drift rate (v), response bias (z), and non-decision time (t) (Fig. 1e, f). We focus our analysis on the first three parameters as t primarily concerns motor and non-decision-making processes. The Bayesian-based HDDM estimates parameters as posterior probability distributions with the mean of the distribution representing the group’s average. The model utilises the Markov Chain Monte Carlo sampling method to estimate the distributions. The prior distribution for each parameter was based on 23 studies that reported the best fitting DDM parameters for multiple cognitive tasks19. The pre-analysis code was written in MATLAB version 2017a and the built-in HDDM python package by19 was used for parameter estimation.
Trials with response times less than 50 ms were discarded from the analysis to ensure model convergence and to constrain the data to realistic response times. The parameters were estimated by drawing 120,000 samples with the first 10,000 samples being discarded as burn-in and saving only every 10th sample. The convergence of the model was assessed by both visual inspection and computation of the Gelman-Rubin statistic, which indicated convergence (R^ < 1.1)20.
Additionally, we also estimated the parameters for accuracy (see7 for details on methods). We estimated all the three HDDM parameters (a, v, and z) in HV and ADHD in placebo and MPH conditions.
Statistical Analyses
In line with the HDDM estimation of parameters, we used Bayesian methods as implemented in JASP for statistical analysis. Bayesian repeated measures ANOVA was used to test the significance across groups, conditions, and their interactions and, if significant, post-hoc Bayesian paired, and independent t-tests were used to assess the mean difference. Evidence for hypothesis testing was inferred from the Bayes Factor (BF10), with a BF10 > 3 indicating moderate evidence and >100 strong evidence in support of the alternate hypothesis20. The Bayes Factor used to report the evidence for (or against) a hypothesis was obtained from JASP. Based on JASP guidelines, the normality and homogeneity of variance was met by the HDDM estimates. The behavioural (response time and accuracy) and demographic measures were analysed using frequentist repeated measures ANOVA and post-hoc Bonferroni corrected independent paired and sample two-tailed t tests using SPSS version 25. The correlation was performed using Bayesian statistics.
Results
Behavioural measures—healthy volunteers and ADHD population
Repeated measures ANOVA showed no significant main effects of drug or group or their interaction in terms of response time or percentage of risky choices (See Table 2).
Reinforcement learning parameters
Repeated measures ANOVA on the reinforcement learning (RL) parameters (beta (β1, β2), learning rate (η1 and η2), and perseveration (ps), model free-model based weight (w)) of ADHD and HV subjects during placebo and MPH condition showed no main effect of drug or group or drug by group interaction. The individual estimates of the parameters are reported in Table 2.
To assess the effects of depression and anxiety on the behavioural measures and reinforcement learning estimates, we ran repeated measures ANOVA of reinforcement learning parameters during placebo and MPH condition with depression and anxiety scores as covariates. We observed no main effect of drug or depression or anxiety or any interaction effects. Similarly, repeated measures ANOVA with covariates on the behavioural outcomes (risk and reaction time) that form the primary inputs to the HDDM model also showed no main effect of drug or group or risk or effect when adjusted for depression and anxiety covariates.
HDDM model estimates of Healthy volunteers and ADHD population
Using the extracted choice (risky vs non-risky) and response time information (Fig. 1f), we estimated the parameters (a, v and z) for ADHD and HV population when on drug or placebo. At baseline on placebo, both ADHD and HV had a response bias (z) towards the more certain choice suggesting lower uncertainty tolerance.
Bayesian repeated measures ANOVA showed no evidence for the main effect of the drug, group, or its interaction for either threshold or drift rate.
For response bias (z), repeated measures Bayesian ANOVA (Fig. 2) showed a very strong evidence for a main effect of drug (BF10 = 6.03 × 1011), group (BF10 = 86344) and group by drug interaction (BF10 = 3.65 × 106). Post-hoc Bayesian independent sample t-tests showed very strong evidence for the patient group to have a higher preference towards the risky choice when on MPH relative to HV (BF10 = 8.94 × 1014) and strong evidence when on placebo relative to HV (BF10 = 20.9). To check the effect of drug individually on each of the groups, we also ran Bayesian paired sample t-tests and show strong evidence for the drug to induce a preference towards the risky choice particularly in ADHD (BF10 = 1.16 × 1010) but also in HV (BF10 = 397.1).
The inclusion of clinical scores such as depression (BDI), anxiety (STAI-T), Inattention (DSM-I) and hyperactivity (DSM-H) scores as covariates, did not alter the strong evidence for the main effect of drug (5.88 × 1011), group (BF10 = 88086) and their interaction (BF10 = 3.9 × 106) without any covariates. There was no evidence for an effect of BDI score (BF10 = 2.7) but was observed for anxiety (BF10 = 36), DSM-I (BF10 = 138) and DSM-H (BF10 = 352). We further show evidence for an interaction effect between drug and anxiety (BF10 = 46), drug and DSM-I scores (BF10 = 1287) and drug and DSM-H scores (BF10 = 213) but not group interaction effects with depression or inattention or hyperactivity
To further understand this interaction effect between drug and clinical scores (anxiety, DSM-I and DSM-H), we calculated Bayesian correlations within individual groups between response bias and clinical scores in drug and placebo conditions. Anxiety and ‘z’ were not correlated within each group (Fig. 3a, b) for drug (rHV = 0.17, BF10 = 0.3; rADHD = 0.17, BF10 = 0.3) or placebo (rHV = −0.01, BF10 = 0.25; rADHD = 0.08, BF10 = 0.3). Similarly, inattention and ‘z’ were not correlated within each group (Fig. 3c, d) for drug (rHV = 0.03, BF10 = 0.26; rADHD = −0.13, BF10 = 0.3) or placebo (rHV = −0.24, BF10 = 0.47; rADHD = .005, BF10 = 0.25). Hyperactivity and ‘z’ were also not correlated within each group (Fig. 3e, f) for drug (rHV = 0.01, BF10 = 0.25; rADHD = −0.2, BF10 = 0.4) or placebo (rHV = 0.26, BF10 = 0.52; rADHD = −0.09 BF10 = 0.27). Our findings suggest the drug alone influences response bias in ADHD subjects irrespective of clinical features.
To highlight the specificity of MPH on risky decision making, we also extracted the parameters (a and v) pertaining to accuracy (See7 for details). Repeated measures ANOVA did not show any evidence for the main effect of group or drug or their interaction for both thresholds as well as drift rate.
Discussion
We show that ADHD patients when off medication have an apriori bias towards risky uncertain choices compared to controls. Methylphenidate enhanced preference towards risky choices (higher apriori bias) in both groups but had a significantly greater effect in the patient population. Thus, methylphenidate appears to shift tolerance towards risky uncertain choices possibly mediated by prefrontal dopaminergic and noradrenergic modulation. Using this novel application of HDDM to the sequential learning task, we focus on choice preference for risk and show the capacity to detect subtle differences, a unique strength of computational psychiatry relative to sole reliance on behavioural outcomes6,7,21.
We have previously dissociated contextual conflict and uncertainty with HDDM using this sequential learning task but did not take into account choice preference7. Here, we focused on choice preference, defining risk as greater outcome variance. This task captures an index of implicit risk as the probabilities are not explicitly known or defined and must be inferred or learned from outcomes. Although it might be tempting to interpret the findings as enhanced risk biases, they are consistent with a more adaptive or functional interpretation of MPH modulating uncertainty processing22 as the behavioural outputs remain unaltered.
Although ADHD has a higher response bias towards risky uncertain choice compared to healthy controls, at baseline both groups displayed greater underlying uncertainty intolerance with a bias towards the more certain choice. Uncertainty intolerance may be an important component of decisional impulsivity with choices made to avoid risky uncertain conditions. For instance, delay discounting may reflect intolerance of the uncertainty surrounding the likelihood of receiving the delayed choice or rapid decisions in the context of uncertainty or conflict that may similarly reflect discomfort with uncertainty. Thus, our results are consistent with results obtained from previous delay discounting studies, where both controls and ADHD participants chose higher delayed rewards in the presence of MPH23,24 possibly also indicating a higher tolerance towards uncertainty.
Our findings show a specific effect of methylphenidate on response bias, which reflects the priors or pre-existing expectations or biases towards the selected choice (here, the risky uncertain choice). This effect of methylphenidate on ADHD is unrelated to comorbid clinical features such as depression or anxiety or ADHD subtypes.
Mechanistically, the response bias parameter has been observed to correlate with the activity of fronto-parietal and fronto-basal ganglia networks11,12, areas dysfunctional in ADHD2. Within the fronto-parietal-basal ganglia circuitry, Lopez and colleagues observed the response bias parameter (estimated from a DDM) to correlate with the activity of ventromedial prefrontal cortical (vmPFC) during a multi-categorical value-based decision-making task25. The vmPFC, in addition to encoding value26,27, is implicated in risk-taking26,28,29, with structural abnormalities such as lower volumes, presence of stroke lesions and volume of damage in the medial orbito-frontal (mOFC) region of the vmPFC associated with increased risk-taking30,31. ADHD adolescents who are at high risk of developing substance abuse had higher vmPFC activation compared to age-matched controls while processing riskier outcomes32. Similar to the task used by Lopez and colleagues, the sequential learning task used in this study is also a value learning task, which requires the participants to learn the values (reward probabilities) of the choices and estimate the risk or uncertainty associated with them. Further imaging studies should focus on the vmPFC, which encodes both value and risk, as a potential anatomical candidate underlying the risk response bias parameter (z).
Mechanistically, MPH enhances the prefrontal dopaminergic and noradrenergic levels by inhibiting the DA and NE transporters and by indirectly activating D1-dopaminergic and α2-norepinephrine33. In ADHD patients, MPH enhanced the hypoactive vmPFC on placebo comparable to the observed levels of healthy controls2. Dopaminergic medications increase risk-taking by changing the subjective weighting of the reward probabilities34 or by increasing the baseline gambling tendency35. In this study, it is very likely to have similar effects where MPH via D1 and α2-norepinephrine receptors enhanced the vmPFC activity (one of brain’s core valuation system36) and altered the subjective value of the choices. This altered representation may then be reflected as a shift in the ‘apriori’ bias (response bias parameter) towards the risky choice. Further studies are required on risk representation to systematically understand the effects of MPH in ADHD subjects.
This study is not without its limitations. We have not systematically varied the dosage of MPH and assessed dose-related effects on risk-taking and response bias. The task primarily designed to be an implicit-value learning task rather than a risk evaluation task may influence the behaviour and the outcome.
To summarise, using a novel computational analysis on choice preference, we show that ADHD subjects have an apriori bias towards risk, enhanced with MPH compared to healthy controls. These findings might suggest a higher tolerance towards risk or uncertainty. Computational estimates may act as potential biomarkers capturing subtle effects of medication on risk behaviours not manifested in behaviours7,36,37. Our findings have relevance not just in ADHD but also in the impulsive-compulsive disorder spectrum, or the effects of cognitive enhancers. A risk-based study in Parkinson’s disease previously showed similar effects between MPH and pramipexole, a D2/D3 dopamine receptor antagonist prescribed for PD38. Critically we show using a single task commonly used to assess goal-directed and habit control, the capacity to assess both contextual uncertainty and conflict7 and risk uncertainty preferences. Our findings highlight the sensitivity of computational models to identify the subtle changes in disease and the effect of drug.
Acknowledgements
This work was supported by a Wellcome Trust Intermediate Fellowship (093881/Z/10/Z) to Dr. Harrison, Brighton and Sussex Medical School and the Dr. Mortimer and Dame Theresa Sackler Foundation. Dr. Voon is supported by a Medical Research Council Senior Clinical Fellowship (MR/P008747/1). Data sharing: supporting data will be made available on request.
Author contributions
A.M.: Substantial contributions to conception and design, analysis and interpretation of data; A.S.: Substantial contributions to conception and design, acquisition of data, and interpretation of data; M.C.: Substantial contributions to conception and design, acquisition of data, and interpretation of data; N.A.H.: Substantial contributions to conception and design, acquisition of data, and interpretation of data; V.V.: Substantial contributions to conception and design, analysis and interpretation of data.
Code availability
The analysis code used to estimate the computational parameters will be made available on request.
Conflict of interest
The authors declare that they have no conflict of interest.
Footnotes
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Schultz W. Neuronal reward and decision signals: from theories to data. Physiol. Rev. 2015;95:853–951. doi: 10.1152/physrev.00023.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Rubia, K. In Oxford Textbook of Attention Deficit Hyperactivity Disorder. 64–72 (Oxford University press, Oxford, 2018).
- 3.Verdejo-García A, Lawrence AJ, Clark L. Impulsivity as a vulnerability marker for substance-use disorders: review of findings from high-risk research, problem gamblers and genetic association studies. Neurosci. Biobehav. Rev. 2008;32:777–810. doi: 10.1016/j.neubiorev.2007.11.003. [DOI] [PubMed] [Google Scholar]
- 4.Breyer JL, et al. Young adult gambling behaviors and their relationship with the persistence of ADHD. J. Gambl. Stud. 2009;25:227–238. doi: 10.1007/s10899-009-9126-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Dekkers TJ, Popma A, van Rentergem JAA, Bexkens A, Huizenga HM. Risky decision making in attention-deficit/hyperactivity disorder: a meta-regression analysis. Clin. Psychol. Rev. 2016;45:1–16. doi: 10.1016/j.cpr.2016.03.001. [DOI] [PubMed] [Google Scholar]
- 6.Huys QJ, Maia TV, Frank MJ. Computational psychiatry as a bridge from neuroscience to clinical applications. Nat. Neurosci. 2016;19:404–13.. doi: 10.1038/nn.4238. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Mandali A, Weidacker K, Kim S-G, Voon V. The ease and sureness of a decision: evidence accumulation of conflict and uncertainty. Brain. 2019;142:1471–1482. doi: 10.1093/brain/awz013. [DOI] [PubMed] [Google Scholar]
- 8.Sethi A, Voon V, Critchley HD, Cercignani M, Harrison NA. A neurocomputational account of reward and novelty processing and effects of psychostimulants in attention deficit hyperactivity disorder. Brain. 2018;141:1545–1557. doi: 10.1093/brain/awy048. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Forstmann, B. U., Ratcliff, R., & Wagenmakers, E.-J. Sequential sampling models in cognitive neuroscience: Advantages, applications, and extensions. Annu. Rev. Psychol.67, 641–666 (2016). [DOI] [PMC free article] [PubMed]
- 10.Ziegler S, Pedersen ML, Mowinckel AM, Biele G. Modelling ADHD: a review of ADHD theories through their predictions for computational models of decision-making and reinforcement learning. Neurosci. Biobehav. Rev. 2016;71:633–656. doi: 10.1016/j.neubiorev.2016.09.002. [DOI] [PubMed] [Google Scholar]
- 11.Forstmann BU, Ratcliff R, Wagenmakers E-J. Sequential sampling models in cognitive neuroscience: advantages, applications, and extensions. Annu. Rev. Psychol. 2016;67:641–66. doi: 10.1146/annurev-psych-122414-033645. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Mulder, M., Van Maanen, L. & Forstmann, B. J. N. Perceptual decision neurosciences—a model-based review. Neuroscience277, 872–884 (2014). [DOI] [PubMed]
- 13.Aron, A. R., Dowson, J. H. Sahakian, B. J. & Robbins, T. W. Methylphenidate improves response inhibition in adults with attention-deficit/hyperactivity disorder. Biol. Psychiatry54,1465–1468 (2003). [DOI] [PubMed]
- 14.DeVito EE, et al. The effects of methylphenidate on decision making in attention-deficit/hyperactivity disorder. Biol. Psychiatry. 2008;64:636–639. doi: 10.1016/j.biopsych.2008.04.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Daw ND, Gershman SJ, Seymour B, Dayan P, Dolan RJ. Model-based influences on humans’ choices and striatal prediction errors. Neuron. 2011;69:1204–1215. doi: 10.1016/j.neuron.2011.02.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Gillan CM, Kosinski M, Whelan R, Phelps EA, Daw ND. Characterizing a psychiatric symptom dimension related to deficits in goal-directed control. Elife. 2016;5:e11305. doi: 10.7554/eLife.11305. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Schultz W, et al. Explicit neural signals reflecting reward uncertainty. Philos. Trans. R. Soc. Lond. B: Biol. Sci. 2008;363:3801–3811. doi: 10.1098/rstb.2008.0152. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Preuschoff K, Bossaerts P, Quartz SR. Neural differentiation of expected reward and risk in human subcortical structures. Neuron. 2006;51:381–390. doi: 10.1016/j.neuron.2006.06.024. [DOI] [PubMed] [Google Scholar]
- 19.Wiecki, T.V., Sofer, I. & Frank, M. J. HDDM: Hierarchical Bayesian estimation of the drift-diffusion model in Python. Front. Neuroinform.7, 14 (2013). [DOI] [PMC free article] [PubMed]
- 20.Krypotos A-M, Beckers T, Kindt M, Wagenmakers E-J. A Bayesian hierarchical diffusion model decomposition of performance in Approach–Avoidance Tasks. Cognition Emot. 2015;29:1424–1444. doi: 10.1080/02699931.2014.985635. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Frank, M. et al. Computational Psychiatry: New Perspectives on Mental Illness. Vol. 20 (MIT Press, 2016).
- 22.Advokat C. What are the cognitive effects of stimulant medications? Emphasis on adults with attention-deficit/hyperactivity disorder (ADHD) Neurosci. Biobehav. Rev. 2010;34:1256–1266. doi: 10.1016/j.neubiorev.2010.03.006. [DOI] [PubMed] [Google Scholar]
- 23.Pietras CJ, Cherek DR, Lane SD, Tcheremissine OV, Steinberg JL. Effects of methylphenidate on impulsive choice in adult humans. Psychopharmacology. 2003;170:390–398. doi: 10.1007/s00213-003-1547-2. [DOI] [PubMed] [Google Scholar]
- 24.Shiels K, et al. Effects of methylphenidate on discounting of delayed rewards in attention deficit/hyperactivity disorder. Exp. Clin. Psychopharmacol. 2009;17:291–301. doi: 10.1037/a0017259. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Lopez-Persem A, Domenech P, Pessiglione M. How prior preferences determine decision-making frames and biases in the human brain. Elife. 2016;5:e20317. doi: 10.7554/eLife.20317. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Schonberg T, et al. Decreasing ventromedial prefrontal cortex activity during sequential risk-taking: an fMRI investigation of the balloon analog risk task. Front. Neurosci. 2012;6:80. doi: 10.3389/fnins.2012.00080. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Domenech P, Redout J, Koechlin E, Dreher J-C. The neuro-computational architecture of value-based selection in the human brain. Cereb. Cortex. 2017;28:585–601. doi: 10.1093/cercor/bhw396. [DOI] [PubMed] [Google Scholar]
- 28.Christakou A, Brammer M, Giampietro V, Rubia KJJON. Right ventromedial and dorsolateral prefrontal cortices mediate adaptive decisions under ambiguity by integrating choice utility and outcome evaluation. J. Neurosci. 2009;29:11020–11028. doi: 10.1523/JNEUROSCI.1279-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Hartstra E, Oldenburg J, Van Leijenhorst L, Rombouts S, Crone EA. Brain regions involved in the learning and application of reward rules in a two-deck gambling task. Neuropsychologia. 2010;48:1438–1446. doi: 10.1016/j.neuropsychologia.2010.01.012. [DOI] [PubMed] [Google Scholar]
- 30.Bechara A, Damasio H, Damasio AR, Lee GPJJON. Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making. J. Neurosci. 1999;19:5473–5481. doi: 10.1523/JNEUROSCI.19-13-05473.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Studer B, Manes F, Humphreys G, Robbins TW, Clark L. Risk-sensitive decision-making in patients with posterior parietal and ventromedial prefrontal cortex injury. Cereb. Cortex. 2013;25:1–9. doi: 10.1093/cercor/bht197. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Hulvershorn LA, et al. Neural activation during risky decision-making in youth at high risk for substance use disorders. Psychiatry Res. 2015;233:102–111. doi: 10.1016/j.pscychresns.2015.05.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Spencer RC, Devilbiss DM, Berridge CW. The cognition-enhancing effects of psychostimulants involve direct action in the prefrontal cortex. Biol. Psychiatry. 2015;77:940–950. doi: 10.1016/j.biopsych.2014.09.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Ojala, K.E. et al., Dopaminergic drug effects on probability weighting during risky decision making. eNeuro5, ENEUEO.0330-18 (2018). [DOI] [PMC free article] [PubMed]
- 35.Rigoli F, et al. Dopamine increases a value-independent gambling propensity. Neuropsycholpharmacology. 2016;41:2658–2667. doi: 10.1038/npp.2016.68. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Lopez-Guzman S, Konova AB, Glimcher PW. Computational psychiatry of impulsivity and risk: how risk and time preferences interact in health and disease. Philos. Trans. R. Soc. B. 2019;374:20180135. doi: 10.1098/rstb.2018.0135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Wallsten TS, Pleskac TJ, Lejuez CW. Modeling behavior in a clinically diagnostic sequential risk-taking task. Psychological Rev. 2005;112:862. doi: 10.1037/0033-295X.112.4.862. [DOI] [PubMed] [Google Scholar]
- 38.Campbell-Meiklejohn D, et al. In for a penny, in for a pound: methylphenidate reduces the inhibitory effect of high stakes on persistent risky choice. J. Neurosci. 2012;32:13032–13038. doi: 10.1523/JNEUROSCI.0151-12.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Costa A, et al. Methylphenidate effects on neural activity during response inhibition in healthy humans. Cereb. Cortex. 2012;23:1179–1189. doi: 10.1093/cercor/bhs107. [DOI] [PubMed] [Google Scholar]
- 40.Schmidt A, et al. Comparative effects of methylphenidate, modafinil, and MDMA on response inhibition neural networks in healthy subjects. Int. J. Neuropsychopharmacol. 2017;20:712–720. doi: 10.1093/ijnp/pyx037. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Pauls AM, et al. Methylphenidate effects on prefrontal functioning during attentional-capture and response inhibition. Biol. Psychiatry. 2012;72:142–149. doi: 10.1016/j.biopsych.2012.03.028. [DOI] [PubMed] [Google Scholar]
- 42.DeVito EE, et al. Methylphenidate improves response inhibition but not reflection-impulsivity in children with attention deficit hyperactivity disorder (ADHD) Psychopharmacology (Berl). 2009;202:531–539. doi: 10.1007/s00213-008-1337-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Shalev L, Gross-Tsur V, Pollak Y. Single dose methylphenidate does not impact on attention and decision making in healthy medical students. J. Neurol. Res. 2013;2:227–234. [Google Scholar]
- 44.Gvirts HZ, et al. The effect of methylphenidate on decision making in patients with borderline personality disorder and attention-deficit/hyperactivity disorder. Int. Clin. Psychopharmacol. 2018;33:233–237. doi: 10.1097/YIC.0000000000000219. [DOI] [PubMed] [Google Scholar]
- 45.Voon V, et al. Waiting impulsivity: the influence of acute methylphenidate and feedback. Int. J. Neuropsychopharmacol. 2015;19:pyv074. doi: 10.1093/ijnp/pyv074. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The analysis code used to estimate the computational parameters will be made available on request.