Cross-species translational paradigms for assessing Positive Valence System as defined by the RDoC matrix

Abstract

Functions associated with processing reward-related information are fundamental drivers of motivation, learning, and goal-directed behavior. Such functions have been classified as the positive valence system under the Research Domain and Criteria (RDoC) criteria and are negatively impacted across a range of psychiatric disorders and mental illnesses. The positive valence system is composed of three comprehensive categories containing related but dissociable functions that are organized into either Reward Responsiveness, Reward Learning, or Reward Valuation. The presence of overlapping behavioral dysfunction across diagnostic mental disorders is in-part what motivated the RDoC initiative, which emphasized that the study of mental illness focus on investigating relevant behavior and cognitive functions and their underlying mechanisms, rather than separating efforts on diagnostic categories (i.e., transdiagnostic). Moreover, the RDoC approach is well-suited for preclinical neuroscience research, as the rise in genetic toolboxes and associated neurotechnologies enables researchers to probe specific cellular targets with high specificity. Thus, there is an opportunity to dissect whether behaviors and cognitive functions are supported by shared or distinct neural mechanisms. For preclinical research to effectively inform our understandings of human behavior however, the cognitive and behavioral paradigms should have predictive, neurobiological, and pharmacological predictive validity to the human test. Touchscreen-based testing systems provide a further advantage for this endeavor enabling tasks to be presented to animals using the same media and task design as in humans. Here, we outline the primary categories of the positive valence system and review the work that has been done cross-species to investigate the neurobiology and neurochemistry underlying reward-related functioning. Additionally, we provide clinical tasks outlined by RDoC, along with validity and/or need for further validation for analogous rodent paradigms with a focus on implementing the touchscreen-based cognitive testing systems.

Graphical abstract text:

This work reviews cross-species paradigms with validity for testing positive valence systems used in clinical populations, identified by the National Institute for Mental Health (NIMH) Research Domain Criterion (RDoC) initiative. Tests available for rats and mice are discussed, including whether they have evidence for face, predictive, or neurobiological validities. A focus is placed on touchscreen tests, but all operant tasks are included in the discussion, and future tests are recommended.

Introduction

Implementing RDoC in preclinical research

Psychiatric disorders continue to be amongst the leading causes of morbidity in today’s society, contributing to the diminished quality of life for millions of afflicted individuals and their caregivers (Rapaport, Clary, Fayyad, & Endicott, 2005; Vigo, Thornicroft, & Atun, 2016). Such conditions, also referred to as mental disorders or mental illness, are traditionally described using the diagnostic categories included in the Diagnostic and Statistical Manual of Mental Disorders (DSM), currently in its fifth edition (DSM-V) (Diagnostic and statistical manual of mental disorders: DSM-5™, 5th ed, 2013). These conditions include schizophrenia, bipolar disorder, major depression disorder, attention deficit hyperactive disorder, and anxiety disorder. While the DSM has been fundamental in identifying sufferers and their treatment, research, and acknowledgement of their legitimate healthcare concerns, the nature of psychiatric disturbances provides a number of challenges for assignment into distinct diagnostic categories. Firstly, diagnoses in psychiatry rely exclusively on the observation of behavioral/cognitive symptoms and subjective reporting, as there are currently no physical or chemical signatures of neural insult that are diagnostic (Insel & Cuthbert, 2015). Such diagnoses are complicated by the multi-faceted nature of symptomology between disorders, which includes a high heterogeneity of symptom presentation and severity within individual diagnostic categories. Furthermore, psychiatric disorders display extensive comorbidity, and the presence of shared symptomology further blurs the line between conditions (S. E. Morris & Cuthbert, 2012). For example, cognitive impairments such as executive dysfunction and abnormalities in reward processing (e.g., anhedonia) are present to some degree in most if not all psychiatric disorders. Even though psychotic symptoms are the characteristic manifestation of schizophrenia, they are not unique to the disorder (Dunayevich & Keck Jr, 2000). For example, over half of individuals with bipolar disorder will also experience psychotic episodes (Pope & Lipinski, 1978{Goodwin, 2007 #967). Distinct diagnostic categorization does not therefore necessarily equate to distinct behavioral profiles between groups.

Unsurprisingly, multiple genome-wide association studies (GWAS) and cross-disorder gene analyses also revealed shared genetic risk factors for psychiatric disorders that span across the boundaries of diagnostic categories (Andreassen, Hindley, Frei, & Smeland, 2023; B. Consortium et al., 2018; C.-D. G. o. t. P. G. Consortium, 2013; O'Donovan & Owen, 2016). Additionally, the presence of certain copy number variant (CNV) mutations confers a substantial risk factor for developing schizophrenia, and to a lesser degree bipolar disorder (W. T. C. C. Consortium et al., 2009; Grozeva, 2010) but have also been linked to neurodevelopmental disorders such as autism spectrum disorder and intellectual disability (Craddock & Owen, 2010; Fiksinski et al., 2017; Kushima et al., 2018). Moreover, there is evidence for shared pathophysiological mechanisms, such as abnormal protein expression, across psychiatric and neurodegenerative diseases (e.g. Alzheimer’s, Parkinson’s, schizophrenia, bipolar disorder), raising the question of whether research and treatment should be focusing on establishing the relationship between general pathophysiology and function outcome, rather than distinct diagnostic categories (Wingo et al., 2022).

These concerns were the genesis of the Research Domain and Criteria (RDoC) initiative, which emphasized focus on the mechanisms underlying cognitive, behavioral, and emotional functions, irrespective of specific diagnostic categories (Cuthbert, 2022; Cuthbert & Kozak, 2013; Insel et al., 2010). Instead, the RDoC matrix outlines multiple functional domains, including cognitive, arousal and regulatory, sensorimotor, social, negative valence, and positive valence systems; the final category being the focus of this review. The development of RDoC represents a conceptual paradigm shift, which transitions emphasis from disorder-based/diagnostic categories to characterizing psychiatric abnormalities in relation to neurological functions (Ayoub, Noback, Ahern, & Young, 2024; Insel et al., 2010; S. E. Morris & Cuthbert, 2012). Thus, by elucidating how aberrant genes or circuitry contribute to specific behavioral/emotional/cognitive functions, it may be possible to shed light onto how genetic predispositions may render vulnerability to a number of [often overlapping] mental illnesses (Kozak & Cuthbert, 2016; Sanislow et al., 2010). This approach would facilitate the characterization of how cognitive functions and behaviors are similar from a mechanistic point of view, but also how they differ, providing the opportunity to identify potential biomarkers of symptomology in disease (Nusslock & Alloy, 2017). From a preclinical research perspective, this approach likely provides a more sustainable avenue for successful translation of findings (Young, 2023; Young, Winstanley, Brady, & Hall, 2017). As informed by GWAS, genetically modified rodent models are developed to recapitulate identified genetic risk factors for psychiatric disorders and are not expected to replicate the entire symptomology of a given diagnostic category (Geyer & Markou, 1995; Markou, Chiamulera, Geyer, Tricklebank, & Steckler, 2009). For example, psychotic symptoms such as delusions and hallucinations are staples of psychiatric diseases and are currently unable to be accurately assessed in rodents (Kotov et al., 2024). Despite this limitation, identified genetic risk factors can be reproduced in animal models to assess their impact on other afflicted domains, such as cognitive or negative symptoms. Thus, preclinical models are an effective tool for dissecting the functional outputs of certain genes or circuit manipulations, despite not being able to replicate all of the facets of clinical disorders. This review provides an overview of the positive valence systems, their dysfunction in disease, and how the component functions are assessed in clinical and preclinical research. Although RDoC’s domains include various paradigms for assessing sub-constructs of the positive valence system, we will focus our discussion on the human paradigms that have, or have the potential to develop, analogous paradigms that available for rodents, thus allowing for cross-species validation to be assessed. Given the rise of touchscreen-based cognitive testing in preclinical research, we evaluate how these systems and similar operant paradigms can be implemented to facilitate cross-species cognitive testing in the positive valence domain.

Positive valence systems

Assessing valence systems in preclinical research

A fundamental driver of animal behavior is to seek out experiences that are pleasurable or desirable and avoid those that are painful or aversive (Skinner, 1963; Thorndike, 1911). Reward and punishment exist as opposing behavioral outcomes and along a continuum of intensity, enabling animals to form associations between stimuli, actions, and outcomes to inform future decisions and make predictions about the external world. In the simplest sense, behaviors are reinforced if they lead to favorable outcomes (positive reinforcement) and suppressed if they lead to negative outcomes (positive punishment) (Keller & Schoenfeld, 1950). Alternatively, behaviors can be suppressed under anticipation so that they will result in the loss of a reward (negative punishment) or facilitated if they will aid in avoiding punishment (negative reinforcement) (Gray, Stafford, & Tallman, 1991; Holland & Skinner, 1961; Murray Sidman, 1962; M. Sidman, 2006). As knowledge of past experiences is integrated with internal drives, valence (i.e., the subjective value) is attributed to environmental stimuli and used to guide goal-directed behavior (Miller & Cohen, 2001). The constructs that underlie responses to aversive and reinforcing stimuli are organized into two categories, the negative and positive valence systems, respectively. The negative valence systems, also known as avoidance systems, refer to functions associated with avoiding and preventing the experience of aversive stimuli. Behaviors associated with processing negative valence include acute, chronic, and future threat avoidance (e.g., fear and anxiety, respectively); as well as the experience of negative emotions (e.g., grief), and the loss of rewarding experiences. Opposingly, positive valence systems, or approach systems, include constructs that promote seeking rewarding experiences, the positive emotions associated with them, and the use of reward to facilitate learning and habit formation (Böttger et al., 2023; Taylor, Pearlstein, & Stein, 2020; Woody & Gibb, 2015).

Dysfunctional processing of positive and/or aversive stimuli are believed to underlie depression-like symptoms observed across mood and psychiatric disorders, where a reduction in positive feelings and motivation to obtain rewards may stem from increases in negative affect and/or a loss of positive affect (Dieterich et al., 2019). Due to the widespread prevalence and societal burden of mood dysregulation, determining the underlying neurobiology and potential therapeutic options is a massive initiative in preclinical research. Our focus will be on functions associated with the positive valence system such as perceiving and integrating reward value, using value representations to learn and make decisions, and the motivation to obtain rewards, which offer an alternative avenue for investigating the mechanism underlying valence processing in a preclinical setting.

The positive valence system as outlined by RDoC consists of three interconnected primary categories that are each composed of multiple sub-constructs. The overarching domains include reward responsiveness, reward learning, and reward valuation, which cover the underlying physiological processing and behavioral outputs that facilitate the motivation for obtaining, and the response to, positive stimuli. If the behavioral endpoint is to seek out or approach positive experiences appropriately, each component of the positive valence system yields an essential function that is integrated amongst the other constructs to drive behavior (Fig. 1). An individual must process positive stimuli as such and have a desire to obtain it, which compose the fundamental building blocks of reinforcement, “liking” and “wanting”, respectively. These constructs are included in reward responsiveness, which provides the foundation for positive stimuli to be used to guide learning and decision-making. Through reward learning, external stimuli are attributed salience as associations are made between actions and rewarding outcomes, and representations are formed that enable predictions to be made about future rewarding situations. Representations of a reward’s “worth”, which is also influenced by temporary internal states (e.g., hungry or satiated), will further inform the degree to which an individual will work to obtain a given reward, which composes reward valuation. A reward’s value guide cost versus benefit-based decision-making, which is necessary for optimizing behavior based on the given context (Salamone & Correa, 2002). While the categories are distinct in the functions they describe, they do not operate in isolation, and the interplay between these constructs is necessary for optimal behavior (Figure 1). Thus, investigating positive valence in preclinical research requires sensitive and selective behavioral paradigms that are able to tease apart discrete behavioral manifestations of the underlying processes, especially if they are to inform human testing.

Figure 1.

Outline of the positive valence system constructs and associated tasks outlined by RDoC. The three primary categories are organized by color and include the primary sub-constructs in corresponding-colored boxes (Reward responsiveness – Blue; Reward learning – Green; Reward valuation – Red). The hierarchal organization (i.e., Top to bottom) represents the dependence that higher order constructs (e.g., reward learning and valuation) have on the foundation of fundamental responsiveness to rewarding stimuli. The direction of the dark arrows shows constructs that directly influence the receiving function. For example, the degree to which one is satiated with a particular reward will influence both the intensity of reward anticipation and how the individual will respond to presentation of the reward. The dotted arrows demonstrate mechanisms of feedback that can influence future responses to reward or updating reward-related representations. For example, changes in the environmental context can lead to reward prediction errors, which are used through reinforcement learning to update the relationship between relevant environmental stimuli and their association with rewarding outcomes and will influence future interactions with situations that were previously associated with reward. * denote preclinical paradigms that require further validation.

Implementing translational cognitive paradigms to probe positive valence systems

To study the mechanisms underlying cognition and behavior, conducting preclinical research in rodents provides an unmatched avenue to probe specific genes and circuits with high specificity and temporal precision to determine their roles in behavior. For this potential to be realized however, tasks used in preclinical research must be translatable to those used in human clinical settings (Grottick et al., 2021; Palmer et al., 2021). This concept, particularly reinforced by the high failure rate from preclinical to clinical stages of drug treatments for brain disorders (Mullard, 2016), has inspired the development of cross-species cognitive and behavioral paradigms that emphasize translational validity. The evolution of clinical testing apparatuses has progressed from pen and paper-based testing to primarily computerized automated systems, utilizing mouse and keyboards, joysticks, and touchscreens. Similarly, the development of automated behavioral paradigms for rodents enables behavioral testing with minimal experimenter influence and protocol deviation, utilizing systems with levers, nose-holes, or touchscreens. For cross-species task development, computerized systems can be used to adapt human paradigms for testing in rodents (reverse translation e.g., (Perry et al., 2009)) or create human variants of animal tasks (forward translation e.g.,(Young, Light, Marston, Sharp, & Geyer, 2009)).

The RDoC matrix includes six domains that are composed of distinct, but related, functional categories, aligning with the concept that certain mental processes possess overlap in terms of underlying neural mechanisms and their contributions to functional behavior. By having standardized testing equipment that can run various cognitive tests on the same platform, task batteries can be developed that probe distinct cognitive and behavioral functions with minimal off-target differences across tasks. A number of cross-species task batteries have already been developed and proposed, including the Cambridge Automated Neuropsychological Test Battery (CANTAB) (T. W. Robbins et al., 1998; T.W. Robbins et al., 1994), the NIMH’s Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS) (Barch et al., 2009; Carter et al., 2011; Young, Jentsch, Bussey, Wallace, & Hutcheson, 2013), the Novel Methods leading to New Medications in Depression and Schizophrenia (NEWMEDS) (Hvoslef-Eide et al., 2015), and the EMOTICOM test battery (Roiser & Sahakian, 2013). (Bland et al., 2016) (Savulich et al., 2018). Although rodent operant and touchscreen systems offer similar advantages in terms of standardization, automation, and opportunities for task development, the rise of touchscreen-based technology may offer substantial benefits for minimizing task differences across species (See Box 1).

Box 1:

Aspects of the touchscreen systems that are well-suited for studies aimed at investigating positive valence system functions.

One of the largest advantages of touchscreen-based systems is the ability to provide more varied stimuli and additionally identical information to human and rodents subjects (e.g., visual images) that require similar motor responses (i.e., finger touch or nose-poke), maximizing face validity (Bussey et al., 2008). An additional feature of the touchscreen apparatus that is well-suited to assessing positive valence functions in preclinical research in particular is the low baseline motoric demand that is associated with nose-poking the touch-sensitive screen. The systems are equipped with a series of infra-red beams that line the front of the screen, and simply require a mouse to break the beam path to generate a response. The low physical requirements have been previously described as beneficial for assessing cognition in models of neurodegeneration or damage that are associated with impaired motor function, such as stroke, Parkinson’s or Huntington’s disease (Bussey et al., 2008; Dumont, Salewski, & Beraldo, 2021; Houlton, Barwick, & Clarkson, 2021; Morton, Skillings, Bussey, & Saksida, 2006; Oomen et al., 2013). In relation to reward-related processes, a key domain is the degree to which one will exert effort to achieve rewarding stimuli, based on the internal reward valuation. As a result, lower baseline effort demands enables a more gradual progression span from low to high demands, providing a larger scale for the assessment of behavior. Moreover, the touchscreens provide flexibility for how effort can be modulated, such as increasing the number of screen responses to get a reward (e.g., Progressive ratio breakpoint (Hailwood, Heath, Robbins, Saksida, & Bussey, 2018; Heath, Bussey, & Saksida, 2015) or by making the stimuli more challenging to interact with by adjusting its location on the screen (e.g., Rearing effort discounting (Lopez-Cruz et al., 2024)) (See category 3). In contrast, a paradigm with a high baseline effort requirement, such as climbing over a barrier to collect a reward, may be less sensitive to subtle changes in motivation. Furthermore, if effort to achieve reward is not the primary measure of focus, higher physical requirements (e.g., learning a location-reward association in a maze) may act as a confound for manipulations aimed at alternative reward functions and not match onto human efforts. As a result, it may be more challenging to identify if a manipulation is affecting reward learning processes (i.e., the intended function) or instead altering the degree of effort an animal is willing to expend to explore the maze and acquire the association, particularly for relevant to human testing. While various behavioral apparatuses have been instrumental for informing our understanding of reward-related functions, the touchscreen systems are well equipped to investigate effort-related reward valuation (Heath, Phillips, Bussey, & Saksida, 2016; Markou et al., 2013; Phillips, Lopez-Cruz, et al., 2018; Young & Markou, 2015).

By maximizing the translational potential of behavioral paradigms, there is opportunity to implement manipulations and recording techniques to identify and characterize neural biomarkers across species underlying disease-relevant functions (Cavanagh et al., 2021; Cavanagh et al., 2022; Noback et al., 2024). Although it is challenging to probe specific circuits and cellular populations in humans, the availability of techniques with cellular and temporal precision readily available for use in rodents. For cellular monitoring, optical techniques enable proxies of neural activity, such as genetically encoded calcium indicators (GECI) (Cui et al., 2013; Flusberg et al., 2008; Lütcke et al., 2010), voltage indicators (GEVIs) (Akemann et al., 2013; Dimitrov et al., 2007), biosensors of neuromodulators (Jing et al., 2018; Patriarchi et al., 2018; H. Wang, Jing, & Li, 2018), and in vivo electrophysiology, to be readily imaged in freely moving rodents. For cellular manipulations, the toolbox of available optogenetic opsins and methods of expressing in target cell populations has grown exponentially, enabling cellular inhibition and activation with various degrees of membrane kinetics and technical application (Bernstein & Boyden, 2011; Bernstein, Garrity, & Boyden, 2012; Förster, Dal Maschio, Laurell, & Baier, 2017; Madisen et al., 2012; Rost, Schneider-Warme, Schmitz, & Hegemann, 2017). Thus, there are a number of avenues for investigating correlative and causative relationships between cellular targets and temporally-specific behavioral events, which is strengthened through combination with automated paradigms possessing cross-species validity. Importantly, such an integrated approach aligns with RDoC’s emphasis on investigating the mechanisms underlying distinct cognitive and behavioral functions, providing insight into potential cellular dysfunctions relevant to psychiatric disorders (See Table 1).

Table 1.

Overview of studies that have implemented optical-based techniques to study mechanisms of the positive valence system in freely moving mice.

Techniques	Positive Valence Domain	Cellular Target	Specificity	Findings
Calcium Imaging	Reward Learning (Pavlovian)	D1R and D2R medium spiny neurons (NAc)	Population level recording	D1R late learning: ↓ activity during S+ approach and ↑ activity following reward consumption.
				D2R late learning: ↑ activity following reward delivery. (Skirzewski et al., 2022).
	Reward Learning (Operant conditioning)	CAMKii-expressing neurons (BLA)	Population level recording	Early learning: ↑ BLA learning at reward deliverxy
				Late learning: ↑ BLA activity at reward-predicting cue (Crouse et al., 2020).
	Reward learning (Probabilistic)	Glutamatergic neurons (vmOFC)	Population level recording	Early learning:↑ activity following informed (reward or not) feedback.
				Late learning: ↓ activity following informed feedback (Barnes et al., 2023).
Neurotransmitter Imaging (Fiber photometry)	Reward Learning (Pavlovian)	Dopamine (NAc)	Population level recording	Early learning: ↑ DA release following reward delivery.
				Late learning: ↑ DA activity during S+ approach. ↓ DA release following reward delivery (Skirzewski et al. 2022).
		Acetylcholine (NAc)	Population level recording	Late learning: ↓ Ach release following reward (Skirzewski et al. 2022).
	Reward Learning (Operant conditioning)	Acetylcholine (BLA)	Population level recording	Early learning: ↑ Ach activity at reward delivery.
				Late learning: ↑ Ach activity at reward-predicting cue (Crouse et al. 2020).
Mini-scopes	Reward Learning (Simple discrimination)	D1R and D2R medium spiny neurons (NAc)	Individual cellular resolution	Heterogeneity within both populations, particularly in D1R neurons. D2R neuron activity primarily ↑ following errors, and ↓ following correct responses. Both signals were strengthened through learning (Nishioka et al., 2023).
	Reward Valuation (Operant progressive lever pressing)	ACC excitatory neurons	Individual cellular resolution	↓ ACC activity prior to progressive lever pressing when rewards are freely available (E. E. Hart, Blair, O'Dell, Blair, & Izquierdo, 2020).
Optogenetics	Reward Valuation (Delayed discounting)	Aldh1a1 GABAergic interneurons (VTA)	↑ Input to VTA	↑ Preference for larger, delayed reward (X. Li et al., 2021).
	Reward Learning (Simple discrimination)	D1R medium spiny neurons (NAc)	Direct Activation	↑ Preference for stimulus paired with NAc activation (Attachaipanich, Ozawa, Macpherson, & Hikida, 2023)
	Reward Learning (Deterministic learning and reversal)	VTA neurons projecting to NAc core	Direct Activation	↓ Early reversals; ↓ win-stay, ↑ lose-shift, ↑ lose-stay (A. K. Radke et al., 2019)
		D1R and D2R medium spiny neurons (NAc)	Direct Inhibition during 1) Post-errors; 2) ITI	D2R post-error: ↓ correct choices; D1R during ITI: ↓ correct choices (Niskioka et al. 2023).
	Reward Learning (Operant conditioning)	Cholinergic NBM-BLA terminals	Direct activation during learning	↑ Correct choices throughout task learning (Crouse et al. 2020).
	Reward Learning (Probabilistic)	Glutamatergic neurons (vmOFC)	Direct activation	↓ Total number of reversals and ↓ win-stay choices. ↑ Total number of reversals (Barnes et al., 2023). ↑ Number of lever presses to obtain a reward (i.e.., ↑ breakpoint) (Zhou et al., 2022).

In this review, we will discuss the three functional categories that comprise the positive valence system and how these functions are or can be tested using touchscreen-based cognitive tasks in humans and rodents. To facilitate the translation of cognitive and behavioral paradigms across species, tasks must provide multiple forms of validity. Most, if not all, will have face validity (phenomenological similarity), more importantly, however, are predictive (including pharmacological), and neurobiological validities (see (Young, 2023)). Predictive validity is the capacity by which a task can predict the outcome of a manipulation in animals as it would occur in humans. Pharmacological predictive validity is more specific and refers to the ability of a task to recreate drug-induced changes in performance consistently across animals and humans. Neurobiological validity refers to confirmation that the same neurobiological circuits underlie the performance of a task in animals and humans. Thus, we will describe the evidence for the validity of cross-species tasks to assess domains within the positive valence system with an emphasis on touchscreen-based paradigms. Finally, we will describe the neurobiology and chemistry underlying the various functions associated with reward-related processing and how dysfunction of these systems can manifest as pathological cognition and behavior.

Category 1: Reward Responsiveness

Deciphering the neural processes involved in reward responsiveness requires tasks that are less cognitively based so-as to refrain from conflation of data with responses elicited by other positive valence domains (i.e., reward learning). Importantly, the components of reward responsiveness provide the foundation for all functions in which reward can be used to modulate current and future behavior, making the ability to properly isolate and test these constructs imperative for research.

Reward anticipation

Reward anticipation shapes future goal-directed behavior, as evidenced by drug-paired cues driving drug-taking and drug-seeking responses across species. The RDoC initiative recommends the computerized monetary incentive delay (MID) task (Knutson, Westdorp, Kaiser, & Hommer, 2000) to measure reward anticipation as it effectively segregates such responses from other reward processes. The MID task was created to dissect neural responses at distinct reward stages in participants performing a simple (i.e., low cognitive demand) instrumental response (Knutson et al., 2000). In this task dissociable anticipatory win-cues signal trials where a participant can earn money after a simple reaction-timed response (Figure 2). Neutral cues instead signal trials where reward cannot be earned, though participants still perform the reaction-timed response. Responses are followed by outcome feedback (e.g., win or loss), enabling distinct incentive -anticipation and -outcome phases. Importantly, the response window is adjusted based on individual reaction-times to reduce potential performance and/or practice effects (Knutson et al., 2000). Thus, while anticipatory win-cues reduce reaction-time relative to neutral cues (Dhingra et al., 2020; Knutson, Fong, Adams, Varner, & Hommer, 2001), studies utilizing this task largely focus on neural recordings, rather than behavior. The inherent low motor and cognitive demand required makes the MID task easy to perform during more complex neurophysiological assessment.

Figure 2.

The monetary incentive delay (MID) task for humans (left) and its’ equivalent for rodents (right). Measuring reward anticipation requires that MID tasks be paired with neurological recording measures, such as fMRI in humans and regional oxygen or calcium imaging in rodents. Subjects learn to associate a win-cue with a rewarding outcome, providing an anticipatory window between cue and reward receipt. Thus, while MID tasks include a component of reward learning, the objective measurement is the neurological correlates of an anticipated reward elicited by an associative cue, rather than specific behavioral endpoints. For rodent research, MIDs can be translated using discriminative stimulus tasks, where an animal is trained to associate a discriminable cue with a rewarding outcome and neural activity can be measured during the anticipatory period between cue and reward receipt.

Early work utilizing the MID task in combination with functional magnetic resonance imagining (fMRI) revealed activation of the ventral striatum and ventromedial prefrontal cortex (vmPFC) by reward anticipation and reward outcome, respectively (Knutson et al., 2001) (Table 2). These data called attention to differential regional recruitment during distinct phases of reward processing and highlighted the ability to temporarily sparse apart neural correlates of reward anticipation. Despite these distinctions, however, subsequent work suggested these processes are entwined, since people with vmPFC lesions demonstrated reduced ventral striatal volume and responsivity to reward anticipation when compared to healthy controls (Pujara, Philippi, Motzkin, Baskaya, & Koenigs, 2016). The MID task has since been adapted to explore the anticipation of other incentives, including social reward (Martins et al., 2021; Rademacher et al., 2010; Spreckelmeyer et al., 2009), food reward (J. J. Simon et al., 2015a), drugs (Nestor & Ersche, 2023), and sexually explicit content (Knutson, Wimmer, Kuhnen, & Winkielman, 2008; Markert, Klein, Strahler, Kruse, & Stark, 2021), emphasizing the flexibility of this task to probe reward anticipation across a variety of reinforcing stimuli. Further fMRI in the these tasks consistently highlighted striatal activation in response to anticipatory win-cues, while also extending more widespread cortical and subcortical activation to reward anticipation, including areas of the cingulate cortex, insula, thalamus, and amygdala (Chen, Chaudhary, & Li, 2022; Dhingra et al., 2020; Dugre, Dumais, Bitar, & Potvin, 2018; Oldham et al., 2018; Wilson et al., 2018). MID task variations include monetary loss trials signaled by alternative cues, creating an opportunity to explore loss anticipation (Knutson et al., 2000). Efforts to parse apart distinct networks associated with incentive versus loss anticipation have revealed overlapping regional recruitment across positive and negative valences, with some contrasts (Chen et al., 2022; Oldham et al., 2018; Wilson et al., 2018). Given our focus on positive valence systems, loss anticipation will not be further discussed.

Table 2.

Evidence for cross-species validation of tasks that assess reward anticipation

Reward Responsiveness – Reward Anticipation
Validity Domain	Human Task – Monetary Incentive Delay	Rodent Task – Discriminative Stimulus Task
Predictive	Bipolar and stimulant use disorders: ↓ striatal activity, ↓ fronto-temporal-limbic connectivity during anticipation (Nestor et al. 2023; Johnson et al. 2019; Patel et al. 2013; Wakatsuki et al. 2022). Schizophrenia: ↓ striatal activation during anticipation.	N/A
Neurobiological	During anticipation: ↑ activity in NAc (Knutson et al. 2001), striatum, cingulate cortex, insula, thalamus, amygdala (Dhingra et al. 2020; Chen et al. 2022; Oldham et al. 2018; Wilson et al. 2018; Dugre et al. 2018), substantia nigra, and VTA (Schott et al. 2018). During anticipation: ↑ DA release in ventral striatum (Schott et al. 2008).	Following S+ response: ↑ O2 signals in NAc (Francois et al. 2012). During S+ approach: ↑ NAc DA release, tuned during learning (Skirzewski et al. 2022).
Pharmacological	Amphetamine: Prolongs ventral striatal tonic DA release; ↓ phasic DA release (Knutson et al. 2004). Ketamine: ↓ ventral striatum response during anticipation (Francois et al. 2016).	Ketamine: ↓ ventral striatum response during anticipation (Francois et al. 2016).

Providing clinical sensitivity, the MID task captures altered neural responses during reward anticipation in neuropsychiatric populations with known reward processing disturbances. For example, compared to healthy participants, people with bipolar or stimulant-use disorder exhibited blunted striatal activation and reduced fronto-temporal-limbic connectivity during reward anticipation (S. L. Johnson, Mehta, Ketter, Gotlib, & Knutson, 2019; Nestor & Ersche, 2023; Patel et al., 2013; Wakatsuki et al., 2022). Similarly, striatal activation during reward anticipation correlated with psychometric symptomology in people with schizophrenia (Juckel et al., 2006; Nielsen et al., 2012; Zeng et al., 2022) and was inversely related with impulsivity as measured by the Barratt Impulsiveness Scale in pathological gamblers and detoxified problem drinkers (Balodis et al., 2012; Beck et al., 2009). As RDoC highlights, abnormalities in reward-related behavioral functions and associated neurophysiological processes span across diagnostic categories in support for a systems and circuitry-based approach for understanding behavioral dysfunction. Hence, the MID task can be a powerful transdiagonistic tool to study the mechanisms and treatment of reward anticipation dysregulation across neuropsychiatric disorders.

Mesolimbic dopamine is highly implicated in reward anticipation. Early studies in nonhuman primates suggested repeated exposure to a reward shifts phasic midbrain dopamine release following the reward to cues that predict reward (W. Schultz, Dayan, & Montague, 1997), replicated in rodents (Day, Roitman, Wightman, & Carelli, 2007). Similarly, in humans performing the MID task, combined fMRI and positron emission tomography (PET) revealed reward anticipatory cues activated dopamine-rich midbrain regions which correlated with ventral striatal dopamine release upon reward receipt (Schott et al., 2008). Additionally, amphetamine hemodynamically altered striatal activation during reward anticipation in healthy participants, in a manner that suggested a blunted phasic, yet prolonged tonic, dopaminergic response (Knutson et al., 2004). Because this task has minimal behavioral requirements, temporal precision to track sub-second fluctuations in neural dynamics is required to delineate neural structures further. These temporally precise neurotechnologies are invasive, however, requiring animal models. For example, fiber photometry and optogenetics could determine distinct neuromodulators and implicated receptor systems that contribute to reward anticipation, and permutations relative to models of neuropsychiatric disorder. Further, integration of neural recordings with pharmacology manipulations can inform the development of targeted therapeutics to treat aberrant neural signaling.

There is no standardized animal equivalent to the human MID task, however discriminative stimulus (DS) tasks may be an avenue for the development of an analog. Importantly, food rewards in animals should not cause concern for interpretation of data across tasks, since the anticipation of food reward elicits striatal activity similar to monetary rewards in humans (J. J. Simon et al., 2015b). In DS tasks, animals are trained to discriminate between two cues that predict reward delivery (CS+), or non-reward (CS-) (Figure 2). Variations of these tasks either require an instrumental response following CS+ presentation to earn reward (Francois, Conway, Lowry, Tricklebank, & Gilmour, 2012) or they non-contingently deliver reward following CS+ presentation (Day et al., 2007), the former more closely resembling the MID task. For example, Francois and colleagues (Francois et al., 2012) trained rats to press a lever that extended into a chamber following the presentation of a CS+. Thus, like the MID task the anticipatory reward cue (CS+) preceded both the response for reward, as well as its delivery. Regional oxygen (O₂) amperometry in this task revealed rats exhibited increased nucleus accumbens (NAc) O₂ signals following a lever-press during the CS+, but not the CS-. Differential O₂ signaling between the CS+ and CS- suggested this activity likely did not reflect motoric consequences of the task. Further, CS- responses did not result in reward delivery, thus reward receipt did not likely contribute to altered NAc activity. These results somewhat match those obtained by human fMRI MID task studies wherein reward anticipation increased striatal activation – highlighting cross-species neurobiological validity. In the same rodent DS task acute sub-anesthetic doses of ketamine attenuated ventral striatum responses to “reward anticipation” in rats, while also reducing such neural activation in humans performing the MID task (Francois et al., 2016), providing cross-species pharmacological predictive validity. Hence, DS tasks may provide a rodent task equivalent to the human MID task.

Importantly, DS tasks can be easily adapted for touchscreen applications in rodents, to further enhance research utility. In a touchscreen version of the task, animals could be exposed to visual cues, as done in humans with the MID, rather than auditory cues to enhance consistency across species in this task. Further, the strength (i.e. Resemblance) of cues to the anticipatory win cues could be altered to permit parametric validations of DS tasks and to probe incentive conditioning generalization. A similar paradigm available in the touchscreen systems is the autoshaping task, in which animals are trained to discriminate between spatially distinct stimuli for reward delivery (C+) or no reward (C-) (Bussey, Everitt, & Robbins, 1997). Thus, time-locked neuronal recording or manipulation techniques to the C+ approach behavior can provide an index of reward anticipation, though could be conflated with motoric activity. Of course, data from these tasks can equally inform reward learning (discussed below). A recent study demonstrated the value of integrating temporally and cellular precise techniques during the C+ approach phase, revealing the interplay between NAc dopamine, cholinergic interneurons, and medium spiny neuron (MSNs) activity during stimulus-reward associations (Skirzewski et al., 2022). Specifically, NAc dopamine responses became specifically tuned during acquisition when mice approached the C+ and was robustly suppressed by reward, corroborating the strong recruitment of the striatum in reward anticipation consistent with human MID task performance. Hence, MID-like tasks for rodents using touchscreen systems can be integrated with neural recording techniques to identify relevant underlying mechanisms of reward anticipation.

Initial Response to Reward

Similar to reward anticipation, consummatory responding may alter the motivation to seek reward and guide goal-oriented behavior. As mentioned, the MID task contains an outcome feedback phase which can be used to probe initial responses to reward. Unlike the widespread cortical and subcortical activation patterns observed during reward anticipation, reward receipt in the MID task more selectively recruits the orbitofrontal cortex (OFC) and prefrontal cortex (PFC) (Knutson et al., 2001; Knutson, Fong, Bennett, Adams, & Hommer, 2003; Oldham et al., 2018). Further, unlike the indistinguishable engagement of brain regions by reward win- and loss-anticipation (see above), the OFC and PFC may differentially respond to win and loss outcomes thereby acting as neural valence markers. For example, medial PFC (mPFC) activity was elevated following the receipt of an anticipated win but lowered when an anticipated win did not occur (Knutson et al., 2003). Similarly, oscillatory patterns, such as the reward positivity (RewP) event related potential (ERP), differentially respond to the receipt of reward versus non-reward, see details below (Broyd et al., 2012; Novak & Foti, 2015).

The RDoC initiative recommended computerized Simple Guessing Tasks to inform initial responses to reward since observations in the MID task may be confounded both by incentive anticipatory responding and practice effects. In one, participants are instructed to guess whether the number between 1 and 9 on a playing card is higher or lower than 5 for a monetary reward, or loss, outcome (Delgado, Nystrom, Fissell, Noll, & Fiez, 2000). Similarly, in the Doors Task participants are asked to choose between two doors to win or avoid monetary loss. Across tasks, outcomes are randomized and not contingent on a response, thereby eliminating potential learning and practice effects. Consistent with the MID task, heightened mesocorticolimbic activation and RewP ERPs were associated with win over loss outcomes in Simple Guessing Tasks (Carlson, Foti, Mujica-Parodi, Harmon-Jones, & Hajcak, 2011; Delgado et al., 2000; Tricomi, Delgado, McCandliss, McClelland, & Fiez, 2006) (Table 3). Instead, loss feedback decreased striatal responses below baseline levels and attenuated reward-delivery ERP signals (Carlson et al., 2011; Delgado et al., 2000; Foti, Weinberg, Dien, & Hajcak, 2011; Tricomi et al., 2006). Thus, striatal activation and reward-related ERP measures act as neural valence indicators (win/lose) of outcomes. Importantly, blunted RewP ERPs during reward receipt appear to be transdiganostic as they occur in individuals with depression (Proudfit, 2015; Weinberg, 2023), attention-deficit/hyperactivity disorder (ADHD) (Kallen, Perkins, Klawohn, & Hajcak, 2020) and in healthy populations following an acute stressor (Burani et al., 2021; Porcelli, Lewis, & Delgado, 2012). Within depressed populations, reduced RewP activity has predicted both subsequent remission, and as well as treatment responsivity (Burkhouse et al., 2018; Klawohn, Brush, & Hajcak, 2021). Further, attenuated neural differentiation between gains and losses as determined by RewP during Simple Guessing Tasks were associated with risk propensity extremes as measured by the Balloon-Analog Risk Task (Huggins, Weinberg, Gorka, & Shankman, 2019). Hence, this altered neural activity pattern may participate in maladaptive decision-making. Thus, RewP may be a prospective biomarker for disordered reward processing across neuropsychiatric disorders, which can be leveraged to develop targeted treatments.

Table 3.

Evidence for cross-species validation of tasks that assess initial response to reward.

Reward Responsiveness – Initial Response to Reward
Validity Domain	Human Task – Simple Guessing Task	Rodent Task – T-Maze Guessing Task
Predictive	↓ RewP ERPs observed in depression (Weinberg et al. 2022; Proudfit et al. 2015), ADHD (Kallen et al. 2020), and following stress (Burani et al. 2021; Porcelli et al. 2012).	N/A
Neurobiological	Following win: ↑ mesocorticolimbic activation; ↑ RewP ERPs (Carlson et al. 2011; Delago et al. 2000; Tricomi et al. 2006). Following loss: ↓ striatal responses; ↓ reward-delivery ERP signals (Carlson et al. 2011; Tricomi et al. 2006; Delgado et al. 2000; Foti et al 2011). During outcome: ↑ vmPFC activity (Knutson et al. 2001). During reward: ↑ OFC activation (Knutson et al. 2001; Knutson et al. 2003; Oldham et al. 2018).	N/A
Pharmacological	N/A	N/A

Simple Guessing Tasks have been utilized in animal studies, though traditionally for assessing stereotypic responding. For example, the T-Maze apparatus is utilized wherein one of two goal boxes are randomly baited with reward. However, unlike humans who receive instruction to randomly choose, animals may form associations between choice behavior (movement to left or right goal post) and reward outcome. While chance reward outcomes prevent associations, neural activity underlying attempted learning may differ from known guessing, a potential translational confound. Simple Guessing Tasks could be adapted to rodent touchscreens, using distinct visual or spatial stimuli that are randomly rewarded throughout a session. In fact, this approach has been used in probabilistic learning tasks (see below). Given the lack of cognitive demand in Simple Guessing Tasks (e.g., absence of learning), such paradigms may be better suited for assessment of initial reward responding. Combining photometry and intracranial drug delivery in these prospective touchscreen tasks would inform both the neural systems and pharmacology relevant to this domain. Validity studies would be required, however, to recreate such RewP outcomes, as seen in Simple Guessing Tasks and in some probabilistic learning tasks (Cavanagh et al., 2021).

Reward Satiation

Responsiveness to a reward is influenced by the degree to which that reward holds subjective value. Reward satiation refers to the (often temporary) loss of incentive value for a reward as a result of experiencing it over time. Thus, reward satiation is tightly linked to the hedonic pleasure associated with reward, signaling the endpoint at which a rewarding stimulus is no longer valued. Satiety is often discussed in the context of consummatory behaviors, where hunger drive is diminished while eating or food preference is reduced when it is overconsumed. The latter refers to reward-specific satiety, resulting in a steeper reduction in the value of a specific reward that does not always generalize (Berridge & Kringelbach, 2008; Rolls, Rolls, & Rowe, 1983), though can occur across sensory domains, including for olfactory, auditory, visual, and tactile stimuli (Cabanac, 1971; Rolls & Rolls, 1997; Triscoli, Ackerley, & Sailer, 2014). Satiety also represents the end of a reward-related cycle incorporating the additional reward responsiveness constructs. Regarding consumption, food is sought out via environmental predictions (i.e., reward anticipation) and consumed (i.e., initial response to reward) until the hedonic value of consumption is depleted (i.e., satiety) (Morten L. Kringelbach, Stein, & Van Hartevelt, 2012). Importantly, satiation reflects an altered internal state that influences how a reward guides future behavior, including its ability to reinforce learning and the motivation to obtain that reward.

In humans, satiation is typically measured using self-reports of their internal state and feelings toward reward-related stimuli. In non-human primates, however, non-verbal paradigms train animals to associate specific food rewards with discrete stimuli, enabling researchers to assess reward preference as satiety occurs (Pastor-Bernier, Stasiak, & Schultz, 2021). Changes in satiety have been observed using neuroimaging, e.g., fMRI revealed those in a hungry state exhibited elevated amygdala, parahippocampal gyrus, anterior fusiform gyrus, OFC, and hypothalamus activity in response to food stimuli (Haase, Cerf-Ducastel, & Murphy, 2009; LaBar et al., 2001; Siep et al., 2009). Alternatively, a state of satiety is associated with increased blood oxygen level–dependent (BOLD) signals in the dorsolateral PFC (dlPFC), and a suppression of activity in reward processing regions, such as the vmPFC, NAc, hypothalamus, insula (Thomas et al., 2015), OFC, and amygdala (Gottfried, O'Doherty, & Dolan, 2003). The OFC is a key region for representing reward valuation (see above) and the association of pleasant or positive feelings towards sensory stimuli in humans (Morten L Kringelbach, 2005). In non-human primates, the OFC contains the secondary taste- and olfactory- cortices where local neurons are activated by the sight and smell of food when hungry, but decrease their firing during satiation (Critchley & Rolls, 1996; Rolls, Sienkiewicz, & Yaxley, 1989). Further, disrupting communication between the OFC and amygdala impairs the natural devaluation of a food reward typically induced by satiation (Baxter, Parker, Lindner, Izquierdo, & Murray, 2000). Thus, changes in satiety can be reflected in changes in neuronal activation.

Satiation is influenced by central and peripheral nervous system signaling molecules and is regulated by both homeostatic and hedonic processes. Interestingly, appetite increased in individuals using antihistamines with antiserotonergic properties (Stone, Wenger, Ludden, Stavorski, & Ross, 1961; Voigt & Fink, 2015). In contrast, serotonergic activation via selective serotonin reuptake inhibitors (SSRIs) or serotonergic agonists reduced food intake and are posited as anti-obesity compounds (Foltin, Haney, Comer, & Fischman, 1996; Hill & Blundell, 1990). Further, activation of serotonin (5-HT)_2C receptors reduced PFC and striatal activation in response to food stimuli, potentially indicative of reduced value representations (Thomas et al., 2018). Targeted hypothalamic 5-HT_2A/2B receptor activation reduced feeding duration while not altering the number of feeding bouts (Gruninger, LeBoeuf, Liu, & Rene Garcia, 2007; Shor-Posner, Grinker, Marinescu, Brown, & Leibowitz, 1986). In humans, 5-HT activation reduced food intake and consummatory rate, while dopaminergic activation also reduced feeding duration but increased feeding bouts and consummatory rate. Hence, dopaminergic modulation may alter the hedonic aspects of feeding while serotonin regulates satiation (Blundell & Latham, 1978; Halford & Harrold, 2012).

In rodents, satiety can be assessed in operant (including touchscreen)-based fixed-ratio (FR) responding paradigms, where animals are required to make a single response for a single food reward throughout the session (Phillips, Heath, Ossowska, Bussey, & Saksida, 2017). Importantly, trial block analyses enable assessment of response-rates across the session, to determine whether a specific manipulation is altering the hedonic value of a reward (responses during early blocks) compared to satiation (responses during later blocks). Although touchscreen FR paradigms remains to be validated, nutritional content modulates satiation rate during performance. Specifically, rewards with a high sugar content resulted in a more rapid decay in response-rate throughout a session (i.e. satiation) compared to a lower sugar content reward (Kim et al., 2017). Interestingly, higher caloric, but not sugar, content increased an animal’s responses to receive a reward (see progressive-ratio task in reward valuation). Thus, the FR paradigm enables identifying factors that modulate sensitivity to satiety that do not necessarily reflect changes in motivational state. In addition to the FR paradigm, most if not all touchscreen paradigms include ancillary measures that can be evaluated as indices of whether satiation may be influencing primary outcomes (e.g., number of trials/responses completed, latencies to make screen responses, and collect rewards). Serotonin depletion modulates various reward-related measures in touchscreen paradigms, e.g., both latency to respond and collect rewards (Alsiö et al., 2021; Desrochers & Nautiyal, 2022), which, may reflect an increase food drive and suppression of satiety. Alternatively, high doses of the serotonin antagonist lorcaserin reduced screen responding and slowed response and reward collection latencies, which coincided with reduced responding in a fixed-ratio operant task (P. J. Fletcher et al., 2023). Finally, global serotonin depletion facilitated the acquisition of autoshaping (described above), sped response latency, and maintained responding to a devalued target (Winstanley, Dalley, Theobald, & Robbins, 2004). Thus, some pharmacological and predictive validity exists for such behavioral measures and the validation of a fixed-ratio paradigm would further aid in identifying if abnormal satiety responses contribute to the behavioral profile of a given manipulation or animal model.

Category 2: Reward learning

Reward learning comprises the ability to form associations between environmental stimuli and rewarding outcomes, and update these associations in a dynamic environment (NIMH, 2023). Impaired reward learning is a common feature of several neuropsychiatric disorders, including schizophrenia (Lancaster et al., 2016), substance use disorders (Knabbe et al., 2022), major depressive disorder (Liverant, Arditte Hall, Wieman, Pineles, & Pizzagalli, 2021), post-traumatic stress disorder (B. H. Morris, Bylsma, Yaroslavsky, Kovacs, & Rottenberg, 2015), and anxiety disorders (Hein, de Fockert, & Ruiz, 2021). Importantly, the presentation and severity of reward learning deficits can differ both across and within diagnostic categories (Der-Avakian, Barnes, Markou, & Pizzagalli, 2016; Kirschner et al., 2024). Furthermore, the ability to use reinforcement feedback is strongly dependent on functions outlined in reward responsiveness, making it important to be able to detect potential impairments in functions such as hedonic processing, simple habit formation, processing unpredictable feedback, or general motivation. The RDoC framework subdomains of reward learning are probabilistic and reinforcement learning, reward prediction error, and habit formation.

Reinforcement learning

The most fundamental form of reward learning is operant (instrumental) conditioning, where subjects learn to make associations between a stimulus, a voluntary response, and a rewarding outcome. The ability to make these initial associations is necessary to perform any task that implements reinforcement for correct responses, regardless of whether those paradigms are assessing executive function, motivation, or memory. Given the ease of establishing a single response when feedback is accurate and consistent (deterministic learning) for humans, these types of paradigms are less commonly implemented than probabilistic learning tasks but are extensively used in animal research. Deterministic learning paradigms typically include a reversal component, where subjects are initially required to learn that a specific stimulus is rewarded (S+), while another is not (S-), which is followed by a reward contingency change. As a result, these paradigms can evaluate initial reward association learning and the ability to adjust behavior in response to a contingency reversal (Butter, 1969; Jones & Mishkin, 1972). While both initial association and contingency reversals require a subject to acquire a new association, the difference is reflected in behavioral deficits observed in clinical populations and the underlying neurobiology. Individuals with schizophrenia exhibit deficits in initial discrimination learning while simple contingency reversals are unaffected (Murray et al., 2008). Alternatively, individuals with depression exhibit a learning deficit following reversal, particularly following unexpected positive feedback (Robinson, Cools, Carlisi, Sahakian, & Drevets, 2012).

Establishing reward-based associations recruits corticostriatal circuits that enable previously irrelevant information to gain salience and are used to drive future behavior (Fuster, 2001; Miller & Cohen, 2001). Importantly, deconstructing the neurobiology underlying reinforcement learning encompasses various reward-related processes such as anticipation and reward prediction/prediction errors (discussed below) that are formed through learning. In humans, ventral striatal activation occurs during responses in reward in the early phases of learning, but then shifts to reward delivery predictive cues once associations are established (McClure, Berns, & Montague, 2003; J. P. O'Doherty, Dayan, Friston, Critchley, & Dolan, 2003), which is also observed in dynamic dopamine activity in non-human primates (Ljungberg, Apicella, & Schultz, 1992; Mirenowicz & Schultz, 1994). When contingencies are reversed, unexpected reward delivery is associated with increased striatal activity, illustrating a central role of striatal dopamine in updating associations (Robinson, Frank, Sahakian, & Cools, 2010). Moreover, this elevation in striatal activity during reversal learning was attenuated in depression, reflecting a possible insensitivity to rewarding feedback (Robinson, Cools, Carlisi, Sahakian, & Drevets, 2012). The differences in initial association and reversal learning are also reflected in the OFC, where damage resulted in deficits after a contingency reversal, while initial reward association was unaffected (Hornak et al., 2004; J. O'Doherty, Kringelbach, Rolls, Hornak, & Andrews, 2001). Human neuroimaging studies reveal that vmPFC activation correlates with the expected value of reward outcome and during reward receipt across learning (Daw, O'Doherty, Dayan, Seymour, & Dolan, 2006; Gläscher, Hampton, & O'Doherty, 2009; Liu, Hairston, Schrier, & Fan, 2011; Tanaka et al., 2016), together suggesting that while the vmPFC appeared to monitor and update reward value, its contributions to learning are necessary only when outcomes are unpredictable. Further, while vmPFC damage impaired reversal learning, damage to the dlPFC did not affect initial association nor contingency reversals (Fellows & Farah, 2003). PFC damage was associated with reduced reward sensitivity, which could underlie deficits in reward-related learning (Manohar & Husain, 2016). The neurochemical contributions to reward learning in humans in the context of probabilistic learning are discussed in more detail below, given its more abundant use in the field.

There has been extensive characterization of the underlying neurobiology of deterministic reversal learning using rodent touchscreen systems. As in the human reversal learning task, rodents learn that responding to one visual stimulus consistently yields a reward (S+) while a second stimulus does not (S-). Following the acquisition of this association, the contingencies are reversed. Dorsal striatum activity (e.g., c-fos imaging) was elevated by late-stage initial discrimination, decreased during early reversal, and increased with the new S+ acquisition. Further, in vivo single unit recordings demonstrated that during late reversal, dorsal striatum neurons were strongly suppressed following correct choices, followed by a sharp increase in activity at reward delivery. Ablation of the GluN2B NMDA receptor subunits from corticostriatal neurons also impaired initial discrimination (Brigman et al., 2013). Consistent with humans, disruption of the OFC impaired reversal, but not initial learning (Alsio et al., 2021; Chudasama & Robbins, 2003; Graybeal et al., 2011; Hervig et al., 2020; Izquierdo, Brigman, Radke, Rudebeck, & Holmes, 2017). During initial discrimination, OFC neurons increased activity on win-stay responses. Immediately after the contingency reversal however, OFC activity was significantly higher during perseverative responses, linked to increased c-fos expression during early reversal (Brigman et al., 2013) and returned to tracking win-stay responses after reversal was learned (Marquardt, Sigdel, & Brigman, 2017). These findings suggest that dynamic regulation of OFC activity is necessary for disengagement of the previous S+ stimulus and biasing responding towards the new S+ stimulus. Fluorescence in situ hybridization revealed that dorsolateral striatum (DLS) D₁R MSNs were significantly more active during early versus late discrimination learning. In contrast, optogenetic D₁R MSNs inhibition early in learning facilitated correct responses (and improved decision-making metrics), where the opposite effect was observed late in learning. Meanwhile, D₂R MSNs suppression reduced errors early in learning and increased correct choices later in learning (Bergstrom et al., 2018). Optogenetically silencing dorsolateral striatal neurons (i.e., via the CAG promoter) increased errors during reversal learning, demonstrating a dissociable role of this region during learning after a contingency reversal (Bergstrom, Lieberman, Graybeal, Lipkin, & Holmes, 2020). Phasic NAc dopamine release occurred after correct responses during initial discrimination, while elevated transient dopamine release was observed prior to reward delivery early in reversal (See reward anticipation and prediction error). Optogenetic inhibition of NAc D2R, but not D₁R, MSNs during the outcome period of error trials significantly impaired task performance during initial discrimination phases (Nishioka et al., 2023). Further, silencing VTA projections to the NAc core during early reversal, which corresponded with the temporal responsiveness of NAc dopamine, increased the number of errors (Radke, Zweifel, & Holmes, 2019). Hence, mesolimbic dopamine has been shown to facilitate reward learning across species, while OFC activity is consistently observed to track the value of stimuli associated with reward, and damage to this region lead to a rigidity in updating new value information. Thus, deterministic learning and reversal paradigms show established neurobiological validity across species.

Pharmacological predictive validity is seen in the reward learning task whereby global 5-HT depletion slowed both initial and reversal learning, consistent with human observations (Rogers et al., 1999). More specifically, mPFC serotonin depletion via 7-dihydroxytryptamine (DHT) impaired initial discrimination, while similarly targeted depletion of serotonin in the OFC specifically impaired reversal learning (Alsio et al., 2021). Reversal learning was facilitated by intra-OFC infusions of a 5-HT_2C-receptor antagonist (Alsio et al., 2015; Boulougouris & Robbins, 2010). Finally, amphetamine increased errors early in reversal in rats, and increased sensitivity to positive feedback as indexed by responses following correct, but not incorrect, outcomes (Stolyarova, O'Dell, Marshall, & Izquierdo, 2014). This finding has been confirmed in humans, with increased self-reported euphoria following amphetamine use correlating with increased neural activation following reward in a monetary reward task (Crane et al., 2018). Thus, the reward learning task has both neurobiological and pharmacological predictive validity across species.

Probabilistic Reinforcement learning

In contrast to deterministic learning, probabilistic learning paradigms introduce response-outcome uncertainty, which represents the uncertain nature of a real environment where a given stimulus does not always predict a consistent outcome. Probabilistic reinforcement learning is disrupted in multiple psychiatric conditions, including major depressive disorder (Klein et al., 1996), schizophrenia (Park et al., 2015), post-traumatic stress disorder (D. T. Acheson et al., 2022), and anxiety disorders (Taylor, Hoffman, & Khan, 2022). Probabilistic reinforcement learning paradigms offer an avenue for investigating whether abnormal reward learning is a function of deficits in processing unpredicted reward-related feedback.

The probabilistic reversal learning task (PRLT) is commonly used in humans and non-human animals, presenting two stimuli, one commonly (S+: 80% reward) and one uncommonly (S-: 20%) rewarded (Figure 3). Thus, subjects learn to optimize their responding to the S+. The reversal component implements a contingency shift of the S- to the S+, following a number of S+ responses (Cools, Clark, Owen, & Robbins, 2002). The PRLT provides metrics including the number of trials required for initial acquisition (reward learning), the number of reversals achieved (cognitive flexibility), win-stay (repeated response after reward) and lose-shift (shift choice after a non-reward).

Figure 3.

The probabilistic reversal learning task (PRLT) for humans (left) and its equivalent in rodents (right). The PRLT provides a cross-species paradigm where participants learn that one stimulus conveys a higher probability of yielding a reward than a concurrently presented less frequently rewarded stimulus. In addition to the initial learning phase, the PRLT contains a reversal component, where the previously attributed reward probabilities are switched between stimuli, allowing for prediction error and cognitive flexibility to be measured. The objective measures of the PRLT include learning which stimulus is the optimal choice to maximize rewards, total reversals, plus indices of sensitivity to feedback including the likelihood of changing responding following a reward (win-stay, win-shift) or the absence of reward (lose-stay, lose-shift) (i.e., the “punishment” included in the rodent task). Stimuli for the clinical task (left) taken from (Dahlia Mukherjee, Filipowicz, Vo, Satterthwaite, & Kable, 2020).

In the probabilistic reward task (PRT; or response-bias probabilistic reward task), participants learn to select an optimal (rich) over a nearly-indistinguishable suboptimal (lean) stimulus (Pizzagalli, Jahn, & O'Shea, 2005). Importantly, the participant does not always receive feedback when responding to a stimulus, with the optimal stimulus simply providing rewarding feedback more often (Figure 4). With such implicit (rather than explicit as above) feedback, healthy individuals develop a bias towards responding to the rich stimulus, despite no improvement in task accuracy (Pizzagalli, Goetz, Ostacher, Iosifescu, & Perlis, 2008; Pizzagalli et al., 2005). Importantly, the development of bias for the rich stimulus is attenuated in depression (Pizzagalli, Iosifescu, Hallett, Ratner, & Fava, 2008). Both probabilistic learning tasks and their respective validations have been recently reviewed in detail (Young, 2023), highlighting the difference of explicit feedback, provided in the PRLT, from the implicit feedback provided by the PRT.

Figure 4.

The probabilistic reversal task (PRT) for humans and its equivalent for rodents. The PRT is another cross-species paradigm for assessing probabilistic reward learning, and similarly requires subjects to respond to a “rich” stimulus that is more frequently reward then a “lean”, less frequently rewarded, concurrently presented stimulus. Similar to the PRLT, the PRT also measures the ability to develop an optimal response strategy (i.e., biasing responding to the rich stimulus), as well as assesses sensitivity to positive (win-stay, win-shift) and negative outcomes (lose-stay, lose-shift). An important component of the PRT is that subjects are not provided feedback on every trial, requiring them to implicitly learn the associations, which contrasts from the explicit learning employed in the PRLT.

Interestingly, people with depression (Esfand, Null, Duda, de Leeuw, & Pizzagalli, 2024; K. Fletcher et al., 2015; W. H. Liu et al., 2011; Pizzagalli, Iosifescu, et al., 2008; Vrieze et al., 2013) and BD (Pizzagalli, Goetz, et al., 2008) exhibit impaired PRLT and PRT performance, but people with schizophrenia only exhibit PRLT deficits (Culbreth, Westbrook, Daw, Botvinick, & Barch, 2016; Reddy, Waltz, Green, Wynn, & Horan, 2016; Waltz & Gold, 2007), not in PRT (Ahnallen et al., 2012; Barch et al., 2017; Heerey, Bell-Warren, & Gold, 2008) (See tables 4 and 5). Relevant to a transdiagnostic approach for PRLT use, people with depression or social anxiety disorder showed higher self-reported symptoms of anhedonia were associated with PRLT impairments relative to healthy participants, but diagnostic categories were not (E. E. Reilly et al., 2020). The PRLT is also sensitive to individuals diagnosed with other psychiatric disorders, such as depression, bipolar disorder, and obsessive-compulsive disorder (OCD) (Young, 2023). Hence, while anhedonia-related symptoms are seen in every group with psychiatric conditions using the PRLT, deficits in implicit reward-learning in the PRT were not.

Table 4.

Evidence for cross-species validation of the probabilistic reversal learning task (PRLT).

Reward Learning – Probabilistic Reinforcement Learning #1
Validity Domain	Human Task – Probabilistic Reversal Learning Task (PRLT)	Rodent Task – Probabilistic Reversal Learning Task (PRLT)
Predictive	Early-life stress exposure induces PRLT deficits (Wilkinson 2021)	Early-life stress exposure impairs PRLT performance in females more strongly than in males (Dutcher 2023, Zuhlsdorff 2023)
Neurobiological	Activation of anterior cingulate cortex Activation of orbitofrontal cortex Activation of ventral medial PFC Impairments associated with reduced striatal D2R expression (Jocham 2009)	Orbitofrontal cortex lesion impairs reversal learning (reviewed in Izquierdo 2018)
	Damage to PFC and temporal cortex impairs PRLT performance following reversal (Swainson 2000)	mPFC lesions do not affect PRLT performance (Birrell and Brown, 2000; Bissonette 2008; Floresco 2008; Churchwell 2009; Cordova 2014) mPFC lesion or stress may enhance PRLT performance (Salazar 2004; Graybeal 2011; Bryce and Howland 2015)
	Frontal midline delta band power associated with reward positivity (Cavanagh 2021)	Frontal midline delta band power associated with reward positivity (Cavanagh 2021)
Pharmacological	Citalopram impaired performance in PRLT (increased errors and response latency) (Chamberlain 2006)	Win-stay behaviors affected by antidepressants Reboxetine: reduced win-stay Venlafaxine: no effect Citalopram: increased win-stay (Wilkinson 2020)
		Ketamine and scopolamine reduced win-stay behaviors and number of reversals achieved (Wilkinson 2020)
	Impairments in rule learning following serotonin depletion; facilitation of PRLT learning by serotonergic agonism (Kanen 2020)	Reduced win-stay rate following 5-HT2C antagonism Increased win-stay rate following 5-HT2C agonism (Phillips 2018)
	Reduced D2R function impairs PRLT performance (Jocham 2009)	D2R antagonism impairs win-stay and lose-shift behavior (Alsio 2019)
		Initial learning improved by nicotine (Milienne-Petiot, 2018)
	Impairments in PRLT performance associated with methamphetamine usage (Robinson 2021)	Impairments in PRLT following methamphetamine treatment (Groman 2018)
Clinical sensitivity	Impairments in schizophrenia (Waltz and Gold, 2007; Culbreth 2016; Reddy 2016)

Table 5.

Evidence for cross-species validation of the probabilistic reward task (PRT)

Reward Learning – Probabilistic Reinforcement Learning #2
Validity Domain	Human Task – Probabilistic Reward Task (PRT)	Rodent Task – Probabilistic Reward Task (PRT)
Predictive	Blunted response in PRT is indicative of future anhedonia (Kangas 2022)	Early-life stress impairs PRT performance (Lamontagne 2021; Hisey 2023)
	Major depressive disorder impairs PRT performance (Hisey 2023)
Neurobiological	Improved PRT learning associated with decreased dopamine transporter binding potential (Kaiser 2018)	D2R agonism decreases response bias; D2R antagonism increases response bias (Der-Avakian 2013)
	Increased dopaminergic activity at reward delivery (Santesso 2009; Bogdan 2011)	Increased activity in anterior cingulate and NAc at reward delivery (Iturra-Mena, 2023)
Pharmacological	No effect of amphetamine on PRT performance (Soder 2021)	Amphetamine increases rich stimulus bias (Kangas 2020; Lamontagne 2021)
	Pramipexole inhibited reward learning (Santesso 2009)	Pramipexole reduces rich response bias (Lamontagne 2021)
		Scopolamine increases rich response bias Oxycodone does not affect rich response bias (Kangas 2020)
Clinical sensitivity	No impairments detected in people with schizophrenia (Heerey 2008; Ahnallen 2012; Barch 2017)

The use of both positive and negative feedback in the PRLT and PRT can dissociate abnormalities in positive and negative valence processing. For example, task deficits in individuals with schizophrenia appear to result from an inability to optimally use both positive feedback (win-stay) and negative feedback (lose-shift) (Reddy et al., 2016), while people with depression seem to be specifically impaired in using positive feedback (reduced win-stay with intact lose-shift) (D. Mukherjee, van Geen, & Kable, 2023) in the PRLT. This difference may stem from variance in the recruitment of underlying mechanisms to support distinct forms of learning. Striatal dopaminergic activity signals receipt of unexpected rewards, which occurs early in primary contingency learning or when contingencies shift. Reciprocal feedback loops with the frontal regions like the dlPFC and OFC facilitate the formation of value representations. In the PRLT, human fMRI revealed elevated activity of ACC, OFC, vmPFC and ventral striatum during final errors (last error before acquiring the new S+) (Cools et al., 2002; Dodds et al., 2008; Jocham et al., 2009). Interestingly, individuals with D2R polymorphisms that result in reduced striatal D₂R expression are impaired in acquiring the new S+ following reversal accompanied by a lack of ventral striatal and OFC activation following final errors (Jocham et al., 2009). Alternatively, unexpected negative feedback reduced OFC and amygdala activation, whereas individuals with bipolar disorder exhibit an attenuation in such reduced activity (Linke et al., 2012). Finally, PFC and temporal cortex damage impaired PRLT performance, especially following reversal (Swainson et al., 2000). While these circuits may underlie the integration and use of explicit feedback, the human neurobiology underlying the implicit learning used in PRT remains to be fully characterized. One study used PET imaging to demonstrate that individuals with lower dopamine transporter binding potential exhibited better PRT learning, accompanied by increased functional connectivity between the nAc and OFC (Kaiser et al., 2018). As is central to the RDoC framework, identifying distinct mechanisms underlying PRLT and PRT task performance may yield neural circuit candidates that distinguish reward-related dysfunction across psychiatric populations. Thus, differences in pathophysiology may manifest as impairments depending on the type of learning being taxed by the given task. Hence, both paradigms offer unique applications for targeting differences in dysfunctional reward learning.

Both the PRLT and PRT have been developed as touchscreen-based tasks for use in rodents. While non-touchscreen PRLT tasks have been extensively used and validated (Young, 2023), validation of the touchscreen-based PRLT, particularly neurobiological validity, is still underway. In a study of several commonly-used SSRIs, reboxetine reduced win-stay behaviors, venlafaxine had no effects, and citalopram increased win-stay behavior and facilitated rule learning in rats (Wilkinson, Grogan, Mellor, & Robinson, 2020). Both acute ketamine and scopolamine reduced win-stay behaviors and the number of reversals of rats (Wilkinson et al., 2020). Additionally, blockade and activation of 5-HT_2C receptors by the 5-HT_2CR antagonist SB 242084 reduced and increased win-stay behaviors, respectively (Phillips, Dewan, et al., 2018), which aligned with a facilitation of PRLT by serotonergic activation in human tasks (Kanen et al., 2020). Finally, D₂R blockade via the D₂R antagonist raclopride disrupted performance on a touchscreen PRLT variant, reducing both win-stay and lose-shift behaviors (Alsio et al., 2019), which is similar to reductions observed in humans with reduced D₂R functioning (Jocham et al., 2009). Additionally, exposure to early-life stress (ELS) (e.g., maternal separation) induced PRLT deficits with sex-specific response profiles (stronger in females) (Dutcher et al., 2023; Zuhlsdorff, Dalley, Robbins, & Morein-Zamir, 2023), which corresponds human PRLT deficits following ELS in humans (Wilkinson, Slaney, Mellor, & Robinson, 2021), providing predictive validity for the PRLT. Disrupting glutamate neurotransmission in early postnatal neurodevelopment (postnatal phencyclidine treatment), a common manipulation to approximate schizophrenia (Grayson et al., 2016), impaired performance in the PRLT (Tranter, Aggarwal, Young, Dillon, & Barnes, 2023). These deficits consisted of an impairment in win-stay responding and a reduction in the rate at which reward prediction errors (RPEs, see below) updated value estimates. Moreover, these impairments were reproduced in a follow-up study that also demonstrated that they may result from hyperactivation of glutamate neurons within the ventromedial OFC (Tranter et al., 2024). Hence, utility of the animal PRLT has enabled the investigation of mechanisms relevant to PRLT including potential mechanisms involved in deficient performance in patient populations.

Cross-species neurobiological validation has been shown for the PRT, as significant increases in ACC and NAc activity were observed in rats at reward delivery on either correct rich or lean responses, which was followed by sharp suppression of activity (Iturra-Mena, Kangas, Luc, Potter, & Pizzagalli, 2023), resembling ERPs observed during the human PRT measured via skin conductance before and after reward delivery (Bogdan, Santesso, Fagerness, Perlis, & Pizzagalli, 2011; Santesso et al., 2009). In the PRT, rich stimulus bias was increased by amphetamine in rats (Kangas, Wooldridge, Luc, Bergman, & Pizzagalli, 2020; Lamontagne, Melendez, & Olmstead, 2018), though it did not affect human PRT performance (Soder et al., 2021). Rich response bias was reduced by the D_2/3 receptor (D_2/3R) agonist pramipexole and glucocorticoid dexamethasone in rats (Lamontagne et al., 2018), and increased by scopolamine and unaffected by oxycodone in rats (Kangas et al., 2020), providing targets for validation in human studies. Further, the PRT similarly demonstrates predictive validity for ELS manipulations, with mice raised under ELS conditions showing impairments in PRT performance consistent with those seen in humans with MDD (Hisey et al., 2023). Finally, nicotine withdrawal affects performance on the PRT similarly in humans and in rats, with a significant reduction in reward responsiveness following 24 hours of withdrawal in both species (Pergadia et al., 2014). Hence, the PRT has some evidence for translational validity though more explicit testing is required.

Together, the touchscreen systems offer multiple paradigms reverse translated from human tasks to determine mechanisms underlying probabilistic reward learning. Future studies of the underlying circuitry will aid in dissecting the mechanisms underlying these different forms of reward learning and developing targeted treatments.

Reward prediction error

The reward learning subdomain of reward prediction error (RPE) concerns the mechanisms involved when an expected reward does not occur. RPE can be categorized as positive error, when an outcome is better than expected, or negative error, when an outcome is worse than expected. A fundamental mechanism by which RPEs are encoded involves changes in the activity of dopaminergic neurons (Wolfram Schultz, 2016; Watabe-Uchida, Eshel, & Uchida, 2017). When an outcome is better than expected, and a positive RPE is elicited, there is an increase in dopamine cell firing. By contrast, when an expected reward is omitted there is a brief pause in dopamine cell activity, encoding a negative RPE. In humans, microelectrode recordings in the substantia nigra were taken while subjects performed a probability learning task (Zaghloul et al., 2009). Unexpected rewards were associated with significantly greater neuron firing than unexpected losses, with no differences following expected outcomes. Dopaminergic signaling is propagated through regions involved in reward learning, including the striatum, amygdala, and frontal cortex, resulting in responsiveness to the RPE signal (McClure et al., 2003; Wolfram Schultz, 2016; Thut et al., 1997). Thus, dopaminergic activity reflects the difference between the expected and observed reward magnitudes. Potentials of dopaminergic neurons therefore can provide a useful biomarker to combine with cognitive and behavioral assessment for RPE.

Tasks such as the PRLT can be implemented to investigate RPE testing, as the shifts in reward contingencies still occur with an unexpected change in stimulus outcome. RDoC, however, highlights Bandit tasks as paradigms for testing RPE, as they assess a participant’s ability to respond to shifting reward contingencies. These tasks are similar in concept to probabilistic reinforcement tasks, but the changes in reward contingencies are not affected by a participant’s responses, instead changing independently over the course of a session (Daw et al., 2006). RPE outcomes are based on the striatum and VTA, as well regions in the vmPFC via dopaminergic projections when making reward-related decisions as shown by fMRI (Bartra, McGuire, & Kable, 2013; Daw et al., 2006; Hare, O'doherty, Camerer, Schultz, & Rangel, 2008) (Table 6). Bandit tasks utilize similar behavioral measures to probabilistic learning paradigms, and can assess distinct behavioral strategies underlying decision-making. Performance in bandit tasks is sensitive to the trade-off between exploiting high-valued choices and exploring alternative options to gain additional information. Optimal task performance requires these mutually exclusive strategies to be balanced, and extremes in either strategies can lead to maladaptive behaviors evident in mental illnesses (Addicott, Pearson, Sweitzer, Barack, & Platt, 2017). Win-stay/lose-shift behavior is improved by baseline anxiety state (Harle, Guo, Zhang, Paulus, & Yu, 2017) but impaired by methamphetamine dependence (Harle et al., 2015). Moreover, anhedonia was associated with an increase in exploration that reduced the preference for the most rewarded option (Harle et al., 2017). In people with schizophrenia, reduced win-stay behavior on probabilistic tasks was also associated with lower RPE signaling in the ventral striatum (Katthagen, Kaminski, Heinz, Buchert, & Schlagenhauf, 2020; Schlagenhauf et al., 2014) and reduced dlPFC activation by positive RPEs (Ermakova et al., 2018) compared to healthy controls. Additionally, a recent meta-analysis reported reduced activation in various brain regions associated with reward learning during RPE processing in schizophrenia, including the striatum, amygdala, thalamus and ACC (X. Yang, Song, Zou, Li, & Zeng, 2024). While additional studies are required to establish associations between predispositions to behavioral response to RPEs (e.g., more or less exploratory or exploitive) and the underlying neural responses, abnormalities in midbrain dopamine signaling and the corresponding RPE disruptions likely contribute to the deficits in probabilistic learning deficits in psychiatric disorders.

Table 6.

Evidence for cross-species validity of the tasks that assess reward prediction error (RPE).

Reward Learning – Reward Prediction Error
Validity Domain	Human Task – Bandit tasks	Rodent Task – Bandit tasks
Neurobiological		Sex differences in spatial restless bandit task – males spend more time exploring the environment (Chen et al 2021)
	Reward prediction error encoded by dopaminergic neurons (Schultz 2016)	Reward prediction error encoded by dopaminergic neurons (Schultz 2016)
Clinical sensitivity	People with schizophrenia showed impaired performance on a bandit task (Cathomas 2021)

Habit learning

Goal-directed behavior utilizes the learned associations between behavioral actions and their predicted outcomes enabling the flexible navigation of dynamic environments to achieve desirable outcomes (S. de Wit & Dickinson, 2009; Dickinson, 1985). When a given response is continuously associated with an outcome, that specific action becomes more dependent on the situation or stimulus by which it is elicited, rather than an action. The behavior then transitions from intentionally goal-directed to an unconscious habit, which generates optimal behavior at a lower cognitive cost (Balleine & Dickinson, 1998). Over time, formed habits can become rigid and inflexible, but can be shifted by focusing on goal-directed actions.

A fundamental challenge in studying habitual behavior is ensuring the distinction between habit and behaviors that implement goal-oriented strategies. Two paradigms, first described by (Dickinson, 1985), were developed to address these concerns, both removing the rewarding component of a behavioral response over time. The first is reward devaluation, in which the reinforcing component of the rewarding stimulus provided to the behavior is gradually reduced over time. Second is contingency degradation, where the association between a given behavior and a reward is altered, such as randomized schedule of delivering reward following the behavior response (reviewed in (Mendelsohn, 2019)). Similar to the latter is extinction, in which a reward is completely removed from the behavioral response. Essentially, a behavior can be considered habitual if it persists in the absence of the pre-existing reward conditions. Thus, cross-species habit tasks need an initial phase of procedural learning to establish stimulus-outcome associations, followed by a phase that substantially changes the response-reward association.

A commonly-used human paradigm is the habit learning task (Sjoerds et al., 2013), and presents participants with a stimulus (S) to which they respond (R) to achieve an outcome (O). The S-R-O flow utilizes goal-oriented behavior, as individuals provide a designated response to achieve a rewarding outcome. Once the S-R-O relationship is established, the flow is simplified to an S-R strategy (habitual response) (Figure 5). Cognitive tasks sensitive to changes in habit learning are designed around gauging an individual’s ability to shift between habitual control and goal-oriented behavior, as well as their ability to develop new habits.

Figure 5.

The habit learning task for humans (left) and the visuomotor conditional learning task (VMCL) for rodents (right). Habit learning tasks consists of two primary phases. The first requires goal-directed reward learning, where subjects learn to perform a behavior (e.g., which hand to move) in response to a stimulus (e.g., a picture of a fruit in the human task) that is associated with a specific outcome (e.g., the presentation of a paired picture for humans, or a food reward for rodents). The ability to form habits is then assessed by the removal of a goal-directed outcome, where the presentation of a stimulus now indistinctive elicits the habitual response. For rodent paradigms, such as the visuomotor conditional learning tasks (VMCL), animals are similarly trained to associate an image with a response to a square located on the right or left of the screen. Habit formation can be tested using a devaluation or extinction protocol, where the outcome of correctly responding to a stimulus (i.e., a reward) is removed or diluted, and researchers can test whether the habitual behavior (i.e., response to the stimulus) is maintained. Thus, habit learning paradigms allow researchers to tease apart goal-directed learning from habitual responding. Schematic for the habit learning task (left) adapted from (Sjoerds et al., 2013).

Abnormalities in habit learning can bias individuals towards increased habitual behaviors or disrupt flexibility. Individuals with substance abuse disorders appear to be more rigid in habitual drug-seeking behaviors, and are insensitive to devaluations in the rewarding properties of a given drug (Dickinson, Wood, & Smith, 2002). Similarly, pathological habit can manifest as compulsion (as in OCD), resulting from an abnormal strengthening of stimulus-response associations and an insensitivity to devaluation, resulting in deficits in balancing habitual and goal-directed behavior (Gillan et al., 2011; Maia, Cooney, & Peterson, 2008). Diseases with striatal degeneration such as Parkinson’s disease may have impaired procedural learning, preventing the formation of initial associations (Hay, Moscovitch, & Levine, 2002). Their deficits manifest as difficulty in shifting from S-R to R-O strategies, and in appropriately responding to devalued outcomes. This pattern indicates an impairment in goal-directed behavior in substance use disorders in combination with an overreliance on previously-developed habitual responses. People with alcohol dependence show difficulty shifting away from habitual control to goal-oriented behavior (Sjoerds et al., 2013), suggesting that the neural mechanisms underlying habit learning are negatively affected by long-term alcohol use. The striatum is engaged during habitual behavior (J. Li & Daw, 2011) (Table 7), with distinct recruitment of prefrontal and striatal activity during habit-formation tasks. Striatal activity was only associated with behavior at stimulus presentation, while PFC was active both when subjects performed an action and during response outcome. These differences may relate to habitual responding and goal-directed activity, respectively (McNamee, Liljeholm, Zika, & O'Doherty, 2015). A recent study demonstrated that while individuals with vmPFC lesions were able to readily establish a habitual association, they were unable to optimize responding once the reinforcer was devalued compared to healthy controls (Reber et al., 2017). The importance of the vmPFC was consistent where significantly increased vmPFC activity during goal-directed compared to habitual responding on an S-R-O task (S. de Wit, Corlett, Aitken, Dickinson, & Fletcher, 2009). Similarly, striatal activity strongly modulated the formation of initial associations, while vmPFC activity is necessary for updating previously established associations. Thus, habit formation can be assessed in people with clinically sensitive tasks, with targets for validation in rodent tasks.

Table 7.

Evidence for cross-species translation for tasks that measure habit formation.

Reward Learning – Habit Formation
Validity Domain	Human Task – Habit Learning Task	Rodent Task – Visuomotor Conditional Learning (VMCL)
Predictive	Substance-abuse disorder: ↓ sensitivity to devaluation OCD: Intact reward learning but ↓ sensitivity to devaluation (Gillan et al. 2011). Parkinson’s disease: ↓ ability to form procedural habits (Hay et al. 2002).
Neurobiological	↑ striatal activity during habitual responding (Li & Daw, 2011; Graybel & Grafton, 2015). ↓ vmPFC activity during habitual compared to goal-directed responding (de Wit et al. 2019). vmPFC lesions ↓ shifting from habitual to goal-directed responding (Reber et al. 2017).	↓ task acquisition following dorsal striatum lesion (Delotterie, 2015). ↓ task acquisition following posterior cingulate cortex lesion (Bussey 1997).

The touchscreen-based visuomotor conditional learning (VMCL) is a procedural learning paradigm (Horner et al., 2013) to assess habitual learning in rodents. In the VMCL task, the animal is presented with one of two easily-discriminable cue images in the center of the screen, followed by the presentation of two white squares on either side. They learn that the presentation of one image is associated with a reward after response at the right stimulus, while the other is associated with a reward after responses to the left (Figure 5). Similar to the human habit learning tasks, it is expected that animals will adopt an S-R habitual strategy over training. Reversal of the contingencies (the correct location associated with the initial cue stimulus) can be used to shift animals back to an R-O strategy. Posterior cingulate cortex lesions impaired mouse performance (Bussey, Everitt, & Robbins, 1997) while performance was independent of the ACC (Bussey, Muir, Everitt, & Robbins, 1997), perirhinal cortex or fornix (Bussey, Duck, Muir, & Aggleton, 2000), the prelimbic cortex, hippocampus (Delotterie et al., 2015), anterior or mediodorsal thalamus (Chudasama & Muir, 2001). Acquisition of the stimulus-association was ablated in animals with dorsal striatum lesions (Delotterie et al., 2015). Thus, striatal dependence in the absence of frontal cortex involvement aligns well with the neurobiological basis of habit formation in humans, providing neurobiological validity. To our knowledge, no publications have implemented a reward devaluation procedure for the VMCL task, which remains to be a critical component for establishing it as a habitual learning paradigm.

Devaluation procedures have been conducted in other touchscreen tasks, including assessments of effortful motivation (Stewart et al., 2022), visual discrimination, and simple stimulus responding (Favier et al., 2020). Impairing striatal cholinergic tone biased animals towards habit formation, as indicated by sustained responding following devaluation, while disrupting striatal glutamate release favored goal-directed behavior (Favier et al., 2020). Thus, optimizing the VMCL task with a devaluation procedure will aid in investigating the mechanisms underlying habitual behavior, providing insight into dissociable contributions to habit formation and goal-directed reward learning. Certainly, a great deal of validation efforts still need to be conducted to assess habit formation across species.

Category 3: Reward Valuation

The construct of reward valuation comprises three sub-constructs pertaining to the computation of a reinforcer’s value as a function of its magnitude and various devaluing factors surrounding its acquisition. Numerous laboratory assessments quantify the degree to which these factors—probability, delay to receipt, and requisite effort—impact reward value in various contexts and clinical populations. Most are decision-based, presenting a choice between rewards of different magnitudes at varying degrees of likelihood or cost. A subset of these assessments (called “discounting” tasks in reference to the devaluation of a reward by one of the above factors) can be readily administered to both humans and rodents. Other, similarly translatable assessments require participants to complete a single repetitive action for reward. Though traditionally administered to rodents using nosepoke- or lever-based operant boxes, these paradigms can be readily implemented in touchscreens. Furthermore, the versatility in stimulus delivery provided by touchscreens enables development of novel paradigms (e.g., Rearing-Effort Discounting task; (Lopez-Cruz et al., 2024)) that mitigate some of the confounds of earlier tasks.

Reward probability

Probabilistic discounting (PD) tasks present choices between small, certain rewards and larger, uncertain ones (Rachlin, Raineri, & Cross, 1991). Magnitudes and probabilities of larger rewards vary across trials and are signaled before choices are made. The large-reward options ultimately become disadvantageous at lower probabilities, and thus their selection decreases concurrently. The resultant “discounting curve” is altered in psychiatric populations (Bernhardt et al., 2017; Dai, Harrow, Song, Rucklidge, & Grace, 2016; Garami & Moustafa, 2019; K. L. Hart, Brown, Roffman, & Perlis, 2019; Schluter & Hodgins, 2021), and cigarette smokers (Reynolds, Richards, Horn, & Karraker, 2004) (but see: (Mitchell, 1999)), supporting the clinical sensitivity of this assay. Shallower discounting curves or overall reductions in PD (i.e., elevated preferences for large, unlikely rewards, indicating less devaluation by unfavorable probability) are often interpreted as impulsivity or risk insensitivity in clinical contexts (Bernhardt et al., 2017; Dai et al., 2016; Schluter & Hodgins, 2021). Conversely, reduced preference for probabilistic, large-reward options (i.e., enhanced PD) is typically interpreted as risk aversion, especially when these options have favorable odds (Garami & Moustafa, 2019; K. L. Hart et al., 2019).

Clinical findings implicate a network of brain regions in PD, including vmPFC, OFC, and ventral striatum (Miedl, Peters, & Büchel, 2012; Mok et al., 2021; Peters & Büchel, 2009; Peters & D'Esposito, 2020; Seaman et al., 2018; P. Wang et al., 2023) (Table 8). FMRI studies identified these structures as non-specific value encoders across several value-based decision making tasks (i.e., probabilistic, delay, and/or effort discounting, see below) (Mok et al., 2021; Peters & Büchel, 2009; Seaman et al., 2018); indeed, people with vmPFC and/or OFC damage exhibited reduced PD in a paradigm incorporating hypothetical rewards and no trial-by-trial feedback (Mok et al., 2021; Peters & D'Esposito, 2020). Meanwhile, trend-level PD deficits in people with gambling disorder were accompanied by diminished positive correlations between subjective value (advantageousness of the probabilistic option) and OFC and ventral striatum activation (Miedl et al., 2012). Pharmacological sensitivity is less supported, however, as the few studies reporting such manipulations describe limited effects. Namely, alcohol dose- and time point-dependently reduced PD in healthy adults (Bidwell et al., 2013) (but not adolescent males at the group level (Bernhardt et al., 2019)), while acute benzodiazepine (A. Acheson, Reynolds, Richards, & de Wit, 2006), opioid (Zacny & de Wit, 2009), cannabinoid (McDonald, Schleifer, Richards, & de Wit, 2003), and psychostimulant (A. Acheson & de Wit, 2008) administration failed to affect this behavior.

Table 8.

Evidence for cross-species validity of tasks that assess probability-based reward valuation.

Reward Valuation – Reward Probability
Validity Domain	Human Task – Probabilistic Reversal Learning Task (PRLT)	Rodent Task – Probabilistic Reversal Learning Task (PRLT)
Predictive	Early-life stress exposure induces PRLT deficits (Wilkinson 2021)	Early-life stress exposure impairs PRLT performance in females more strongly than in males (Dutcher 2023, Zuhlsdorff 2023)
Neurobiological	Activation of anterior cingulate cortex Activation of orbitofrontal cortex Activation of ventral medial PFC Impairments associated with reduced striatal D2R expression (Jocham 2009)	Orbitofrontal cortex lesion impairs reversal learning (reviewed in Izquierdo 2018)
	Damage to PFC and temporal cortex impairs PRLT performance following reversal (Swainson 2000)	mPFC lesions do not affect PRLT performance (Birrell and Brown, 2000; Bissonette 2008; Floresco 2008; Churchwell 2009; Cordova 2014) mPFC lesion or stress may enhance PRLT performance (Salazar 2004; Graybeal 2011; Bryce and Howland 2015)
	Frontal midline delta band power associated with reward positivity (Cavanagh 2021)	Frontal midline delta band power associated with reward positivity (Cavanagh 2021)
Pharmacological	Citalopram impaired performance in PRLT (increased errors and response latency) (Chamberlain 2006)	Win-stay behaviors affected by antidepressants Reboxetine: reduced win-stay Venlafaxine: no effect Citalopram: increased win-stay (Wilkinson 2020)

Rodent PD tasks operate on similar contingencies as the human task and are traditionally administered in lever- (Cardinal & Howes, 2005) and nosepoke-based (A. Acheson, Farrar, et al., 2006) operant systems. Touchscreen PD programs exist, although they are mostly similar to earlier operant versions and are rarely used (Abela & Chudasama, 2013) (Rojas, Curry-Pochy, Chen, Heller, & Grissom, 2022). Consistent with human assessments, rodents repeatedly choose between one option that always yields a small reward and another option that yields a large, probabilistic reward. Whereas probabilities of the large-reward option are modulated from trial to trial in human tasks, rodent testing sessions are typically organized block-wise, with probabilities held static for several consecutive trials (i.e., trial blocks). Reward likelihood of the probabilistic option steadily increases or decreases across blocks (e.g., (Cardinal & Howes, 2005)). These procedural differences must be considered when comparing findings across species (see below).

Neurobiological validity of rodent PD tasks is generally supported, with rodent OFC and ventral striatum playing roles in subjective value calculation consistent with clinical data. Inactivation of medial OFC (but not OFC in general; (St Onge & Floresco, 2010)) reduced rat PD (Stopper, Green, & Floresco, 2014), consistent with observations from human studies (Peters & D'Esposito, 2020) (Table 8). Interestingly, however, the only extant touchscreen-based study incorporating neurological manipulations reported increased PD in complete OFC-lesioned rats, albeit in a rarely used paradigm that changed probabilistic contingencies between sessions (Abela & Chudasama, 2013). Importantly, these rats were also not trained on the task until after surgery, raising the potential confound of non-specific learning effects of the lesion. The inconsistency between touchscreen PD data and those of clinical and lever-based paradigms may therefore have arisen from procedural deviations rather than the touchscreen system itself. Meanwhile, in a non-touchscreen-based paradigm, inactivation of the NAc generally reduced choice of the probabilistic option during trial blocks in which that option was advantageous or neutral (i.e., had high subjective value) (Stopper & Floresco, 2011). There is therefore some neurobiological validity between human and rodent PD tasks in general, although further validation is necessary for touchscreen-based paradigms. The effects of mPFC manipulations are less straightforward, however. Specifically, mPFC inactivation exerted opposite effects on discounting of the large reward depending on whether its probability increased or decreased across blocks (St Onge & Floresco, 2010). These data indicate a role for mPFC in tracking contingency changes, but do not align with outcomes of clinical tasks that modulate probabilities between individual trials. While functional differences between rodent and human cortical subregions should be noted when extrapolating findings across species (Seamans, Lapish, & Durstewitz, 2008; Uylings, Groenewegen, & Kolb, 2003), such parameter effects indicate deeper issues with the translatability of the task itself.

Pharmacological predictive validity of rodent PD tasks is partially supported by findings that acute ethanol reduced discounting in male rats but not females (Wallin-Miller, Chesley, Castrillon, & Wood, 2017), despite no gender-specific effects in human PD (Bidwell et al., 2013). Meanwhile, another study detected no significant effects of acute ethanol on PD in intact adult male rats, although intermittent adolescent ethanol exposure reduced PD in adulthood (Boutros, Semenova, Liu, Crews, & Markou, 2014). Finally, acute and chronic amphetamine consistently reduced PD in rats (Floresco & Whelan, 2009; Islas-Preciado et al., 2020; Ozga-Hess & Anderson, 2019; St Onge, Chiu, & Floresco, 2010; St Onge & Floresco, 2009; Yates et al., 2020) but not humans (A. Acheson & de Wit, 2008), further challenging the assay’s pharmacological predictive validity.

Inconsistencies between rodent and human findings may be partially explained by differences in testing procedure. For example, rodents must be well-trained in the PD task, exhibiting stable baseline performance prior to assessment, whereas clinical tests are administered in single sessions. By the time rodents reach stability, they have consolidated the reward contingencies (becoming habitual, see above) and can likely predict probability changes (St Onge & Floresco, 2010), thus raising the issue of differentiating actual decision-making from habitual response patterns. This complication arises from the block-wise structure of the preclinical task. When combined with daily training, this predictability of contingency shifts enables animals to refine performance strategies across several sessions, possibly to the detriment of the task’s predictive validity. Indeed, the discounting curve is less steep when the large-reward lever’s probability does not consistently ascend or descend across blocks, but instead undergoes a “mixed” block-wise progression (i.e., 100%, 12.5%, 25%, 50%; (St Onge et al., 2010)). Rat PD is therefore facilitated when the change in this lever’s utility is unidirectional and easier to learn across training sessions. An additional concern is “anchoring,” whereby response patterns are unduly influenced by the original contingency of the probabilistic lever. Anchoring is particularly persistent in mouse PD, evident even after months of training in a touchscreen-based task (Rojas et al., 2022). Thus, procedural differences across species may underlie the lack of consistent neurobiological and pharmacological validation.

Additional disparities in pharmacological effects across human and rodent PD may arise partly from the specifications of clinical assessments. The task utilized in the human amphetamine study (as well as the benzodiazepine, opioid, and cannabinoid studies; see above) presented PD trials interleaved with delay discounting trials (i.e., choice between small, immediate versus larger, delayed rewards), with no trial-by-trial feedback provided (i.e., reward versus no reward) until all trials had been completed (Richards, Zhang, Mitchell, & de Wit, 1999). While it is unclear whether the juxtaposition of probability and delay discounting trials would affect participants’ behavior in either condition, this design cannot be administered to rodents as it is unlikely that they can be trained to alternate between these trial types within a single session. The absence of immediate reinforcement in the clinical task is similarly problematic. Furthermore, rodent paradigms can rarely defer reward delivery until the end of the session since consistent reinforcement is necessary to maintain responding. Deferred reinforcement in clinical testing therefore removes the construct of reward processing that is central to animal assessments, whereas the effects of certain drugs on animal PD may be confounded by their concurrent effects on reward processing (especially amphetamine; (Leith & Barrett, 1976)). Finally, pharmacological manipulation of rodent PD is also confounded by the block-wise structure of the task. Similar to mPFC inactivation (St Onge & Floresco, 2010), amphetamine exerted opposite effects on PD depending on whether the odds for the large-reward lever ascended or descended across the session (with no effect at all in a “mixed progression” PD task) (St Onge et al., 2010). Hence, task parameters clearly impact outcomes as rodents are tested in traditional operant chambers.

The versatility in stimulus presentation offered by touchscreens may provide an opportunity to mitigate some of these shortcomings. For example, given that earlier operant systems could only deliver choice stimuli that were qualitatively identical, any within-session contingency modulation had to be signaled via several “forced-choice” trials preceding blocks of identical “free-choice” trials (Cardinal & Howes, 2005). Touchscreen systems may enable rodents to be trained to associate individual, visually distinct stimuli with specific rewards and probabilities that are presentable successively alongside other stimuli in a single, non-block-arranged PD assessment. The ability to modulate reward contingencies across individual trials similarly to clinical assessments would circumvent the confounds associated with block-wise task structures.

Delayed reward

Delay discounting (DD) tasks quantify the degree to which the subjective value of a reward is reduced by the time required to wait for its receipt. Similar to PD tasks, subjects are presented with options yielding rewards of different magnitudes, with the devaluation of the larger reward resulting from its delayed delivery. Human DD tasks (Scheres, de Water, & Mies, 2013) are typically hypothetical in nature, offering participants the choice between immediate versus delayed sums of imaginary money. Certain variants do offer actual money (or other reinforcers), meanwhile, honoring the conditions of every trial (defined as “real” or “experiential” tasks (Reynolds & Schiffbauer, 2004)) or of one or more random trials (defined as “potentially real” tasks (M. W. Johnson & Bickel, 2002); c.f. “hypothetical” tasks, in which all rewards are imaginary) (Figure 6). Interestingly, humans perform questionnaire-based DD tasks similarly across “potentially real” and hypothetical reward conditions (M. W. Johnson & Bickel, 2002). Meanwhile, operant choice procedures providing real-time reinforcement (i.e., seconds or minutes after each decision; (M. W. Johnson, 2012) produce faster rates of DD in humans than questionnaire-based assessments utilizing longer time-scales (i.e., days or months after completion of the session) (Navarick, 2004). Thus, trial-by-trial reinforcement influences human performance of these tasks.

Figure 6.

The delay discounting (DD) task for humans (left) and the touchscreen equivalent developed for rodents (right). DD tasks assess whether subjects would rather wait to obtain a large reward than receive a small reward immediately, providing measures of impulsivity and the degree to which a reward is devalued by the interval to its delivery. Human subjects are provided with verbal prompts while rodents must learn reward contingencies over several training sessions. Performance is measured either by: (1) overall percentage of choices of the delayed versus immediate reward; or (2) a discounting curve illustrating the step-wise reduction in large-reward choices as the delay to its receipt increases. The clinical task depicted here (developed by Johnson, 2012; left) is a rare example of an operant choice DD task utilizing experiential rewards. While this design optimizes cross-species translatability of findings, most clinical studies of DD utilize questionnaire-based assessments and hypothetical or “potentially real” rewards.

DD is commonly interpreted as a measure of impulsivity and often assessed alongside PD in clinical populations with impaired impulse control. Elevated DD (preference for small, immediate rewards) is observed across several such disorders (Jackson & MacKillop, 2016) (Ahn et al., 2011) (Petry, 2001) (Heerey, Robinson, McMahon, & Gold, 2007) (Pennisi et al., 2023) (Amlung, Vedelago, Acker, Balodis, & MacKillop, 2017; Phung, Snider, Tegge, & Bickel, 2019) (Table 9). Neurobiological study of human DD emphasizes cortical and striatal areas, with hippocampal and insular contributions. Consistent with the previously discussed roles of vmPFC, OFC, and ventral striatum in encoding value during decision making (Mok et al., 2021; Peters & Büchel, 2009; Seaman et al., 2018), people with vmPFC and/or OFC lesions and consequent PD deficits were also impaired in DD (hypothetical and “potentially real” paradigms, no trial-by-trial feedback; (Mok et al., 2021) (Peters & D'Esposito, 2020)). Transcranial direct current stimulation over the OFC or dlPFC significantly modulated choice behavior for hypothetical monetary gains and losses during DD (He et al., 2016; Hecht, Walsh, & Lavidor, 2013; Moro et al., 2023; Shen et al., 2016; Xiong et al., 2019). Ventral striatal dopamine D_2/3R binding is also implicated in DD, as demonstrated by an inverse relationship between DD and [11C]raclopride binding potential in people with gambling disorder (questionnaire, hypothetical rewards) (Joutsa et al., 2015). Meanwhile, people with insula damage demonstrated reduced DD for hypothetical monetary rewards (i.e., elevated preference for larger, delayed rewards) (Sellitto, Ciaramelli, Mattioli, & di Pellegrino, 2015). Finally, the hippocampus may also contribute to DD, potentially via simulating future outcomes and thereby informing subjective value signaling in prefrontal areas (Peters & Büchel, 2010). This theory arises from evidence that individuals with hippocampal and associated temporal lobe damage are only impaired in DD while: (1) actively simulating hypothetical monetary rewards (engaging in “episodic future thinking,” or visualizing themselves spending imaginary money; (Kwan et al., 2012; Kwan, Craver, Green, Myerson, & Rosenbaum, 2013; Palombo, Keane, & Verfaellie, 2015; Patt, Hunsberger, Jones, & Verfaellie, 2023)); or (2) playing for non-monetary experiential rewards (Patt et al., 2023). Therefore, while overall choice behavior may not appreciably differ between hypothetical and real-reward DD paradigms in healthy populations (M. W. Johnson & Bickel, 2002), additional brain regions and neural processes are nevertheless active when outcomes are projected and/or expected during decision-making. This distinction must therefore be considered when assessing cross-species disparities in the effects of experimental manipulations.

Table 9.

Evidence for cross-species validity of tasks that assess delay-based reward valuation.

Reward Valuation – Reward Delay
Validity Domain	Human Task – Delay Discounting (DD)	Rodent Task – Delay Discounting (DD)
Predictive	↑ DD in people with ADHD (meta-analysis) (Jackson and MacKillop, 2016).	↑ DD in spontaneously hypertensive rats (Wooters and Bardo, 2011). ↑ DD in spontaneously hypertensive rats (Fox et al., 2023). ↑ DD in frontal dopamine-depleted rats; Freund et al., 2014
	↑ DD in people with more severe alcohol use disorder than those with less severe alcohol use disorder (Phung et al., 2019). ↑ DD across alcohol, tobacco, cannabis, stimulant, opiate, and gambling addictions (meta-analysis) (Amlung et al., 2017)	↑ DD in rats bred for high alcohol drinking (Wilhelm and Mitchell, 2008). ↑ DD in cocaine-self-administered rats (Mendez et al., 2010).
Neurobiological	OFC damage ↑ DD (Peters et al., 2020). Transcranial direct current stimulation over OFC ↓ DD (Moro et al., 2023).	Medial and lateral OFC lesions respectively ↓ and ↑ DD (Mar et al., 2011).
	Bi-frontal direct current stimulation (left dlPFC facilitated and right dlPFC inhibited) ↓ DD (Hecht et al., 2013). Anodal stimulation left dlPFC ↓ DD (He et al., 2016). Anodal stimulation of left dlPFC ↓ DD, cathodal stimulation left dlPFC ↑ DD (no effect of either on right dlPFC) (Shen et al., 2016). vmPFC damage ↑ DD (Mok et al., 2021).	mPFC inactivation ↑ DD (Churchwell et al., 2009).
	Inverse relationship between DD and ventral striatal [11C]raclopride binding potential in people with gambling disorder (Joutsa et al., 2015).	NAc D2/3R availability inversely correlated with DD (Barlow et al., 2018).
	Insula damage ↓ DD (Sellitto et al., 2015)	Intra-insula D1R blockade ↑ DD (Pattij et al., 2014).
	Medial temporal lobe damage did not affect DD (Kwan et al., 2012). Medial temporal lobe damage did not affect DD (Kwan et al., 2013). Medial temporal lobe damage ↑ DD specifically during concomitant episodic future thinking (Palombo et al., 2015). Medial temporal lobe damage ↓ DD in experiential but not hypothetical assessments (Patt et al., 2023).	Hippocampus lesion ↑ DD (Cheung and Cardinal, 2005). Ventral hippocampus lesion ↑ DD (Abela and Chudasama, 2013).
Pharmacological	Amphetamine (10 and 20 mg, acute) ↓ DD (de Wit et al., 2002).	Amphetamine (1–2.3 mg/kg, i.p., acute) ↓ DD (Winstanley et al., 2003). Amphetamine (0.25 mg/kg, i.p., acute) ↓ DD (Floresco et al., 2007). Amphetamine respectively ↑ and ↓ DD in cued (1.0 and 1.6 mg/kg, i.p., acute) and un-cued paradigms (0.3 mg/kg, i.p., acute) (Cardinal et al., 2000). Amphetamine (1.0 and 1.7 mg/kg, i.p., acute) ↑ DD (Slezak and Anderson, 2009). Amphetamine (0.80 and 1.20 mg/kg, i.p., acute) ↑ DD (adjusting amount procedure) (Helms et al., 2006).
	Haloperidol (2 mg, acute) ↓ DD (Wagner et al., 2020). Haloperidol (1.5 mg, acute) did not affect DD (Pine et al., 2010).	Haloperidol (.01–0.1 mg/kg, i.p., acute) ↑ DD (Boomhower and Rasmussen, 2014).
	Acute testosterone gel (150 mg) ↑ DD in healthy men (Wu et al., 2020). Acute testosterone gel (50 mg) did not affect DD in healthy men (Ortner et al., 2013).	Chronic testosterone (7.5 mg/kg, s.c., 15 weeks) ↓ DD in male rats (Wood et al., 2013).
	Diazepam (20 mg, acute) did not affect DD (Acheson et al., 2006).	Diazepam (1–10 mg/kg, i.p., acute) dose- and strain-dependently modulated DD (Huskinson and Anderson, 2012).

Clinical DD tasks are somewhat pharmacologically sensitive, although effects are not always reproducible. For example, d-amphetamine reduced DD in healthy humans (H. de Wit, Enggasser, & Richards, 2002) (but not in a paradigm presenting interleaved DD and PD trials (A. Acheson & de Wit, 2008)), as did the dopamine D₂R antagonist haloperidol (at 2 mg (Wagner, Clos, Sommer, & Peters, 2020), but not 1.5 mg (Pine, Shiner, Seymour, & Dolan, 2010)). These findings may indicate a U-shaped relationship between dopamine tone and delay-based reward valuation in humans whereby both increases and decreases in dopamine transmission increase tolerance to delayed delivery of large rewards. Additionally, acute application of testosterone gel elevated DD in healthy men (at 150 (Wu et al., 2020) but not 50 mg (Ortner et al., 2013)), with no effects of acute diazepam (A. Acheson, Reynolds, et al., 2006), oxycodone (Zacny & de Wit, 2009), THC (McDonald et al., 2003), or bupropion (A. Acheson & de Wit, 2008) on DD trials interspersed with PD trials. Thus, few extant studies describe drug effects on DD (and even fewer do so separately from concurrent PD assessment), rendering the pharmacological sensitivity of these assays inconclusive.

Rodent DD procedures are largely identical to those of preclinical PD tasks and exclusively operate on “experiential,” or real-reward contingencies, versus the hypothetical and “potentially real” designs common in clinical study. Touchscreen DD paradigms exist (Abela & Chudasama, 2013) (Figure 6), although lever- and nosepoke-based assessments are more commonly used and are largely identical in terms of procedure. As with PD paradigms, rodent DD tasks typically proceed block-wise, with delays associated with the larger reward progressively increasing or decreasing across testing sessions. Hence, rodents must be well-trained on the DD task prior to assessment, producing similar limitations as those described above for PD tasks (e.g., habitual responding; (Slezak & Anderson, 2009)). Consistently, certain pharmacological effects are dependent upon order of delay presentation (i.e., ascending versus descending; (Tanno, Maguire, Henson, & France, 2014)), although anchoring of response patterns based on original task contingencies is less persistent in mouse DD than PD (Rojas et al., 2022). Alternatively, rodents may be tested in a less common “adjusting amount” procedure, which: (1) adaptively changes magnitude of the immediate reinforcer in response to animals’ last choice and; (2) eschews block-wise organization by instead modulating delay duration across several testing days (Richards, Mitchell, de Wit, & Seiden, 1997).

Increased DD was detected in the spontaneously hypertensive (Fox et al., 2023; Wooters & Bardo, 2011) and frontal dopamine depletion (Freund et al., 2014) rat models of ADHD, as well as in rat models of stimulant and alcohol use (Mendez et al., 2010; Wilhelm & Mitchell, 2008), supporting the predictive validity of rodent DD tasks (Table 9). Neurobiological validity of rodent DD tasks is also supported, although little work has been done using touchscreens. While inactivating rodent OFC in general produced mixed results on DD performance (Abela & Chudasama, 2013; Stopper et al., 2014; Winstanley, Theobald, Cardinal, & Robbins, 2004), lesions of discrete medial versus lateral subregions respectively increased and decreased DD (Mar, Walker, Theobald, Eagle, & Robbins, 2011). Meanwhile, inactivating rodent mPFC (Seamans et al., 2008; Uylings et al., 2003) increased choice of the small, immediate reward (Churchwell, Morris, Heurtelou, & Kesner, 2009) (but see: (Cardinal, Pennicott, Sugathapala, Robbins, & Everitt, 2001)), consistent with dlPFC involvement in human DD (see above). Ventral striatal dopamine D_2/3R binding is similarly involved across clinical and rodent DD tasks, with accumbal D_2/3R -availability inversely correlated with DD in rats (Barlow et al., 2018). Furthermore, while lesion/inactivation studies targeting insular regions have yet to be conducted on rodents, intra-insular dopamine D₁R blockade increased DD in rats (Pattij, Schetters, & Schoffelmeer, 2014), demonstrating this structure’s involvement in DD across species. Contrary to clinical findings however, hippocampal lesions increased DD in rats (Abela & Chudasama, 2013; Cheung & Cardinal, 2005) (including in a touchscreen paradigm; (Abela & Chudasama, 2013)), although both the specificity of the respective lesions in human versus rodent studies as well as the contingency of clinical findings on such factors as “episodic future thinking” must be considered when comparing findings across species.

Pharmacological validity of preclinical DD tasks is difficult to determine due to the relative scarcity of human pharmacological studies. Nevertheless, inter-species pharmacological effects on DD studies tend to be inconsistent. For example, while the reduction of human DD by amphetamine (H. de Wit et al., 2002) has been reproduced in rats (Floresco, Tse, & Ghods-Sharifi, 2008; Winstanley, Dalley, Theobald, & Robbins, 2003), the opposite effect was reported in rats and mice tested in less common paradigms utilizing cued delays and adjusting amount procedures (Cardinal, Robbins, & Everitt, 2000; Helms, Reeves, & Mitchell, 2006; Slezak & Anderson, 2009). Cross-species disparity is also evident from other studies. For example, DD was increased by haloperidol in rats (Boomhower & Rasmussen, 2014) but decreased (or unaffected; (Pine et al., 2010)) in humans (Wagner et al., 2020). Meanwhile, chronic testosterone treatment increased preference for larger, delayed rewards in male rats (Wood et al., 2013), contrary to the effects of acute testosterone in men (Wu et al., 2020), while diazepam dose- and strain-dependently modulated DD in rats (Huskinson & Anderson, 2012) but did not affect humans (concurrent DD/PD assessment; (A. Acheson, Reynolds, et al., 2006)). Thus, pharmacological predictive validity of preclinical DD tasks is currently lacking.

The question of reinforcement complicates comparison of DD findings across species. A relatively small portion of extant human DD literature utilizes explicit (i.e., non-hypothetical) reinforcers, and of these studies, only a small subset deliver reinforcement following individual trials within the session itself (Reynolds & Schiffbauer, 2004), versus hypothetical days or months. As with certain PD variants discussed above, the majority of human DD paradigms thereby remove the component of reward receipt/consumption from the assessment, a fundamental component of preclinical operant paradigms that can also influence human performance of non-monetary DD tasks (Navarick, 2004). While touchscreen operant systems may conceivably enable eschewal of block-wise session structures in discounting tasks and thereby prevent perseverative responding (see “Reward Probability”), they are unlikely to facilitate replacement of real-time reinforcement. Cross-species consistency may therefore be best enhanced by simply standardizing testing procedures within clinical and preclinical realms, ultimately facilitating translation by reducing the number of task variants.

Effort for reward

Progressive ratio breakpoint

Progressive ratio breakpoint tasks (PRBTs) require participants to complete a gradually increasing number of effortful actions (cognitive or physical) in exchange for a fixed reward (Strauss et al., 2016; Wolf et al., 2014). The task ends when the participant determines that the requisite effort exceeds the value of the reward and stops responding, i.e., reaches their “breakpoint”. While the PRBT has long-been used in animal research to study effortful motivation (Hodos, 1961) and characterize the reinforcing properties of drugs (Brady & Griffiths, 1976), these paradigms have only recently begun to gain footing in clinical studies. Task procedures are highly consistent across species, the primary difference being the modality of response (Figure 7). Moreover, all clinical PRBTs incorporate some form of response feedback (with many delivering actual rewards), thereby addressing limitations pertaining to reward processing discussed for PD and DD tasks.

Figure 7.

The progressive ratio breakpoint task (PRBT) for humans (left) and the operant and touchscreen-based equivalents for rodents (middle and right respectively). The PRBT is a cross-species paradigm that measures the degree to which a subject will expend physical effort to achieve a reward. In humans, effort is typically quantified by joystick rotations or button presses, while rodents are required to nosepoke into an aperture (middle) or on a touchscreen (right). The number of responses required to earn a reward is progressively increased across the session. The primary outcome variable is the point at which the subject stops responding, i.e., the “breakpoint”.

The PRBT demonstrates both clinical and preclinical sensitivity. Reduced breakpoints are reliably detected in people with schizophrenia (Strauss et al., 2016; Wolf et al., 2014) and unipolar or bipolar depression (Hershenberg et al., 2016), which have been recreated in animal models (Tam et al., 2010) (Dieterich, Liu, & Samuels, 2021; Dieterich et al., 2019; Picard et al., 2021) (Young et al., 2015) (Table 10). Meanwhile, a touchscreen paradigm detected reduced breakpoints in a mouse model of Huntington’s disease, consistent with clinical findings (Heath et al., 2019). Few clinical studies to date have collected neurophysiological measures alongside PRBT assessment, although some convergent findings across species support neurobiological validity of the rodent tasks. The ventral striatum is implicated in both human and rodent performance, with lower activation correlating with lower breakpoints in people with schizophrenia (Wolf et al., 2014) and NAc inactivation reducing breakpoints in rats (Moscarello, Ben-Shahar, & Ettenberg, 2010). Increased parietal alpha power was also observed in humans and mice as they neared their breakpoints (Cavanagh et al., 2021; Noback et al., 2024), constituting a common electrophysiological biomarker of performance across species. Finally, pharmacological predictive validity is supported whereby human and rodent breakpoints were similarly elevated by acute amphetamine (Bensadoun, Brooks, & Dunnett, 2004; Hailwood et al., 2018; Heath et al., 2015; Noback et al., 2024).

Table 10.

Evidence for cross-species validity of tasks that assess effort-based reward valuation – Progressive ratio breakpoint.

Reward Valuation – Effort #1
Validity Domain	Human Task – Progressive Ratio Breakpoint (PRBT)	Rodent Task – Progressive Ratio Breakpoint (PRBT)
Predictive	↓ breakpoints in people with schizophrenia (Wolf et al., 2014). ↓ breakpoints in people with schizophrenia (Strauss et al., 2016).	↓ breakpoints in Sp4 hypomorphic mice (trend-level) (Young et al., 2015).
	↓ breakpoints in people with unipolar or bipolar depression (Hershenberg et al., 2016).	↓ breakpoints in mice following chronic corticosterone (Dieterich et al., 2019). ↓ breakpoints in mice following social defeat stress (Dietrich and Samuels, 2021). ↓ breakpoints in mice following chronic unpredictable mild stress (Picard et al., 2021).
	↓ breakpoints in people with Huntington’s disease (Heath et al., 2019).	↓ breakpoints in R6/1 mice (Heath et al., 2019).
Neurobiological	Lower ventral striatal activation correlates with lower breakpoints in people with schizophrenia (Wolf et al., 2014).	NAc inactivation ↓ breakpoint (Moscarello et al., 2010).
	↑ parietal alpha power approaching breakpoint (Cavanagh et al., 2021). ↑ parietal alpha power approaching breakpoint (Noback et al., 2024).	↑ parietal alpha power approaching breakpoint (Cavanagh et al., 2021). ↑ parietal alpha power approaching breakpoint (Noback et al., 2024).
Pharmacological	Amphetamine (10 and 20 mg) ↑ breakpoint (Noback et al., 2024).	Amphetamine (1 mg/kg, acute) ↑ breakpoint (Bensadoun et al., 2004). Amphetamine (1 mg/kg, i.p., acute) ↑ breakpoint (Heath et al., 2015). Amphetamine (1 mg/kg, i.p., acute) ↑ breakpoint (Hailwood et al., 2018). Amphetamine (0.3 mg/kg, i.p., acute) ↑ breakpoint (Noback et al., 2024).

As mentioned above, a touchscreen version of the PRBT is available that is capable of sustaining vigorous responding despite the lower level of tactile feedback provided by this system relative to more traditional operant boxes (Hailwood et al., 2018; Heath et al., 2015) (Figure 7). The touchscreen PRBT is sensitive to both pharmacological (Hailwood et al., 2018; Heath et al., 2015; Heath et al., 2016) and genetic manipulations (Heath et al., 2019) and the electrophysiological biomarker, parietal alpha, also increases approaching breakpoint (Noback et al., 2024) (Table 10). Interestingly, the touchscreen task detected a greater amphetamine-induced increase in breakpoint than had been previously reported using a nosepoke aperture-based system (Bensadoun et al., 2004; Heath et al., 2015), suggesting an increased sensitivity to performance-enhancing manipulations (Heath et al., 2015).

Incorporation of this task into a battery of touchscreen-based paradigms would enable assessment of effort-based reward valuation/effortful motivation alongside other cognitive domains in individual cohorts of animals (Heath et al., 2015). Such a design would in turn facilitate multifaceted characterization of animal models and determination of any motivational contributions to other behaviors (e.g., (Kwiatkowski et al., 2020; Roberts et al., 2021; Roberts et al., 2019)).

Effort-Based Decision-Making Tasks

While the PRBT enables assessment of effortful motivation across species, it is limited in that its outcome is unitary (i.e., simply when the participant chooses to stop responding). However, other tasks exist which offer choices between exerting effort for larger rewards versus nominal effort for smaller rewards. The archetypal clinical test of effort-based decision-making is the effort expenditure for rewards task (EEfRT; (Treadway, Buckholtz, Schwartzman, Lambert, & Zald, 2009)). The EEfRT is a multi-trial paradigm in which participants repeatedly choose between fulfilling easy versus difficult physical response criteria in exchange for “potentially real” monetary rewards of varying amounts (defined in Delayed reward). Reward magnitudes associated with the difficult option are modulated across trials, as is the probability that either choice would be rewarded at all. Furthermore, real-time feedback is provided during and after responding, indicating: (1) percent completion of the trial’s response criterion; (2) attainment or failure of the criterion; and (3) reward or non-reward (Figure 8).

Figure 8.

The effort-expenditure for rewards task (EEfRT) for humans (left) and the operant and touchscreen-based equivalent tasks for rodents (right). In the human task, subjects decide whether to make more button presses with their non-dominant hand (more challenging) for a large reward or fewer button presses with their dominant hand (easier) for a smaller reward. Both reward magnitude and probability of reward delivery are modulated from trial to trial. In the rodent effort discounting (ED) task (middle), rodents choose between completing many nosepokes for a large reward or a single nosepoke for a smaller reward. The number of responses required for the larger reward increase across trial blocks. The rearing effort discounting (RED) task (right) is a recently developed touchscreen-based effort-based decision-making task that provides the choice between earning small and large rewards by responding to, respectively, a stimulus presented at floor level (easy option) and one presented higher on the screen that requires the animal to rear up in order to reach it (hard option). Primary outcome measures for these tasks are the number of choices of the more challenging option versus the easier one, while ancillary measures such as attrition across the task provide indices of motivational failure over time.

Clinical sensitivity of the EEfRT is well-established, with deficits identified in people with major depression (Treadway, Bossaller, Shelton, & Zald, 2012), schizophrenia (e.g., (Barch, Treadway, & Schoen, 2014)), and autism spectrum disorder (Damiano, Aloi, Treadway, Bodfish, & Dichter, 2012) (Table 11). As with PD and DD tasks, converging evidence implicates frontal and striatal areas in EEfRT performance. Resting left frontal cortical activity predicted greater effort expenditure when probability of reward was sub-optimal (Hughes, Yates, Morton, & Smillie, 2015; Kaack, Chae, Shadli, & Hillman, 2020). Accordingly, transcranial direct current stimulation over the left dlPFC selectively increased effortful choice in trials with low reward probability or high potential reward (Ohmann, Kuper, & Wacker, 2018). Additionally, dopamine response in vmPFC and left caudate nucleus to amphetamine administration positively correlated with effortful choice in low-probability trials (Treadway, Buckholtz, et al., 2012). Meanwhile, NAc activity positively correlated with effortful choice in intermediate- and high-probability trials (Huang et al., 2016), while caudate activity was negatively correlated with high-effort selection during high-probability trials (X. H. Yang et al., 2016). While pharmacological studies utilizing the EEfRT are few, amphetamine (but not caffeine, nicotine, naltrexone, THC, or CBD; (Lawn et al., 2016; Nunez et al., 2022; Wardle, Treadway, & de Wit, 2012)) consistently increased effortful choice in healthy participants (Soder et al., 2021; Wardle, Treadway, Mayo, Zald, & de Wit, 2011), as seen in the PRBT.

Table 11.

Evidence for cross-species validity of tasks that assess effort-based reward valuation – Effort Expenditure for Rewards Task.

Reward Valuation – Effort #2
Validity Domain	Human Task – Effort Expenditure for Rewards Task (EEfRT)	Rodent Task – Effort Discounting (ED)	Rodent Touchscreen Task – Rearing Effort Discounting (RED)
Predictive	↓ effortful choice in people with major depression (Treadway et al., 2012).	↑ ED in mice following acute restraint stress (Shafiei et al., 2012). ↑ ED in rats following social stress (Lemon and Del Arco, 2022). ↑ ED in mice gestated under winter-like light cycle (Roberts and O’ Connor et al., 2023).	N/A
	↓ effortful choice in people with schizophrenia (Barch et al., 2014).	↑ ED in mice gestated under winter-like light cycle (Roberts and O’ Connor et al., 2023).
Neurobiological	Transcranial direct current stimulation over left dlPFC ↑ effortful choice (low-probability or high-reward trials) (Ohmann et al., 2018). Resting state left frontal cortex activity predicted effort expenditure (low-probability trials) (Hughes et al., 2015). Resting state left frontal cortex activity predicted effort expenditure (low-probability trials) (Kaack et al., 2020). vmPFC dopamine responsivity positively correlated with effortful choice (low-probability trials) (Treadway et al., 2012).	mPFC lesions ↑ ED (T-maze paradigm) (Walton et al., 2002). ↓ mPFC D2R expression associated with ↓ ED (Simon et al., 2013).	N/A
	NAc activity positively correlated with effortful choice in people (intermediate- and high-probability trials) (Huang et al., 2016). Caudate activity negatively correlated with effortful choice (high-probability trials) (Yang et al., 2016). Left caudate dopamine responsivity positively correlated with effortful choice (low-probability trials) (Treadway et al., 2012).	NAc core inactivation ↑ ED (Ghods-Sharifi et al., 2010). Intra-NAc dopamine D2/3R agonist ↑ ED (Bryce and Floresco, 2019).
Pharmacological	Amphetamine (20 mg, acute) ↑ effortful choice (Wardle et al., 2011). Amphetamine (20 mg, acute) ↑ effortful choice (Soder et al., 2021).	Amphetamine (0.5 mg/kg, i.p., acute) ↑ ED (↓ effortful choice) (Floresco et al., 2007). Amphetamine (0.1 mg/kg, i.p., acute) ↓ ED (↑ effortful choice) in mice gestated under winter-like light cycles (but ↑ ED in controls) (Roberts and O’Connor et al., 2023).	Amphetamine (0.25 and 0.50 mg/kg, acute) ↓ ED (↑ effortful choice) in haloperidol-treated (0.5–1.5 mg/kg) mice (Lopez-Cruz et al., 2024).

Similar to the PRBT, the EEfRT was preceded by several similar rodent tasks. An early such task utilized a modified T-maze paradigm whereby rats were simultaneously offered obstructed access to a preferred reinforcer (blocked by a scalable barrier) and unobstructed access to a smaller quantity of standard lab chow (Salamone, Cousins, & Bucher, 1994). Development of an operant effort discounting (ED) task enabled more trials and removed the confound of repeatedly handling animals during assessment. The ED task utilizes a similar two-lever design as PD and DD paradigms, whereby a large reward is offered at a progressively increasing or decreasing response cost alongside a smaller reward available at a consistently low cost (Floresco et al., 2008). As with the human EEfRT, the primary outcome is choice of high versus low effort options throughout the session, albeit in a systematic block-wise fashion (Figure 8).

Preclinical ED tasks detected effortful choice deficits in mice gestated under a winter-like light cycle (a risk factor for schizophrenia, depression, and bipolar disorder; (Disanto et al., 2012)) (Roberts, O'Connor, Kenton, Barnes, & Young, 2023), as well as in rodent stress models (Lemon & Del Arco, 2022; Shafiei, Gray, Viau, & Floresco, 2012), supporting preclinical sensitivity (Table 11). The effects of rodent neurological manipulations largely converge with clinical findings, demonstrating overall neurobiological validity of these assays. For example, medial frontal cortex lesions biased rats towards low-effort options in the T-maze paradigm (Walton, Bannerman, & Rushworth, 2002), and inactivating the NAc core reduced effortful choice in the operant ED task (Ghods-Sharifi & Floresco, 2010). Furthermore, application of the dopamine D_2/3R agonist quinpirole to either NAc core or shell similarly increased rat ED (Bryce & Floresco, 2019), while lower mPFC D₂R expression was associated with less ED in rats (N. W. Simon et al., 2013). Additionally, pharmacological predictive validity of the rodent ED task is supported by findings that amphetamine consistently increases effortful choice across humans and rodents (Floresco et al., 2008; Soder et al., 2021; Wardle et al., 2011) (Roberts et al., 2023) (but see: (Floresco et al., 2008)), as seen in the PRBT.

While the convergence in neurobiological substrates of effort-based reward valuation is promising, it should be noted that most of the clinical imaging findings reported above were specific to conditions of variably low, intermediate, or high reward probability. For example, dopamine function was most strongly associated with willingness to expend effort during low-probability EEfRT trials, in which both effort and probabilistic response costs were high (Treadway, Buckholtz, et al., 2012). Varying reward probabilities in the EEfRT was partly intended to ensure that subjects’ trial-to-trial performance reflected spontaneous decision making, as this added complexity would likely make calculation of long-term response strategies untenable (Treadway et al., 2009); however, this design may conflate effort- and probability-based reward valuation. While the relative contributions of effort versus probability discounting to EEfRT performance may be parsed by computational modeling (Soder et al., 2021), cross-species comparison of findings is nevertheless complicated by the fact that no existing preclinical paradigms modulate both dimensions simultaneously. Even with the advantages offered by touchscreen operant systems, such a complicated, multi-variable procedure may be beyond the ability of rodents to learn.

Some shortcomings of the rodent ED task are addressable by touchscreen paradigms. For example, completion of high-effort response criteria in this task carries a concurrent temporal cost, as it takes more time to complete multiple lever presses than one. A given manipulation’s effects on ED can therefore be conflated with its effects on DD. Indeed, 0.25 mg/kg and 0.50 mg/kg amphetamine originally produced opposite effects in a standard rat ED paradigm; however, follow-up testing using an “equivalent delay” ED task (with time to reward delivery normalized across low- and high-effort options) and a separate DD task parsed this effect profile into separate, dose-specific actions on these two distinct forms of decision making (Floresco et al., 2008). Furthermore, the block-wise task organization complicates ED assessment in a similar manner as PD and DD, with the order of response costs (ascending versus descending) strongly influencing the shape of the resultant discounting curve (Floresco et al., 2008). Both of these limitations are addressed by the novel touchscreen Rearing-Effort Discounting (RED) task (Lopez-Cruz et al., 2024). The RED task presents rodents with two stimuli requiring individual nosepokes; one stimulus yields a small reward and is readily accessible (floor-level), while the other yields a larger reward, but is positioned such that the rodent must rear to reach it. The height of the large-reward stimulus (hence requisite effort), is modulated from trial to trial, although the temporal cost remains consistent. The spatial arrangement of the stimuli enables explicit signaling of effort costs within individual trials, enabling circumvention of trial blocks. Both greater rearing distance and dopamine D₂R blockade (haloperidol) reduced selection of the high-effort stimulus while amphetamine rescued the haloperidol-induced deficit, demonstrating parametric and pharmacological validation (Lopez-Cruz et al., 2024). Although additional validation is required, the RED task is a prime example of how touchscreen operant systems may be leveraged to eliminate confounding factors and off-target cognitive demands in the assessment of reward valuation.

Discussion

Here, we reviewed the positive valence system as outlined by the RDoC framework and evaluated the potential of preclinical rodent research to investigate the neurobiology underlying these functions in non-pathological and disease states. Given the potential for cross-species translation, we also identified the utility and validity of touchscreen-based rodent tasks that assess such positive valence domains. Touchscreen-based tasks have been most commonly implemented to investigate aspects of reward learning with evidence for neurobiological, pharmacological, and predictive validity. Although the touchscreen systems have been less extensively used to assess functions of reward valuation, there are available paradigms for discounting (both probability and delay) and effort-based motivation (e.g., PRBT and RED task), along with an extensive literature using operant paradigms. Reward responsiveness is the least characterized of the RDoC domains in preclinical research, which may in part be due to the lack of relevant behavioral endpoints in paradigms such as the MID (i.e., reward anticipation) or simple guessing task (i.e., initial response to reward), thus necessitating the incorporation of neural recoding measures to capture reward-related activity. Nonetheless, equivalent task designs can be easily adapted for rodents and pairing in vivo techniques such as imaging calcium or voltage indicators, electrophysiology, or mini-scopes could provide a precision approach for investigating the mechanisms underlying reward anticipation and initial response (see Table 1). Additionally, measures of reward satiation can be assessed in essentially all operant and touchscreen paradigms, represented in the response rate and latencies at the end of the session compared to the start, or separately in fixed ratio paradigms. Overall, there are a number of established and recently developed rodent paradigms suitable for investigating the positive valence systems, and future validation will aid in establishing paradigms that can determine neural circuitry, characterize clinically informed disease models, and test potential therapeutic compounds.

As discussed, there is an extensive amount of shared neurobiological circuitry across the RDoC categories, as evidenced by imaging and lesion studies in both humans and rodents. Commonly recruited brain structures include the OFC - strongly implicated in reward value representation and updating - and the striatum - dopaminergic signaling facilitates the anticipation of reward, the detection of unexpected reward, and using contextual associations to promote reward-related actions. Thus, future efforts aimed at dissociating whether neurobiology is similarly recruited by related neural computations (e.g., anticipating reward and prediction errors), as opposed to behavioral paradigms requiring various reward-related functions (e.g., probabilistic reversal learning), will be important for identifying mechanisms that are shared and distinct across reward-related functions. Relatedly, there is tremendous overlap in the reward-related behavioral abnormalities that are present across human psychiatric disorders. For example, individuals with schizophrenia and depression both show impairments in probabilistic learning, exerting effort to obtain rewards, and discounting future rewards, which may be the result of reductions in hedonic capacity and/or reward-seeking motivation (Höflich, Michenthaler, Kasper, & Lanzenberger, 2019; Oorschot et al., 2013; Treadway & Zald, 2011), but not when that learning is implicit (Danion, Meulemans, Kauffmann-Muller, & Vermaat, 2001; Perry, Light, Davis, & Braff, 2000; Soler, Ruiz, Dasí, & Fuentes-Durá, 2015; Sponheim, Steele, & McGuire, 2004). Importantly, the degree of individual symptom severity, rather than diagnostic category, appears to be a strong predictor of reward-related dysfunction as observed in depression, anxiety, and schizophrenia studies (Lewandowski et al., 2016; Erin E Reilly et al., 2020; Strauss et al., 2015). The degree of overlap in neurobiology underlying the fundamental behavior and symptomology in disease not only supports a transdiagnostic approach for clinical populations but highlights the necessity of preclinical approaches with high specificity, both in the behavior being assessed and the neural circuits being targeted. Together, the touchscreen systems may provide a valuable platform for applying the RDoC framework to preclinical research. This approach can be maximized by combining translational tasks developed by forward or reverse translation with recording and manipulation techniques possessing high cellular specificity and temporal precision, enabling researchers to dissect the mechanisms underlying positive valence functions.

Future Directions

It is important to note the potential limitations and considerations that should be addressed in future research investigating positive valence systems in rodents. First, while visual stimuli can be similarly presented to humans and rodents on a computerized screen, the method of task interaction differs across the human paradigms described in this review. While rodents can nosepoke or press at a touch-sensitive screen, humans may instead use a joystick, keyboard, or mouse clicks, all of which are more common than an actual touchscreen (though they can be used). Using the same presentation media (e.g., a screen), is however, an important component of establishing tasks with cross-species translation, increasing the likelihood of connecting visual and cortical regions (Norman et al., 2021). Moreover, tasks that are designed for rodents can be forward translated for use in humans, facilitating increased use of human touchscreen tasks. Such an approach was taken with the paired associative learning task (PAL), which identified identical deficits in humans possessing a genetic risk factor for schizophrenia and mice with the corresponding mutation (Nithianantharajah et al., 2013; Nithianantharajah et al., 2015). Finally, the global availability of smart phones enables implementing touchscreens in human testing that can take place outside of a laboratory utilizing ecological momentary assessment approaches (Barch et al., 2023; Bomyea et al., 2021; Titone et al., 2022). Thus, touchscreen use could provide an avenue for delineating neural mechanisms underlying positive valence across species, with applicability to clinical testing.

Additional limitations for evaluating positive valence system functions in rodents, are also worth mentioning. The degree of variance in movement requirements within cross-species tasks is an important consideration given the corticostriatal dopaminergic contributions to both reward-related functions and action initiation/inhibition. While chamber systems offer a lower physical requirement compared to maze-based paradigms, differences in effort to achieve rewards between species may be a factor when comparing human and rodent results, even in comparable tasks. The touchscreen PRBT of motivational effort, however, was successful in identifying motivational deficits in humans with Huntington’s disease (Heath et al., 2019) and schizophrenia (Bismark et al., 2018; Young et al., 2015) and their corresponding mouse models, in addition to other disease states. Additionally, the potential for movement artifacts to interfere with neural activity related to reward processes during testing combined with recording techniques must always be considered and can be minimized with appropriate signal filtering and baselining. Furthermore, causal manipulations (e.g., optogenetics), that target corticostriatal circuitry should consider temporal epochs in which stimulation may be delivered outside of movement windows to avoid off-target behavioral perturbations. Lastly is the nature of the rewarding stimulus, which is typically monetary incentive in humans and food reward in rodents. As mentioned, there is evidence that anticipating food or money rewards elicits similar striatal activity in humans (J. J. Simon et al., 2015a). Similarly, when individuals valued food, money, and social rewards equally, the rewards were indistinguishable in their ability to drive stimulus-association learning (Lehner, Balsters, Herger, Hare, & Wenderoth, 2017). That said, ancillary reward-related functions such as reward satiation may be less of a consideration for monetary rewards during a testing session compared to food stimuli, again emphasizing the importance of implementing trial analyses in rodent paradigms to assess loss of reward valuation across a session. Lastly, the requirement of food restriction in rodent-based paradigms is an important consideration, as perpetual states of hunger may be associated with elevated levels of stress hormones and altered reward seeking, and may not be represented in the reward drive being assessed in humans (e.g., need for financial compensation) (Mallien et al., 2016; Quante et al., 2023).

While the touchscreen systems provide an unparalleled ability to present various forms of basic and complex stimuli to rodents, one limiting aspect in regard to assessing positive valence systems is the inability to test social cognition. One of the most debilitating consequences of reward-related dysfunction in psychiatric disease is the loss of interest and/or motivation to engage in social interactions, impeding development of interpersonal relationships and the ability to use social stimuli to navigate the world (Bellack, Morrison, Wixted, & Mueser, 1990). Unsurprisingly, deficits in reward valuation or motivation that are detected using other rewards (e.g., money) can also manifest as reduced social drive, which is persistent in conditions such as schizophrenia (Gard et al., 2014; Mueser, Bellack, Douglas, & Morrison, 1991), depression (Hirschfeld et al., 2000; Rhebergen et al., 2010), and PTSD (Brancu et al., 2014; Scoglio et al., 2022). Rodents are social animals and rats and mice will exert effort on a novel lever-pressing PRBT to achieve access to a social interaction (Lee, Venniro, Shaham, Hope, & Ramsey, 2024; Ramsey, Holloman, Hope, Shaham, & Venniro, 2022; Ramsey, Holloman, Lee, & Venniro, 2023; Venniro & Shaham, 2020). While such a task would require modification of the touchscreen chamber to include a compartment for the social partner, the PRBT component is already established in the touchscreens. Alternatively, human social tasks generally present participants with images of social stimuli and touchscreens do afford the ability to present essentially any visual stimulus to the rodents, including videos or 3-dimensional dynamic graphics. Interestingly, previous studies have shown that rats exhibit a place preference for a chamber equipped with a video recording of a conspecific over a barren chamber (Yakura et al., 2018). While the absence of olfactory and tactile interaction may limit the motivational capacity of virtual stimuli to drive rodent behavior, it would be interesting to investigate if pharmacological compounds that increase sociability in humans and rodents could modulate rodent’s behavioral responses to social stimuli presented on the touchscreen.

Overall, abnormalities in positive valence system functioning are a consistent and debilitating collection of symptoms present in various psychiatric disorders. In-line with the RDoC framework, studies in humans and rodents demonstrate a high degree of overlap in terms of the underlying neurobiology that supports these functions, as well as the consequences of pharmacological intervention. As efforts continue to further characterize the neurobiology and neurochemistry that facilitate reward-related processing, validated cross-species paradigms are a necessity for maximizing the translational potential of findings from preclinical to clinical settings. The touchscreen systems provide a valuable resource for this endeavor, enabling researchers to administer extremely similar, if not identical, tasks to humans and rodents. The success of translation, however, is dependent on establishing pharmacological, neurobiological, and predictive validity for rodent paradigms to ensure that analogous behavioral functions and neurological systems are being measured across species.

Abbreviations

5-HT

Serotonin

ADHD

Attention-deficit/hyperactivity disorder

ACC

Anterior cingulate cortex

ACh

Acetylcholine

BOLD

Blood oxygen level–dependent

BLA

Basolateral amygdala

CBD

Cannabidiol

CNTRICS

Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia

CNV

Copy number variant

D1R

Dopamine receptor D1

D2R

Dopamine receptor D2

D2/3R

Dopamine receptor D2/3

DA

Dopamine

DD

Delay discounting

dlPFC

Dorsolateral prefrontal cortex

DLS

Dorsolateral striatum

DS

Discriminative stimulus

DSM

Diagnostic and statistical manual of mental disorders

EEfRT

Effort expenditure for rewards task

ED

Effort discounting

ELS

Early life stress

ERP

Event related potential

fMRI

Functional magnetic resonance imagining

FR

Fixed ratio

GABA

Gamma-aminobutyric acid

GECI

Genetically encoded calcium indicators

GEVI

Genetically encoded voltage indicators

GWAS

Genome wide association studies

MID

Monetary incentive delay

mPFC

Medial prefrontal cortex

MSN

Medium spiny neurons

NAc

Nucleus accumbens

NBM

Nucleus basalis magnocellularis

NEWMEDS

Novel Methods leading to New Medications in Depression and Schizophrenia

NIMH

National Institute of Mental Health

NMDA

N-methyl-D-aspartate

O2

Regional oxygen

OCD

Obsessive compulsive disorder

OFC

Orbitofrontal cortex

PAL

Paired associative learning

PD

Probabilistic discounting

PET

Positron emission tomography

PFC

Prefrontal cortex

PRBT

Progressive ratio breakpoint task

PRLT

Probabilistic reversal learning task

PRT

Probabilistic reward task

RDoC

Research Domain and Criteria

RED

Rearing-effort discounting

RewP

Reward positivity

RPE

Reward prediction error

S-R-O

Stimulus – response – outcome

SSRI

Selective serotonin re-uptake inhibitor

THC

Delta-9-tetrahydrocannabinol

vmOFC

Ventromedial orbitofrontal cortex

vmPFC

Ventromedial prefrontal cortex

VMCL

Visuomotor conditional learning

VTA

Ventral tegmental area