An Integrative Perspective on the Role of Dopamine in Schizophrenia

Abstract

We propose that schizophrenia involves a combination of decreased phasic dopamine responses for relevant stimuli and increased spontaneous phasic dopamine release. Using insights from computational reinforcement-learning models and basic-science studies of the dopamine system, we show that each of these two disturbances contributes to a specific symptom domain and explains a large set of experimental findings associated with that domain. Reduced phasic responses for relevant stimuli help to explain negative symptoms and provide a unified explanation for the following experimental findings in schizophrenia, most of which have been shown to correlate with negative symptoms: reduced learning from rewards; blunted activation of the ventral striatum, midbrain, and other limbic regions for rewards and positive prediction errors; blunted activation of the ventral striatum during reward anticipation; blunted autonomic responding for relevant stimuli; blunted neural activation for aversive outcomes and aversive prediction errors; reduced willingness to expend effort for rewards; and psychomotor slowing. Increased spontaneous phasic dopamine release helps to explain positive symptoms and provides a unified explanation for the following experimental findings in schizophrenia, most of which have been shown to correlate with positive symptoms: aberrant learning for neutral cues (assessed with behavioral and autonomic responses), and aberrant, increased activation of the ventral striatum, midbrain, and other limbic regions for neutral cues, neutral outcomes, and neutral prediction errors. Taken together, then, these two disturbances explain many findings in schizophrenia. We review evidence supporting their co-occurrence and consider their differential implications for the treatment of positive and negative symptoms.

Studies using positron emission tomography (PET) and single-photon emission computed tomography (SPECT) have shown that presynaptic striatal dopamine function is increased in schizophrenia and correlates with positive symptoms (1). Specifically, schizophrenia involves increased dopamine synthesis in the striatum (1–3), even in medication-naive prodromal patients (4). Furthermore, patients at ultra-high risk of psychosis who later transition to psychosis have greater dopamine synthesis than those who do not (5) and show an increase in dopamine synthesis from the prodromal stage to psychosis (6). Amphetamine-induced dopamine release is also increased in schizophrenia (1), including in medication-naive patients (7), and correlates with the extent to which amphetamine worsens positive symptoms (7). Baseline dopamine levels are also increased in schizophrenia (1). These abnormalities are localized predominantly in the associative striatum (4,8–10).

Dopamine neurons fire tonically and phasically, leading, in the striatum, to tonic dopamine concentrations and spikes in those concentrations called transients, respectively (11,12). PET and SPECT’s poor temporal resolution implies that they measure tonic dopamine or the occurrence of transients over sustained periods.

Amphetamine increases spontaneous dopamine transients—transients that are not time-locked to identified stimuli or events—in the striatum (Figure 1A, B) (11,13). Furthermore, whereas at moderate doses, amphetamine increases appropriate (adaptive) striatal dopamine transients to a reward-predicting cue, at high doses, it blunts these adaptive transients (Figure 1C) and disrupts the appropriate behavioral responses—while still increasing spontaneous transients (Figure 1D) (13). Amphetamine also increases tonic striatal dopamine, but that effect is small and short lived (13).

Figure 1.

Amphetamine, at high doses, increases spontaneous dopamine (DA) transients while simultaneously blunting adaptive transients for relevant stimuli, as measured by fast-scan cyclic voltammetry in the striatum. (A) A high dose of amphetamine (right) markedly increases the number of spontaneous transients (red asterisks) relative to the unmedicated state (left). (B) A high dose of amphetamine markedly increases the frequency, amplitude, and duration of spontaneous transients. Values indicated are as percent increases over the predrug state. (C) A reward-predicting cue (presented at time 0) elicits a cue-locked transient in the unmedicated state and under saline (left two panels). A moderate dose of amphetamine increases this transient (third panel), but a large dose of amphetamine virtually abolishes it (right panel). (D) Even though a high dose of amphetamine virtually abolishes the adaptive transient for the reward-predicting cue, it markedly increases spontaneous transients in the same task (measured in the 10 seconds before cue presentation). Adapted, with permission, from Daberkow et al. (13).

Excessive amphetamine-induced striatal dopamine release in schizophrenia therefore likely reflects increased spontaneous transients or possibly increased tonic dopamine; it seems less likely to reflect adaptive stimulus-driven transients because these studies take place at rest, without rewards or reward-predicting cues. Increased spontaneous transients and increased tonic dopamine would each also explain all of the other PET and SPECT findings. Increased spontaneous transients in schizophrenia may reflect inappropriate, “chaotic” phasic firing of dopamine neurons (14–16).

The findings concerning amphetamine’s effects on striatal dopamine may be directly relevant to understand psychosis. Amphetamine and other psychostimulants can cause or exacerbate psychosis (17,18); at high doses, all psychostimulants increase spontaneous dopamine transients in the striatum (11). This article will demonstrate that the idea that schizophrenia may similarly involve increased spontaneous transients in the striatum (or, less likely, increased striatal tonic dopamine) explains many laboratory findings that correlate with positive symptoms and may help explain positive symptoms themselves. In addition, the idea that schizophrenia also involves decreased adaptive transients in the striatum for relevant stimuli explains many laboratory findings that correlate with negative symptoms and may help explain negative symptoms themselves. The plausibility of the coexistence of these two disturbances in schizophrenia is supported by their coexistence under high doses of amphetamine (13), which are psychotogenic.

These dopaminergic disturbances might be caused by multiple etiopathogenetic mechanisms, including mechanisms affecting other neurotransmitter systems. For example, ketamine, a psychotogenic N-methyl-D-aspartate (NMDA) antagonist (19), produces disturbances in striatal dopamine similar to those observed in schizophrenia, including increased amphetamine-induced striatal dopamine release and increased striatal dopamine (although the latter has not always been replicated) (20). Thus, NMDA hypofunction in schizophrenia (21) could cause psychosis at least partly through effects on dopamine (22). In fact, ketamine and phencyclidine, another psychotogenic NMDA antagonist, increase spontaneous firing and bursting in dopamine neurons (23,24), so they may increase spontaneous transients. Causal interactions between NMDA dysfunction and dopaminergic dysfunction may be bidirectional (25); for example, dopaminergic dysfunction likely affects NMDA-based synaptic plasticity, which may play a role in schizophrenia (26).

COMPUTATIONAL ROLES OF DOPAMINE

Striatal medium spiny neurons (MSNs) containing D₁ receptors are part of the direct (Go) pathway, which facilitates (gates) the most appropriate actions; striatal MSNs containing D₂ receptors are part of the indirect (NoGo) pathway, which suppresses inappropriate actions (27–29). Computationally, Go and NoGo pathways likely reflect the positive and negative values of actions, respectively, with actions being selected as a function of the difference between these two values (Box 1; Figure 2) (30). Actions to be selected may therefore elicit activity in both their Go and NoGo striatal representations (31).

Box 1. A Computational Account of the Role of Dopamine in Learning and Performance.

The opponent-actor-learning (OpAL) computational model provides an integrated account of the distinct roles of dopamine in learning and performance/motivation (30). OpAL is a generalization of the standard actor-critic model (32) that captures two important aspects of the neurobiology of the basal ganglia: the existence of separate direct (Go) and indirect (NoGo) pathways, and the influences of dopamine during learning and performance on each of these pathways (Figures 2 and 3). OpAL is an abstract version of a more detailed neurocomputational model that incorporates these aspects of basal ganglia structure and function (27).

OpAL, like the actor-critic, includes a critic that learns the values of states, V(s), using the standard temporal-difference-learning equation:

$$V(s)\leftarrow V(s)+{\alpha }_c\delta ,$$ (1)

Where α_c is the critic’s learning rate and δ is a prediction error (PE), given by:

$$\delta =r+V(s^{\prime })-V(s),$$ (2)

Where r is the actual reinforcement received, and V(s′) is the value of the new state (32).

In the actor-critic, there is a single actor that learns the preferences for actions in given states (32). OpAL, however, includes two opponent actors to model the separate Go and NoGo pathways (Figure 2). Learning in these pathways is characterized by the following equations, respectively:

$$G(s,a)\leftarrow G(s,a)+{\alpha }_{G}G(s,a)\delta ,$$ (3)

and

$$N(s,a)\leftarrow N(s,a)+{\alpha }_{N}N(s,a)[-\delta ],$$ (4)

where G(s,a) and N(s,a) represent the Go and NoGo values for action a in state s, and α_G and α_N are the learning rates for each pathway.

The symmetric effects of δ on G and N capture the fact that phasic-dopamine increases induce long-term potentiation and long-term depression in the direct and indirect pathways, respectively, whereas phasic-dopamine decreases may have the opposite effects (Figures 2A and 3A–C) (29,38).

For simplicity, the previous equations use a single value for δ. However, positive and negative values of δ, which we represent by δ⁺ and δ⁻, are signaled by phasic dopamine-neuron bursts and pauses, respectively (32), and these may be differentially disrupted in a given disorder. For example, low levels of dopamine, as in unmedicated Parkinson’s disease, might lead to low δ⁺ but unimpaired or even exaggerated δ⁻, thereby explaining why unmedicated Parkinson’s patients have impaired Go learning but preserved or improved NoGo learning (28). The impaired Go learning and blunted signaling of positive PEs found in schizophrenia may similarly reflect low adaptive δ⁺ (see text).

Actions are selected in OpAL using a softmax function, as in the actor-critic (32), but allowing the gain of each pathway to be modulated independently:

$$p(a\mid s)=\frac{e^{{\beta }_{G}G(s,a)-{\beta }_{N}N(s,a)}}{{\displaystyle \sum _i}{e}^{{\beta }_{G}G(s,i)-{\beta }_{N}N(s,i)}},$$ (5)

where p(a|s) is the probability of selecting action a in state s, β_G and β_N are parameters that determine the extent to which the Go and NoGo pathway, respectively, influence choice, and the sum is over all available actions in state s (see also Figure 2B).

Dopamine during choice is assumed to increase β_G and decrease β_N because dopamine increases the excitability of Go MSNs through its action on D₁ receptors and decreases the excitability of NoGo MSNs through its action on D₂ receptors (Figures 2A and 3D) (29). Thus, for example, low levels of dopamine, as in unmedicated Parkinson’s disease, would lead to low β_G and high β_N, thereby causing learned NoGo values to be weighted more strongly than learned Go values, which in turn produces a tendency for inaction. A simple mathematical formulation of these effects of dopamine during choice on β_G and β_N is to make

$${\beta }_G=\beta (1+\rho )$$ (6)

and

$${\beta }_N=\beta (1-\rho )$$ (7)

where β is a constant, and ρ, which can vary between −1 and 1, represents the amount of dopamine present during choice (30).

In the original OpAL model, ρ was assumed to represent dopamine levels during choice, and these levels were manipulated to simulate changes in tonic dopamine induced by pharmacological manipulations (30). However, phasic-dopamine responses (elicited, for example, by reward-predicting cues) also invigorate action and influence choice (61,103,119), as does optogenetic stimulation of dopamine neurons using parameters that elicit naturalistic-like phasic responses (36). These findings are perhaps unsurprising given that, from the perspective of striatal D₁ and D₂ receptors, what likely matters, at least as a first approximation, is the overall amount of dopamine impinging on them. A better model is therefore that ρ represents the total amount of dopamine during choice, which depends both on tonic levels of dopamine, τ, and on any PEs, δ, elicited by cues present during, or shortly before, choice:

$$\rho =\tau +\delta .$$ (8)

Combining Equations 5–8 gives the following choice equation:

$$p(a\mid s)=\frac{e^{\beta (1+\tau +\delta )G(s,a)-\beta (1-\tau -\delta )N(s,a)}}{{\displaystyle \sum _i}{e}^{\beta (1+\tau +\delta )G(s,i)-\beta (1-\tau -\delta )N(s,i)}}.$$ (9)

In short, phasic dopamine following choice or state transitions affects the learning of state values (Equation 1) and of Go and NoGo state-action values (Equations 3 and 4, respectively; Figures 2A and 3A–C). Tonic and phasic dopamine during choice, in contrast, affect the amplification of Go versus NoGo values (Equation 9; Figures 2A and 3D), thereby affecting performance.

Figure 2.

Effects of dopamine in the striatum, and mechanisms of action selection in the basal ganglia. (A) Effects of dopamine (DA) on plasticity and excitability (gain) of striatal medium spiny neurons (MSNs) of the direct (Go) and indirect (NoGo) basal ganglia pathways. The current state or stimulus, s, is represented in cortex. Corticostriatal synapses onto D₁-containining MSNs represent the positive value of learned associations between states or stimuli s and actions a [G(s,a); Box 1]; corticostriatal synapses onto D₂-containining MSNs represent the negative value of learned associations between states or stimuli and actions [N(s,a); Box 1]. Phasic dopamine bursts following an action strengthen corticostriatal synapses to Go MSNs through D₁-mediated long-term potentiation and weaken corticostriatal projections to NoGo MSNs through D₂-mediated long-term depression (indicated by the circles with a plus and a minus sign, respectively) (Equations 3–4 in Box 1; Figure 3A, B). Phasic dopamine dips following an action may have the opposite effects (Figure 3C). Dopamine during choice amplifies the gain of Go MSNs (β_G) by increasing their excitability through D₁ receptors and reduces the gain of NoGo MSNs (β_N) by decreasing their excitability through D₂ receptors (indicated respectively by the circle with a multiplication sign and the tandem circles with a minus sign and a multiplication sign) (Equations 6–9 in Box 1; Figure 3D). The output of Go MSNs reflects learned Go values [G(s,a)], modulated by the gain of the Go pathway (β_G), which can be represented mathematically as β_G × G(s,a). Similarly, the output of NoGo MSNs reflects learned NoGo values [N(s,a)], modulated by the gain of the NoGo pathway (β_N), which can be represented mathematically as β_N × N(s,a). (B) Action-selection mechanisms in the basal ganglia. Go and NoGo values [G(s,a) and N(s,a), respectively] are specific for each state-action [(s,a)] pair. Illustrated are three possible actions (labeled 1, 2, and 3) for a given state s. Each action has its own G(s,a) and N(s,a) values, which are determined by the strength of the corticostriatal synapses from the cortical representation of state s to Go and NoGo MSNs, respectively, for that state-action pair [(s,a)]. The output of Go and NoGo MSNs is determined by these learned values [G(s,a) and N(s,a), respectively] modulated by the gain of the respective pathway (β_G and β_N, respectively), yielding the same products as in panel (A) [β_G × G(s,a) and β_N × N(s,a), respectively]. The projections from all basal ganglia nuclei—striatum, globus pallidus external segment (GPe), globus pallidus internal segment (GPi), and substantia nigra pars reticulata (SNr)—are inhibitory. In simplified terms, if the projection neurons in an area receive afferent inhibitory projections, that area can be seen as flipping the sign of the information in those afferent projections. This process is represented in the graph by circles with a minus sign inside. Under this simplified conceptualization, the GPe can be seen as flipping the sign of β_N × N(s,a), yielding −β_N × N(s,a). The GPi then combines (sums) its two incoming inputs [β_G × G(s,a) and −β_N × N(s,a)], but since its incoming projections are inhibitory, it flips the sign of those inputs, yielding −β_G × G(s,a) + β_N × N(s,a). Finally, given that the projections from the GPi to the thalamus are also inhibitory, the thalamus flips the sign again, yielding β_G × G(s,a) − β_N × N(s,a). The cortex therefore receives information about the difference β_G × G(s,a) − β_N × N(s,a) for each action a available in the current state s. (Note that these differences are the values of the exponents in Equation 5 in Box 1.) Lateral inhibition in cortex then implements a competitive dynamics that performs action selection using these differences (approximated in Equation 5 in Box 1 using a softmax). In short, the best action in a given state s is determined on the basis of the differences β_G × G(s,a) − β_N × N(s,a) for all actions a available in s (Equations 5 and 9 in Box 1). This account is, of course, greatly simplified—for example, it does not take into account the full complexity of the basal-ganglia anatomy, it assumes that competition via lateral inhibition occurs only in cortex, and it assumes that all processing other than the competition approximated by the softmax is linear. It has the advantage, however, of clearly linking each structure and processing step in the basal ganglia to a simple, well-defined mathematical operation, and of showing how all of those operations work together to implement a sensible action-selection algorithm (Box 1).

Go and NoGo values are learned on the basis of phasic changes in dopamine-neuron firing (Figures 2A and 3A–C). The phasic firing of a sizeable proportion of dopamine neurons signals positive prediction errors (PEs), which occur when outcomes are better than expected (32–34). These signals cause long-term potentiation in the Go pathway and long-term depression in the NoGo pathway (29), increasing and decreasing Go and NoGo values, respectively (Figures 2A and 3A, B). Thus, actions that are followed by positive PEs become more likely to be selected again. Indeed, optogenetically induced phasic firing of dopamine neurons (35,36) or of D₁ MSNs (37) causes appetitive conditioning. Phasic pauses in firing, in contrast, signal negative PEs (32), which occur when outcomes are worse than expected. Reduced dopamine causes long-term depression in the Go pathway and long-term potentiation in the NoGo pathway (29,38,39); dopamine dips caused by phasic pauses may have similar effects (Figures 2A and 3C), making the preceding action less likely to be selected. Indeed, optogenetic inhibition of dopamine neurons (36,40) or excitation of D₂ MSNs (37) causes aversive learning. These ideas have been formalized in biologically detailed (27) and more abstract models (Box 1; Figures 2 and 3A–C) (30).

Figure 3.

Effects of dopamine (DA) on plasticity and excitability (gain) of striatal direct (Go) and indirect (NoGo) medium spiny neurons (MSNs). (A) Reference scenario against which the figures in the remaining panels should be compared. In this scenario, we assume that the Go and NoGo corticostriatal synapses [G(s,a) and N(s,a), respectively] for the state-action pair under scrutiny have the same initial weights. (B) If the person (or animal) executes action a in state s, and that is followed by a phasic dopamine burst (corresponding to a positive prediction error; Box 1), the Go weight for that state-action pair [G(s,a)] is increased, and the NoGo weight for that state-action pair [N(s,a)] is decreased [compare the thickness of the arrows depicting the corticostriatal synapses with each other and with those in panel (A)] (Equations 3 and 4 in Box 1; Figure 2A). Thus, the next time the person (or animal) is in state s, it will have a greater tendency to choose that action [compare the size of the Go and NoGo MSNs, which are intended to depict activation levels, with each other and with those in panel (A), or compare the size of the arrows departing from Go and NoGo MSNs, which convey the same information]. (C) If the person (or animal) executes action a in state s, and that is followed by a phasic dopamine dip (corresponding to a negative prediction error), the Go weight for that state-action pair [G(s,a)] is decreased, and the NoGo weight for that state-action pair [N(s,a)] is increased [compare the thickness of the arrows depicting the corticostriatal synapses with each other and with those in panel (A)] (Equations 3 and 4 in Box 1; Figure 2A). Thus, the next time the person (or animal) is in state s, it will have less tendency to choose that action [compare the size of the Go and NoGo MSNs (or of the arrows that depart from them) with each other and with those in panel (A)]. (D) If dopamine during choice is increased, either because tonic dopamine is increased or because the cues presented themselves elicit a dopamine burst (positive prediction error), the activity of Go MSNs is increased, and the activity of NoGo MSNs is decreased [compare the size of Go and NoGo MSNs (or of the arrows that depart from them) with each other and with those in panel (A)], resulting in greater weighting of positive relative to negative values and therefore a greater tendency to select the action (Equations 6–9 in Box 1; Figure 2A). This effect is due to gain modulation of corticostriatal synapses rather than to changes in their strength [note that the arrows depicting the weights of corticostriatal synapses are unchanged relative to panel (A)]. Thus, this effect during choice is separate from the effects on learning. However, the two effects interact because the gain modulation acts on the learned synaptic weights (Equations 5 and 9 in Box 1; Figure 2A).

In addition to the effects of phasic dopamine changes after choice, which support learning, dopamine has strong effects during choice, affecting performance. Specifically, dopamine increases the excitability of Go MSNs and decreases the excitability of NoGo MSNs (29), thereby increasing the gain of Go (positive) and decreasing the gain of NoGo (negative) values, respectively (Box 1; Figures 2A and 3D) (27,30). Pharmacological studies confirm that dopamine affects both performance and learning (30,39,41,42). Dopaminergic manipulations during choice, in ways that could not have affected learning, show that increasing dopamine increases the weighting of positive relative to negative values, confirming an effect on performance (43,44). Dopaminergic manipulations during learning modulate PE signaling (45–48), in ways that are predictive of subsequent choice (45–47), confirming an effect on learning. Consistent with these dual effects, optogenetic stimulation and inhibition of dopamine neurons cause appetitive and aversive learning, respectively, if done at outcome, but increase and decrease approach behavior, respectively, if done during choice (36).

These basal ganglia learning and selection mechanisms also apply to cognitive “actions.” For example, Go signals cause working-memory updating, whereas NoGo signals prevent such updating, protecting current representations (49). Indeed, dopamine manipulations have similar effects on the ability to gate relevant stimuli into working memory versus ignoring distractors as they do on learning from positive versus negative outcomes, respectively (50).

ABERRANT LEARNING FOR IRRELEVANT STIMULI IN SCHIZOPHRENIA

Findings

Behaviorally and autonomically, schizophrenia patients, compared to controls, respond less to relevant cues (i.e., cues that predict reinforcement) and more to neutral cues, although they respond more to relevant than to neutral cues (51–54). In a task in which one cue feature predicts reward and another does not, psychotic (or psychotic-like) symptoms correlate with an increased tendency to consider the irrelevant feature also predictive of reward in unmedicated participants at ultrahigh risk for psychosis (55), medicated schizophrenia patients (56), and Parkinson’s patients given D₂ agonists (57). Reaction-time measures also show inappropriate, increased learning for the irrelevant feature in schizophrenia patients (58). Medicated patients further show a decreased tendency to learn about the feature that does predict reward, both in explicit reports and in reaction times (56).

The neural findings in these studies similarly show that relative to controls, patients activate the midbrain, ventral striatum (VS), and other limbic regions more for neutral cues and outcomes and less for relevant cues and outcomes (51–54). Increased midbrain activation to a neutral relative to a relevant cue correlated with delusions in one study (52).

Relation to Dopamine Function

In short, schizophrenia is associated with (a) increased behavioral, autonomic, and neural responding for neutral stimuli, which correlates with positive symptoms, and (b) decreased responding for relevant stimuli. Increased responding for neutral stimuli can be explained by increased spontaneous transients, which would cause aberrant learning for those stimuli (Figure 4), or by increased tonic dopamine, which could increase overall gain (59), thereby causing a general tendency for increased responding. Decreased responding for relevant stimuli can be explained by blunted adaptive transients, which would cause impaired learning for those stimuli.

Figure 4.

Increased spontaneous dopamine transients in the striatum explain several neural and behavioral laboratory findings in schizophrenia that correlate with positive symptoms and help to explain positive symptoms themselves. Increased spontaneous dopamine transients (green) have specific effects on computational variables (orange-brown) that, in turn, cause specific neural and behavioral disturbances that have been found in the laboratory in schizophrenia (blue, with numbers in parenthesis referring to relevant citations). In real life, the same alterations in the computational variables may cause specific neurocognitive disturbances (blue-red gradient) that, in turn, cause positive symptoms (red). The same computational alterations can also explain dyskinesia associated with schizophrenia (dotted red). In more detail, increased spontaneous dopamine transients that follow neutral stimuli function as positive prediction errors (PEs) for those stimuli, causing increased midbrain activity for “neutral PEs,” as has been observed in schizophrenia (53). According to Equation 1 (Box 1), these inappropriate positive PEs cause increased, inappropriate value learning for neutral stimuli, which in turn causes increased activation of value regions, such as the ventral striatum (VS), for neutral stimuli, as has been observed in schizophrenia (51,52). This activation, particularly for the midbrain (52), may also reflect the increased PEs that occur when the stimulus is presented. The inappropriate value learning for neutral stimuli may also cause increased autonomic activation for those stimuli, as has also been observed in schizophrenia (51). In real life, the inappropriate value learning may lead to aberrant valuation of stimuli, thoughts, percepts, etc., possibly contributing to positive symptoms. In addition, according to Equation 3 (Box 1), the inappropriate positive PEs cause inappropriate direct-pathway (Go) learning for neutral stimuli-action pairs, leading to inappropriate behavioral responding to neutral stimuli, as has also been observed in schizophrenia (55,56). When applied to the cognitive domain, this inappropriate Go learning may lead to learned gating of aberrant thoughts and percepts, possibly contributing to positive symptoms. When applied to the motor domain, this inappropriate Go learning may lead to dyskinesia, which is associated with schizophrenia even in antipsychotic-naive patients (148). The fact that increased spontaneous dopamine transients may be the common cause of all of the depicted laboratory-based deficits (blue boxes) and also contribute to positive symptoms (red box) explains the correlations between these laboratory deficits and positive symptoms (52,55–57). Gray boxes identify relations between concepts. ↑ means increased; ↓ means decreased. BOLD, blood oxygen level–dependent.

REINFORCEMENT LEARNING IN SCHIZOPHRENIA

Schizophrenia patients show preserved hedonic responses (60), which is not surprising from a dopaminergic perspective, as dopamine is not involved in hedonics (61).

Disturbances in Reinforcement Learning and PE Signaling

Impaired Go Learning and Blunted PE Signaling in Medicated Patients

Medicated schizophrenia patients exhibit impaired Go learning but preserved NoGo learning (62–66). Further supporting an impairment in Go learning, medicated patients fail to learn to speed up for cues for which faster responses give greater rewards (56,63,67). The impairment in Go learning correlates with negative symptoms (62,63,66), which seems intuitive: impaired learning from rewards with preserved learning from punishments could produce avolition (60). Consistent with the impairment in Go learning, medicated patients show blunted neural responses for positive PEs in the striatum, midbrain, and other limbic regions (53,60,68,69), which correlate with negative symptoms (60,68).

Consistent with spared NoGo learning, medicated patients show normal activity for negative PEs induced by reward omission (68) and, in extrastriatal areas, even show increased activation for losses (70). However, medicated patients show reduced aversive Pavlovian conditioning (51,52) and blunted activity for PEs elicited by aversive stimuli (52)—findings that may reflect the possible involvement in aversive conditioning of phasic responses in a subset of dopamine neurons (71–73).

Impaired Go Learning and Blunted PE Signaling Induced by Antipsychotics

In short, medicated patients have impaired Go learning, blunted neural activation for reward PEs, reduced aversive conditioning, and blunted neural activation for aversive PEs. Whether these effects are related to schizophrenia or to antipsychotics is unclear, however, because antipsychotics produce all of these effects (41,46–48,74–76).

As noted previously, reinforcement-learning disturbances in medicated patients correlate with negative symptoms. Anti-psychotics cause effects akin to negative symptoms (14,77,78), so they could be a common cause of reinforcement-learning disturbances and some forms of negative symptoms in medicated patients; this would help to explain the correlation between these two disturbances. Of course, not all negative symptoms are caused by antipsychotics: negative symptoms have been recognized since before antipsychotics existed (79). In fact, antipsychotics modestly improve negative symptoms (80), but that improvement seems to be in secondary negative symptoms, so it may result from improvements in positive symptoms (81).

Impaired Go Learning and Blunted PE Signaling in Unmedicated Patients

Studies in unmedicated patients provide some evidence for impaired Go learning—specifically, reduced learning from rewards (82) and reduced speeding up for cues for which faster responses give greater rewards (67)—and for blunted VS and midbrain activity for PEs (53,82). However, the number of studies is too small to support robust conclusions.

Other evidence also suggests that these deficits may relate to schizophrenia rather than just to antipsychotics. In controls, methamphetamine, at psychotogenic doses, impairs learning from rewards and blunts VS PE signaling (83). Also in controls, increased dopamine synthesis in the VS is associated with blunted VS PE signaling (84) and with aberrant learning for neutral stimuli, with blunted VS PE signaling correlating with aberrant learning (85). These findings suggest a possible association between increased VS synthesis, blunted VS PE signaling, and aberrant learning, all of which are found in schizophrenia.

Relation to Dopamine Function

Increasing dopamine generally improves Go and impairs NoGo learning or performance (28,30). The findings of impaired Go learning and blunted reward PE signaling in unmedicated patients therefore seem difficult to reconcile with a simple striatal hyperdopaminergia hypothesis. Reduced adaptive dopamine transients for relevant cues and outcomes, however, would explain all of the findings reviewed previously: impaired Go learning and aversive conditioning, and blunted activation for positive and aversive PEs (Figure 5).

Figure 5.

Blunted adaptive dopamine transients in the striatum explain several neural and behavioral laboratory findings in schizophrenia that correlate with negative symptoms and help to explain negative symptoms themselves. Blunted adaptive dopamine transients (green) have specific effects on computational variables (orange-brown) that, in turn, cause specific neural and behavioral disturbances that have been found in the laboratory in schizophrenia (blue, with numbers in parenthesis referring to relevant citations). In real life, the same alterations in the computational variables likely cause decreased valuation of stimuli and events (blue-red gradient), which, in turn, causes at least some forms of primary negative symptoms (red). The same disturbances can also explain Parkinsonism associated with schizophrenia (dotted red). In more detail, blunted adaptive dopamine transients (i.e., blunted transients for relevant stimuli and outcomes) cause blunted prediction error (PE) signaling, which has been observed in schizophrenia in many studies (52,53,60,68,69,82). According to Equation 1 (Box 1), reduced PE signaling causes reduced value learning, which, given that the ventral striatum (VS) represents value (32), in turn causes reduced VS activation during reward anticipation, as has also been observed in schizophrenia in many studies (86–92). Some of the findings of reduced VS activation during reward anticipation could also be due to the blunted PE signaling (dashed arrow) because, with learning, PEs move from outcomes to the cues that predict them (12), and the blood oxygen level–dependent (BOLD) response to the cue could extend into the reward-anticipation period. In real life, reduced value learning could lead to reduced valuation of stimuli, events, and situations, possibly contributing to negative symptoms. According to Equation 3 (Box 1), reduced PE signaling also causes reduced direct-pathway (Go) learning, thereby leading to reduced learning from rewards, as has also been observed in multiple studies in schizophrenia (56,62–67,82). In real life, the impaired Go learning may lead to reduced learning to perform actions that lead to positive outcomes, which, especially in the face of preserved indirect-pathway (NoGo) learning, may contribute to negative symptoms. Decreased Go learning may also lead to Parkinsonism, which, despite being commonly associated with antipsychotics, is associated with schizophrenia even in antipsychotic-naive patients (148). According to Equations 6–8 (Box 1), adaptive transients that occur when reward-predicting cues are presented amplify Go signals (i.e., increase β_G) and reduce NoGo signals (i.e., reduce β_N). As a result, positive values are given more weight than negative values, facilitating (a) choice of rewarding options, (b) effortful responses for reward, and (c) fast, invigorated responding (30). Blunted adaptive transients cause a reduction of these effects, leading to (a) difficulties choosing rewarding options, which may contribute to the observed deficits in choice after Go learning, (b) decreased tendency to make effortful responses for reward, and (c) longer reaction times, all of which have been found in schizophrenia (56,62–67,82,100,107,109). Indeed, in animals, inhibiting dopamine-neuron firing during choice decreases choice of rewarding actions (149), and dopamine-neuron firing for a reward-predicting cue correlates negatively with reaction times (150). Increased reaction times may also be caused by reduced Go learning. The decrease in adaptive transients for reward-predicting cues may be further compounded by the decreased value learning, which will make those cues have lower value and therefore elicit smaller PEs (whose signaling will then itself be reduced even further because of the blunted PE signaling). The fact that blunted adaptive dopamine transients may be the common cause of all of the depicted laboratory-based deficits (blue boxes) and at least some forms of primary negative symptoms (red box) explains the widely replicated correlations between these laboratory deficits and negative symptoms (60,62,63,66,68,86–88,100). Gray boxes identify relations between concepts. ↑ means increased; ↓ means decreased.

Blunted VS Activation During Reward Anticipation

Findings

VS activation during reward anticipation is blunted in drug-naive patients, unmedicated patients, patients on first-generation antipsychotics, unaffected patient siblings, and healthy controls high on psychotic-like symptoms (86–94). Reduced VS activation correlates with increased negative symptoms, even in unmedicated patients (86–88,93,94).

Relation to Dopamine Function

VS activation during reward anticipation relates positively to VS dopamine (95–97), so blunted VS activation is difficult to reconcile with a simple striatal hyperdopaminergia hypothesis. Reduced PE signaling (i.e., reduced adaptive dopamine transients), however, explains straightforwardly the blunted VS activation during reward anticipation, through two mechanisms (Figure 5). First, given that PEs also occur upon presentation of reward-predicting cues (32), reduced PE signaling would directly cause blunted VS activation upon cue presentation. Second, reduced PEs would cause reduced value signals (Equation 1 in Box 1); given that the VS likely represents value (32), the reduced value signals would produce blunted VS activation during reward anticipation. Interestingly, amphetamine administered to healthy participants also reduces VS activation during reward anticipation (98) [but see O’Daly et al. (99)] and blunts PE signaling and value representations (83).

Reduced Willingness to Expend Effort for Rewards

Findings

In tasks that assess willingness to exert efforts for rewards, medicated schizophrenia patients choose high-effort options less often than controls do, specifically in high-reward conditions, to an extent that correlates with negative symptoms (100). Antipsychotics decrease high-effort choices (101), so whether these findings are attributable to medication remains unclear. One study, however, found the same effects in a small subsample of unmedicated patients (102).

Relation to Dopamine Function

Reduced adaptive transients would explain patients’ reduced tendency to choose high-effort options for high rewards (Figure 5). Phasic dopamine release upon presentation of cues that indicate high reward availability amplifies striatal Go relative to NoGo values (Equation 9 in Box 1; Figures 2A and 3D), invigorating behavior and emphasizing benefits over costs (30). Blunted cue-evoked dopamine transients would produce less amplification of Go relative to NoGo values and therefore less tendency to choose high-effort options. This effect would be especially noticeable with high rewards, as is indeed reported in schizophrenia (100), because high rewards would cause substantial invigoration in controls but not in patients.

DELUSIONS AND HALLUCINATIONS: ABERRANT GATING OF THOUGHTS AND PERCEPTS

How can increases in striatal spontaneous dopamine transients or tonic dopamine cause psychosis? One hypothesis suggests that inappropriately timed dopaminergic signals assign aberrant incentive salience (61) to external and internal stimuli and events (14). The equivalent idea under our computational conceptualization is that spontaneous dopamine transients assign aberrant value to irrelevant stimuli, events, thoughts, percepts, and other external and internal experiences (Figure 4). Value and incentive salience, however, depend mostly on dopamine in the limbic, not associative, striatum (97,103). Go/NoGo gating in the cognitive domain provides a mechanism linking dopamine specifically in the associative striatum to delusions and hallucinations (Figure 4). Concretely, high tonic dopamine could cause Go gating of aberrant thoughts and percepts; alternatively, or additionally, spontaneous dopamine transients could reinforce aberrant gating. Furthermore, the high frequency of spontaneous transients could mean that the more an aberrant thought or percept is gated, the more it is reinforced, thereby crystallizing delusions and hallucinations.

Some preliminary evidence supports this gating hypothesis. Specifically, dopamine infusion into the caudatoputamen activates auditory cortex, via striato-pallido-thalamo-cortical projections, thereby demonstrating how excessive striatal dopamine could cause auditory hallucinations (104,105). Furthermore, coinfusion of a D₂ antagonist prevents the dopamine-induced activation of auditory cortex (104).

INCREASED SPONTANEOUS DOPAMINE TRANSIENTS VERSUS INCREASED TONIC DOPAMINE

Thus far in the article, increased spontaneous dopamine transients and increased tonic dopamine explained the same findings, making it difficult to adjudicate between them. Tonic dopamine and spontaneous transients may both be increased in schizophrenia; indeed, tonic and phasic dopamine may correlate positively because possibly only neurons that are firing tonically can be recruited to burst-fire (106). However, the hypothesis that schizophrenia involves increased spontaneous dopamine transients seems more consistent with the existing evidence than the hypothesis that it instead, or additionally, involves increased tonic dopamine.

Increased spontaneous transients explain directly, through reinforcement-learning equations, the increased behavioral, autonomic, and neural responses to neutral stimuli and neutral PEs (Figure 4). Increased tonic dopamine explains the neural findings only under the assumption that it increases striatal gain. Tonic dopamine does increase the gain (excitability) of Go MSNs, but it decreases the excitability of NoGo MSNs (27,29). Striatal blood oxygen level–dependent responses would therefore have to reflect mostly activation of Go neurons to be amplified by tonic dopamine: a possible but untested assumption.

The hypothesis that schizophrenia involves increased tonic dopamine is also at odds with some evidence. Increased tonic dopamine would amplify Go relative to NoGo striatal representations (Equation 9 in Box 1; Figures 2A and 3D), which would (a) increase effort; (b) increase vigor, reflected, for example, in reduced reaction times; and (c) increase weighting of positive values, thereby increasing discriminability between choices with different positive values (30). Schizophrenia patients show the opposite effects: (a) decreased effort (100); (b) psychomotor slowing (107); and (c) reduced weighting of, and ability to discriminate between, positive (and negative) values (108,109). Furthermore, increasing tonic stimulation of striatal dopaminergic receptors (e.g., with dopamine agonists in Parkinson’s disease) impairs NoGo learning (110,111), which in schizophrenia is preserved. Finally, increased tonic dopamine would not explain the formation of specific, recurrent delusions and hallucinations.

CLINICAL IMPLICATIONS

Effects of Antipsychotics on Positive and Negative Symptoms

As discussed previously, schizophrenia involves impaired Go learning and blunted PE signaling, which may relate to negative symptoms, and antipsychotics may aggravate these reinforcement-learning deficits and some negative symptoms. Indeed, antipsychotics, administered chronically, reduce dopamine-neuron firing (112), so they may blunt adaptive dopamine transients, in addition to blunting their postsynaptic effects through D₂ blockade. The consequent aggravation in reinforcement-learning deficits and some negative symptoms may help explain the poor adherence to antipsychotic treatment. Blunting of dopamine transients, however, may be precisely what improves positive symptoms—albeit by reducing spontaneous transients.

Treating Negative Symptoms

If some negative symptoms are caused by blunted adaptive dopamine transients, increasing phasic dopamine could improve negative symptoms. Indeed, low or moderate doses of psychostimulants, which increase adaptive transients (Figure 1C, third panel), and low doses of amisulpride, which increase phasic dopamine by preferentially blocking D₂ autoreceptors (113), may improve negative symptoms (113–115). These treatments, however, may also increase spontaneous transients, aggravating positive symptoms. Indeed, the amount of amphetamine-induced dopamine release correlates with both improvement of negative symptoms and aggravation of positive symptoms (7).

Earlier in the article, we used the findings that high doses of psychostimulants cause both increased spontaneous and decreased adaptive transients (Figure 1) as proof that these disturbances can coexist. That does not necessarily imply, however, that the mechanism that causes these disturbances is the same in schizophrenia and with high doses of psychostimulants. If it were, even low doses of psychostimulants might aggravate negative symptoms, as patients might already be in a “high-psychostimulant-like state” (Figure 1C, right panel). Interestingly, amphetamine has in fact sometimes been reported to aggravate negative symptoms (116). Conceivably, psychostimulants may ameliorate or aggravate negative symptoms depending on whether, in a given patient, they increase or decrease adaptive transients, respectively—which, in turn, could depend on the mechanism underlying blunted adaptive transients in that patient.

Substance Use Disorders and Schizophrenia

Self-medication for decreased adaptive transients and their associated negative symptoms may explain the high prevalence of substance use disorders in schizophrenia (117). Repeated substance use may increase phasic dopamine signals for relevant stimuli and outcomes (118,119), which could explain the association of substance use disorders with reduced negative symptoms (120). Unfortunately, all drugs commonly abused by schizophrenia patients increase spontaneous burst-firing in dopamine neurons and spontaneous striatal dopamine transients (11,118,119), which likely explains their association with increased positive symptoms (120) and why substance use disorders increase risk for schizophrenia (117).

Time Course of Action of Antipsychotics

Antipsychotics cause quick improvements in positive symptoms (121) that then continue to build up. An explanation for this combination of immediate and gradual effects is suggested by studies showing that D₂ blockade affects both performance, which leads to immediate effects, and learning, which leads to gradually accumulating effects (30,39,122). D₂ blockade increases both activity and plasticity in NoGo MSNs (39,123,124), which are involved in NoGo performance and learning, respectively (27,28,30). In the motor domain, the effect on performance immediately decreases the tendency for action (39,123,124); the effect on learning additionally causes gradually learned inaction (39,122), consistent with the progressive aggravation of Parkinsonism in antipsychotic-treated patients (125). These ideas extend naturally to psychosis, under the hypothesis that positive symptoms correspond to excessive gating (excessive Go) of abnormal thoughts and percepts. Specifically, antipsychotics may immediately reduce the gating of psychotic symptoms by increasing NoGo activity and gradually decrease such gating further through NoGo learning.

RELATION TO OTHER DEFICITS AND NEURAL SYSTEMS

We have focused on the role of disturbances in striatal dopamine in schizophrenia. Others have explored computationally the role of other biological disturbances (126–132). Hierarchical predictive-coding models generalize some of the issues we addressed (Box 2).

Box 2. Hierarchical Bayesian Predictive-Coding Models.

Hierarchical Bayesian predictive-coding models provide a generalization of some of the issues we addressed. These models generalize the notions of expectation and prediction error (PE) into a general theory about the hierarchical organization of the brain, in which top-down glutamatergic projections from higher to lower cortical areas signal expectations and bottom-up glutamatergic projections from lower to higher areas signal PEs (132). These models address a broad range of findings that suggest that disturbances in the formation or use of expectations and in the signaling of PEs are prevalent in schizophrenia in domains that extend beyond reinforcement learning (129–131,144). One theory, derived from work with ketamine-induced psychoses, suggests that N-methyl-D-aspartate (NMDA) receptor hypofunction might impair the formation and use of cortical top-down glutamatergic expectations (145). Such blunted top-down signaling of expectations could explain various findings in schizophrenia, such as reduced mismatch-negativity signals and reduced sensitivity to perceptual illusions (129,144), and may also contribute to impairments in reinforcement learning (66). Based on microdialysis findings in rats that ketamine increases glutamate in prefrontal cortex (146), this theory further suggests that schizophrenia might also involve excessive and dysregulated cortical bottom-up glutamatergic signaling of PEs (through alpha-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid [AMPA] receptors), which in turn could cause aberrant percepts and aberrant gating of irrelevant information (145). Interestingly, and consistent with our proposal that schizophrenia involves biological disturbances akin to those caused by high doses of psychostimulants, amphetamine also increases glutamate in multiple regions, including the frontal cortex, and induces various alterations in glutamatergic receptors in those regions (147).

The disturbances in striatal dopamine could originate in upstream brain regions or cognitive processes. For example, schizophrenia patients have deficits in pattern separation (133), possibly due to hippocampal disturbances (134), and in working memory (135), possibly due to prefrontal hypodopaminergia (136,137) and associated hypofrontality (138). These deficits could make keeping track of stimuli and contingencies difficult, leading patients to generalize inappropriately across stimuli, which could explain the reduced responding to relevant stimuli and increased responding to neutral stimuli. Indeed, schizophrenia patients overgeneralize (139), and some of their reinforcement-learning deficits may be due to working-memory disturbances (140). Patients’ impairment in explicitly reporting cue-outcome contingencies (55,56) further points to cognitive difficulties. Prefrontal hypodopaminergia could itself cause disturbances in striatal dopamine (137). Conversely, the striatal dopaminergic disturbances could cause these cognitive deficits: increased spontaneous and decreased adaptive transients could cause increased gating of irrelevant and decreased gating of relevant information, respectively, into working memory and possibly into episodic memory.

Patients may also be impaired in representing expected value (60,66,108,109) because of working-memory or orbitofrontal cortex (OFC) disturbances. Indeed, patients are impaired in value-based choices even in tasks without learning (108,109). They are also impaired in using a model of task space in a reversal-learning task (82), an OFC-dependent function (141). Difficulties representing values may sometimes account for impairments in Go learning (66). However, OFC value representations influence (142), and are influenced by (45,143), dopaminergic signaling. Furthermore, psychotogenic psychostimulant doses—which cause dopaminergic disturbances similar to those that we suggest underlie schizophrenia—disrupt the representation of expected value in ventromedial prefrontal cortex (83).

As an exclusive explanation, cognitive disturbances imply nonspecific impairments that are inconsistent with findings of specificity in schizophrenia (e.g., reduced Go learning and activation for positive PEs with preserved NoGo learning and activation for negative PEs). Furthermore, explanations that do not postulate a deficit in PE signaling run into a difficulty: mathematically, PE and value should correlate negatively (Equation 2 in Box 1), so accounts that explain decreased signaling for one generally will predict increased signaling for the other. Blunted signaling of both, as in schizophrenia, can, however, be explained by assuming that the primary deficit is blunted PE signaling, which causes reduced value learning (Equation 1 in Box 1; Figure 5).

CONCLUSIONS

The hypothesis that schizophrenia involves increased spontaneous transients and reduced adaptive transients explains multiple findings (Figures 4 and 5) and makes novel predictions (Supplement). Increased spontaneous transients explain many findings that correlate with positive symptoms and may help explain positive symptoms themselves (Figure 4); reduced adaptive transients explain many findings that correlate with negative symptoms and may help explain primary negative symptoms themselves (Figure 5). Postulating these two dopaminergic disturbances does not violate parsimony because high psychostimulant doses, which are psychotogenic, cause these two disturbances (11,13); furthermore, several mechanisms could explain their coexistence in schizophrenia (Supplement). For example, disturbances in Ca_v1.2 channels may increase spontaneous transients and decrease adaptive transients or disrupt other mechanisms necessary for reward learning (Supplement).

Our account has important implications for treatment. Many drugs, including antipsychotics, likely affect spontaneous and adaptive transients similarly, so they may have opposite effects on positive symptoms and primary negative symptoms. Escaping this predicament may require independently affecting spontaneous versus adaptive transients.