Movement vigor as a reflection of subjective economic utility

Abstract

To understand subjective evaluation of an option, various disciplines have quantified the interaction between reward and effort during decision-making, producing an estimate of economic utility, i.e., the subjective ‘goodness’ of an option. However, variables that affect utility of an option also influence vigor of movements toward that option, i.e., reaction-time plus movement-time. For example, expectation of reward increases speed of saccadic eye movements, whereas expectation of effort decreases this speed. These results imply that vigor may serve as a new, real-time metric with which to quantify subjective utility, and that control of movements may be an implicit reflection of the brain’s economic evaluation of the expected outcome.

Utility and its limitations

The concept of utility, defined as the subjective valuation of an outcome, has intrigued investigators in multiple fields of research. Economists have quantified utility so to produce greater good via public policy, while neuroscientists have estimated utility so to understand the neural basis of decision-making. To measure utility, both fields have relied on behavioral data, as manifested via choices that subjects make. However, choice data describe ordering of preferences, not degree of preferences. That is, knowing that one prefers A to B does not allow one to assign a numeric scale that reflects subjective value of A with respect to B. One way to circumvent this limitation is to use choice lotteries, where probability is used as a scale with which to quantify utility (Fig. 1, Key Figure). However, this approach is also problematic because a person’s beliefs can subjectively distort perceptions of objective probability. Given these limitations, is there a behavioral measure other than choice that can be used to infer utility?

Fig. 1. Key Figure.

Hypothetical relationship between utility, choice, and movement vigor. Utility: The utility function for apples (the function U (a), shown in blue, where a is number of apples, is concave, exhibiting diminishing marginal returns. This means that the more apples one has in hand, the smaller is the utility of an additional apple. The utility of one apple is determined by finding the risky choice that an individual has equal preference for (the indifference point). For example, in this subject, for one apple the equivalent risky lottery is a 50% chance of three apples. That is, U(1) ≈ 1/2U(3). If we assume that U(1) =1 utils, then we infer that U(3) =2 utils. For 3 apples, we observe U(3) ≈1/2U(10). From this we infer that U(10) =4 Similar indifference points are obtained for all possible outcomes (# of apples) and the resulting curve yields the utility function. Vigor: defined as the reciprocal of the time it takes to reach to a given number of apples (reciprocal of the sum of reaction and movement time). On each trial, some number of apples is presented at an eccentric location, resulting in reaching movement. Reaches toward two apples will be faster than toward one apple, but the margin will decrease as the number of apples increases. Not only would reaches be more vigorous towards the higher utility option, but the difference in vigor between options should scale with the difference in utility.

Recent research has revealed that factors that affect preferences, such as reward and effort, also affect movements. For example, if you prefer A to B, then you are likely to move with a shorter reaction-time and greater velocity towards A. Because vigor, defined as reaction-time plus movementtime, is a continuous variable (in contrast to discrete), it can provide a real-valued scale to quantify the difference between movement toward A vs. B. Therefore, vigor adds a continuous dimension to the ordinal scale of choices, with the intriguing possibility that this dimension may overcome some of the limitations inherent in inferring utility from choice behavior. Here, we summarize experimental data regarding effects of reward and effort on movement vigor, and then ask whether vigor can serve as a proxy for utility.

A brief history of utility

Theories in behavioral economics were often developed in reaction to paradoxes or contradictions that were noted in real-life choices made by humans. Bernoulli, for instance, one of the pioneers in this area, was intrigued by the observation that people did not always try to maximize their monetary gain. To account for this, he introduced the concept of maximizing an abstract quantity called utility [1]. Samuelson [2] outlined the mathematical limitations of this approach, demonstrating that utility could not be viewed as an indicator of preference on a numeric scale, but rather an indicator of relative preference along an ordinal scale. von Neumann and Morgenstern [3] suggested that a rational person should try to maximize the expected value of utility. They proposed that in order to compute a utility, one could proceed by observing choices between combinations of outcomes and probabilities. Critically, this approach relied on the assumption that individuals, in their internal models of the world, could objectively represent probabilities. However, a number of experiments that were performed to test this theory contradicted its predictions [4,5], demonstrating that humans did not always make choices that maximized expected utility. Rather, people seemed to rely on subjective estimates of probability that were not always aligned with the objective ones. Thus, a revision was introduced by Kahneman and Tversky [6] termed Cumulative Prospect Theory, an approach that incorporated empirical concepts such as loss aversion, reference point, and probability transformation.

Despite these advances, the problem remained that utility, as estimated via measures of choice, is confounded by subjective estimates of probability (the subjective estimate of 10% probability may not be equal to 10%). Furthermore, as shown in Fig. 1, estimating utility requires repeated assay of preference between various options. As a result, while subjective utility is by definition the major factor behind decision-making, it remains difficult to measure.

Increased reward increases movement vigor

As scientists began to study the neural basis of decision-making, they used movements as a read out of the decision-making process. For instance, in animal studies employing eye movements as the choice-reporting method, if the animal preferred A to B, it expressed its choice by making a saccade toward A. However, it soon became clear that movements were more than just a proxy for the choice. Rather, humans and other animals moved faster and with a shorter reaction-time toward items that they valued more. Because many factors that affected utility of an action also affected the vigor with which that action was performed, the results raised the possibility that vigor could serve as an additional scale with which utility may be inferred.

For example, Kawagoe et al. [7] flashed a target light, and then after a delay, instructed a monkey to move its eyes to the remembered location of the target. In a given block of trials one out of four locations was paired with reward (juice). Saccade peak speed was higher, and reaction-time was shorter, when that movement was toward a rewarding target.

In the real world we do not make saccades to earn juice. Rather, we move our eyes to orient our fovea, where visual acuity is maximal, towards a specific region of the visual scene, thereby allowing the brain to harvest information from that region. When given a choice of two images, a face or an object, people tend to prefer the face image, choosing it first and spending greater time looking at it [8]. Saccades are also faster toward facial images, and people are willing to pay a greater effort cost to view those images [8,9] (Fig. 2A). In this sense, facial images have greater utility.

Fig. 2.

Saccade vigor is modulated by both expected reward and expenditure of effort. A. (Left panel) As subjects fixated a cued location on one side of a screen, an image was briefly flashed on the other side of the screen. Upon removal of the fixation point, subjects were to saccade to the new location at which time the image would reappear. (Right) Saccade peak velocity is higher in anticipation of viewing a facial image as compared with image of noise. B. Subjects made out-and-back reaching movements in four different directions and were either predictably rewarded or not rewarded for each trial. Reaction-times (left) and reach speeds (right) were faster when reaching in a rewarded direction compared to when that same direction was not rewarded. Error bars are between-subject whereas change is within-subject. From [10]. C. Expectation of effort increases reaction-time. With increasing target distance, reaction-times for the arm and eye increased. In all graphs, curves are averages across subjects. Shaded areas or bars represent SEM. From [15].

Increased utility also affects reaction-time and velocity of reaching movements. Summerside et al. [10] asked subjects to reach to one of four quadrants, one of which was associated with reward. There were no instructions or time limits to complete the reach, and because the target was very large, there was only minimal dependence of reward on accuracy of the movement. Reaches toward rewarding targets had shorter latency and higher peak speed (Fig. 2B). In a more ecologically relevant experiment, Sackaloo et al. [11] asked volunteers to reach for a candy bar and found that subjects reached faster toward their favorite candy.

When rewards are probabilistic, saccade peak velocity and reaction-times are modulated by the expected value of reward. Milstein and Dorris [12] controlled expected value of reward via the probability of stimulus location, and its reward value. They found that when a stimulus had a large expected value, saccades toward the stimulus had a shorter reaction-time. Seideman et al. [13] trained monkeys to make a saccade under reward uncertainty. As the probability of reward increased, so did the saccade speed that indicated the animal’s choice. Thura et al. [14] found that in the context of high expected rate of reward, deliberation time was shorter, and saccade peak velocity was higher.

These results suggest that a critical factor that affects movement vigor is the reward that the individual expects to harvest following completion of that movement. Generally, as reward increases, animals exhibit a shorter reaction-time and a greater movement speed. The real-world implication is that if you see a dear friend across the street, the utility for the steps you are about to take toward your friend is higher than if you see someone that you may not be so fond of. As a result, you will walk faster toward your friend, arriving there earlier.

Increased effort decreases movement vigor

Suppose that a constant amount of reward is promised following completion of a movement. As the movement goal is placed farther away, the effort associated with that movement increases, reducing its utility. Reppert et al. [15] measured eye and head movements of people as they reached toward visual stimuli and found that as target distance increased, saccade and reach reaction-times both increased (Fig. 2C). Rosenbaum [16] asked volunteers to put their right and left index fingers at a start location and then await instructions on where to reach. Instructions regarding the target were provided at the start location. Reaction-times were longer when acquisition of reward required a greater travel distance.

Stelmach and Worringham [17] instructed people to produce isometric forces and found that reaction-times became longer when the required force magnitude was larger. Ivry [18] instructed subjects to produce a constant force but for variable periods of time. Reaction-time increased as the expected duration of force production increased. Therefore, reaction-times were longer when acquisition of reward required a greater amount of force production.

A simple way to modulate the effort required for reaching is via varying the mass that one has to carry. The human arm has a mass distribution that resembles a heavy object when it moves in some directions (elbow movements), and a light object when it moves in other directions (shoulder movements). People reach faster when they move their arm in the directions that have a lower mass [19,20].

Effort can also modulate speed of walking. The metabolic cost of locomotion, an objective measure of effort, increases with steeper inclines [21,22], and both humans and horses prefer slower gait speeds at steeper inclines [21,23]. Carrying additional loads also leads to greater metabolic costs [24,25]. Similarly, with added load both humans and horses (and likely other animals) prefer to walk slower [24,26].

Taken together, these data suggest that under constant reward conditions, the brain chooses a longer latency to start a movement toward a high effort stimulus, and makes that movement with a lower speed.

The link between utility and vigor

In principle, why should the utility that the brain assigns to an option affect how the brain controls the movement toward that option? The reason may be that the animal’s choices and movements both contribute to an important currency: the total capture rate, defined as the sum of all rewards acquired, minus all efforts expended, divided by total time. This currency plays a fundamental role in the longevity and fecundity of animals [27], suggesting that living one’s life in a way that increases the capture rate has evolutionary advantages. Because movements require expenditure of effort, and utility driven decisions affect reward accumulation, a policy that maximizes the capture rate seems a natural way to coordinate motor control with decision-making [8,28]. That is, the link between movements and decision-making arises because both are elements of behavior that the brain must control to maximize a single currency: the capture rate.

Foraging provides a useful paradigm to investigate this possible link. During foraging, animals make decisions regarding how long to stay and accumulate reward from one patch, and then move with certain speed to another patch, abandoning the remaining reward in the patch left behind. A recent report [8] asked whether the decisions that subjects made regarding how long to stay in one patch, and the vigor that they expressed during travel, were driven by a common desire to maximize the capture rate. To formalize the predictions, they applied Marginal Value Theorem [29], which specified how reward and effort should affect decision-making and motor control. The theorem stated that the optimum movement duration was met when the average rate of effort expended during the movement was equal to the negative of the average capture rate experienced by the subject. As a result, if the environment presented a host of high utility patches, then the movement speed should increase, while harvest duration at each patch should decrease. The theory mathematically linked vigor, decision-making, and utility under a single, normative framework.

To test the theory, Yoon et al. [8] varied reward via image content, and effort via image eccentricity, then measured how these changes affected decision-making (gaze duration) and motor control (saccade speed). Following a history of low rewards, people increased gaze duration, lingering longer to gaze at an image, and decreased saccade speed, moving slower to the next reward site. These results extended earlier observations regarding effects of reward rate on harvest duration [30]. In addition, Yoon et al. [8] observed that in anticipation of future effort, subjects lowered saccade speed, and increased gaze duration. The results confirmed many (but not all) of the theory’s predictions, suggesting that both the choices that people made regarding harvest duration, and the vigor with which they moved from one reward site to another, were consistent with an attempt to maximize the total capture rate.

Application of this theory predicts that increased vigor is not an obligatory consequence of increased utility. For example, in an environment where moving slower produces greater reward, utility and vigor will not increase together. However, during natural foraging where rewards are finite and there is competition from other predators, increased reward should accompany increased vigor because increased vigor will lead to increased capture rates.

There are a number of questions regarding vigor as a proxy of utility (see Box 1). For example, when deliberation time is short, visual salience of the stimulus (luminance, color) distorts subjective utility [31,32] (we tend to pick the brightest object, rather than the object we prefer, as measured without time pressure). Under time constraints, visual salience may also influence vigor, but this question has yet to be examined.

Boxes.

Box 1: Is vigor a reflection of utility or motivation?

Decisions are influenced by not only the economic utility of the option, but also its visual salience and motivational value [31,32]. Utility is positive for items we value, but negative for items we avoid. In contrast, saliency is a measure of importance for that item. Saliency signals are greater for stimuli that impact the organism more, having utility that is either highly valuable or highly aversive. That is, on the utility scale, negative values are bad and should be avoided (punishments), whereas positive things are good and should be pursued (rewards). However, on the same scale, saliency is U-shaped: one is motivated and attentive to both the very bad things and the very good things. Distinct regions of the cortex may encode saliency and utility [79]. Is vigor a reflection of saliency, or utility?

Kobayashi et al. [80] trained monkeys to view a cue that on each trial determined what would happen if the animal performed the correct movement: get reward (juice), avoid punishment (airpuff), or simply hear a sound. Motivational requirements were highest for reward and punishment trials, and lowest for neutral trials. In contrast, the juice stimulus had the highest utility, whereas the airpuff stimulus had the lowest utility. Saccade speed was highest for the reward trials, and lowest for airpuff trials. Vigor appeared to be a reflection of utility, not motivation.

Leathers and Olson [66] trained monkeys to associate cues to reward and penalty. Once the cues disappeared, the monkey made its choice by making a saccade to where the preferred option was located. The large penalty possessed lower value than a small penalty, but was emotionally more potent. A small reward possessed higher value than a large penalty, but was emotionally less potent. Neurons in lateral intraparietal area (LIP) responded not with the utility of the stimulus in their response field, but with its emotional valence. However, unlike the LIP neurons, stimuli that had negative utility produced saccades that had lower reaction-times. Thus, in contrast to LIP neurons, saccade reaction time was modulated by stimulus utility, decreasing on trials when large reward was at stake.

Together, these results suggest that vigor is largely a reflection of the utility of the act, and not merely a reflection of the motivational properties of the stimulus that directs the act.

Outstanding Questions

• How does vigor vary with economic properties of the option, i.e., the magnitude, risk, and delay to harvest of reward and expenditure of effort?

• How does vigor vary with non-economic properties of the option such as visual salience or its emotional and motivational content?

• Is movement vigor affected by cognitive effort as well as physical effort?

• Do between-subject differences in vigor reflect subjective differences in evaluation of reward and effort?

• Is vigor a trait-like feature of individuality? Do people who reach fast also tend to walk fast? Do people who speak rapidly also tend to have fast saccades?

• How does vigor vary with the history of reward and effort? Are there differential effects of decreasing utility through low reward vs. high effort, or increasing utility through high reward vs. low effort?

• Does change in utility affect all movements, even those not relevant to the current task? For example, following high reward rate in a saccade task, do reaching movement show high vigor?

• What is the nature and degree of connectivity between regions in prefrontal and parietal cortices that encode subjective value, and regions in basal ganglia that encode vigor of movements?

• How does vigor vary with resource availability? Animals become increasingly risk-seeking as they approach starvation. Does vigor reflect a changing preference for risk with resource constraints such as fatigue?

Vigor as a real-time proxy for utility during decision-making

Suppose you are at the supermarket and are trying to decide between two different jams. As you stand there looking at the jars, you make saccades that bring your eyes from one option to another. After a few seconds, you decide which one you prefer, and then you reach and pick up your chosen item. As you deliberate, does the velocity with which you move your eyes toward each stimulus reflect the real-time utility that your brain assigns to that option?

Reppert et al. [33] measured eye movements as people considered two options: a small amount of money that they could have immediately, and a larger amount that they could have later (Fig. 3A). As subjects deliberated, they moved their eyes from one stimulus to another, and then indicated their choice by pressing a key (Fig. 3B). Saccade peak speed was high during the deliberation period, and then dropped right after decision-time. That is, movement speed was higher if the goal location contained information that was needed for the purpose of decision-making. Importantly, the last saccade just before the key-press had higher speed if it was toward the more valued option (Fig. 3C). Around decision-time, the difference in saccade speed toward the immediate option relative to the delayed option was positively correlated with the difference in the subjective values of the two options. Therefore, the difference in the subjective utility of two options was correlated with the difference in saccade speed toward the two options.

Fig. 3.

Vigor may be a real-time proxy for utility during deliberation. A. Volunteers made choices between a small, immediate monetary reward and a larger, but delayed reward. Options were presented on a computer screen; as subjects made their choices, their eye position was tracked. B. Eye position traces during a typical trial, demonstrating saccade patterns. The subject made saccades between the options, and at some point indicated her choice (red arrow). C. (Left panel) Saccade velocity relative to baseline was elevated as the decision time approached, and abruptly decreased after the decision. (Right panel) Saccades to the preferred option were faster than saccades to the non-preferred option (as measured immediately before and after the time of decision). D. The difference in saccade velocity between the immediate to delayed reward (delayed – immediate) correlated with the difference in utility between the delayed and immediate reward. From [33].

Between-subject differences in vigor

Among healthy people there is diversity in vigor: some people tend to consistently move rapidly, while others tend to move slowly, as evidenced by their saccades [15,34–36], and their reaching movements [15] (Fig. 4A). In fact, the typical trajectory of the eyes during a saccade is so specific to each person that it can be used as a form of identification [35]. Those who make faster eye movements tend to have shorter saccade reaction-times, and those who make faster arm movements tend to have shorter reach reaction-times (Fig. 4B). There is a strong positive relationship between vigor of arm and head movements (Fig. 4C). That is, individuals who move their arm fast tend to also move their head fast [15]. This suggests that vigor may be a trait, cutting across modalities of motor control.

Fig. 4.

Between-subject differences in vigor. A. Eye velocity traces during saccades in the horizontal direction in two young adults (left). Subject 4H consistently moved his eyes with greater velocity than subject 16P. Data from [34]. (Right and middle) Head and hand velocity profiles during reaching from two other subjects. The targets were placed across the midline at distances of 15–18 cm. Shaded areas are SD over all trials for each subject. Data from [15]. B. (Left) Velocity profiles for 12–14° horizontal saccades in high vigor and low vigor individuals, selected from the top and bottom quartile of a population of around 300 subjects. (Right) Hand velocity during reaching for individuals with high and low vigor. Shaded areas are SEM. Data from [15]. Individuals with high vigor responded sooner and started their movements earlier than individuals with low vigor. C. Individuals with faster reaching movements also generated faster head movements. Each data point depicts a single subject. From [15]. D,E. Peak horizontal saccade velocity decreases with age after the teenage years, while saccade reaction-time increases with age beyond the teenage years. Error bars are SEM. Data from [55].

Why should there be between-subject differences in vigor? Let us consider three possibilities: differences in biomechanics, differences in movement accuracy, and finally, differences in subjective evaluation of utility.

Biomechanics (limb size, inertia, etc.) differs between individuals, which translates into differences in energy expenditure during movements. One could speculate that the greater the effort it takes to move a body part in a specific individual, the slower that person may opt to move. Berret et al. [37] explored this question in the context of reaching movements. They asked people to point with their arm to visual targets and found that reach speed differed between subjects, but tended be relatively constant within each subject. Once biomechanics-related costs were accounted for, the differences in reach speed persisted. Therefore, vigor difference among people was not due entirely to differences in biomechanics.

Another possibility is that between-subject differences in vigor are due to speed-accuracy tradeoff: perhaps people who move vigorously are sacrificing accuracy for arriving sooner. Reppert et al. [15] considered this hypothesis and found that in both saccades and reaching there was no relationship between vigor and accuracy. That is, people who moved faster were as accurate as those who moved less vigorously.

Another possibility is that vigor differences may reflect between-subject differences in the willingness to exert effort. For example, in a task where people were asked to press a key a number of times for a given amount of money, some preferred the low-reward/low-effort option, while others chose the high-reward/high-effort option [38]. This implies that for a given amount of reward, the degree to which people are willing to exert effort varies between healthy individuals. Importantly, these differences were correlated with between-subject differences in the neural circuits that evaluated reward and effort: individual differences in dopamine binding potential in the striatum and the prefrontal cortex were positively related to the willingness to expend effort [39]. As we will note below, some of these same circuits, particularly in the striatum, are also involved in motor-control, modulating the vigor with which movements are performed [40–42].

The relationship between differences in vigor and willingness to exert effort was further explored by Summerside et al. [43], using a paradigm where walking/running represented low/high effort rates, respectively. The results indicated that some participants chose a gait-speed based on minimizing total effort, whereas others chose a gait-speed that minimized total time. Interestingly, individuals who placed a higher relative value on time preferred to run faster than individuals who placed a higher relative value on effort expenditure.

These limited data raise the possibility that there may be a relationship between how individuals evaluate reward and effort for the purpose of decision-making, and how they evaluate these same variables for the purpose of making a movement. Overall, movement vigor may be an individual trait, that does not seem to reflect a willingness to accept inaccuracy, but rather demonstrates a propensity to expend effort [15].

Age related changes in vigor

One of the most prominent effects of aging is reduced vigor. For example, walking speed declines with age [44]. With aging there is also an increase in the effort requirements of motion: the effort that one must expend in order to walk at a given speed, as quantified via metabolic cost, is higher in the elderly [45,46]. Furthermore, aerobic capacity is lower in the elderly [47,48]. Thus, with age there is both an increase in the absolute energetic cost of walking, and an increase in the energetic cost relative to the maximum capacity.

To understand the mechanism behind these changes, Coen et al. [47] measured skeletal muscle mitochondrial capacity and efficiency in older adults and found that mitochondrial capacity, muscle efficiency, and aerobic capacity all co-varied with preferred walking speed. This raised the possibility that with aging, there were changes in metabolism which produced greater energy requirements for movement. Because movement vigor appears to be a reflection of subjective evaluation of reward and effort [20,49–52], an increase in the objective cost of movement is likely to contribute to reduced vigor in aging [20].

Aging also affects decision-making. For example, one can be offered an amount of money that can be attained soon, and a greater amount that can be attained later. A person with a steep temporal discount function would be unwilling to wait, choosing to take the smaller amount now, and not wait for the larger but delayed reward. Generally, the rate of temporal discounting tends to be highest in the teen years, and then declines with age [53]. In theory, if movements are made in anticipation of reward, then a reduction in the discount rate should make movements slower [49,54]. These age-dependent changes in decision-making are loosely mirrored in age-dependent changes in vigor: saccade velocity is highest in the teen years, and then declines with age (Fig. 4E) [55]. Similarly, saccade latency is lowest in the teen years, and then increases with age (Fig. 4E).

Between-subject differences in decision-making and motor control

The age-related changes in impulsivity and vigor raise the intriguing hypothesis that between-subject differences in vigor may be partly due to differences in how the brain computes utility [49]. A commonly used method to assess decision-making traits is via questionnaires, where one determines the response to queries such as “do you often buy things on impulse”, or “are you good at waiting patiently”. Impulsive people show diminished midbrain D2/D3 auto-receptor availability, which results in increased dopamine release in the striatum [56]. However, people who exhibit high vigor do not appear impulsive in these questionnaires [34,37] . Furthermore, between-subject differences in temporal discounting, as measured in a task where rewards were largely hypothetical, do not correlate with between-subject differences in vigor [33]. Furthermore, movement vigor is not a purely implicit measure of utility, because one can motivate oneself to move faster or slower [57]. However, in a task where subjects had to wait to improve their probability of success, and results of each decision had immediate and real consequences on the reward rate (Fig. 5), people who were less willing to wait exhibited greater vigor in their movements [34]. The question of whether between-subject differences in vigor are related to between-subject differences in subjective evaluation of utility remains to be addressed.

Fig. 5.

Relationship between willingness to wait and movement vigor. A. The trial began with a central fixation spot. Two targets were presented at 20° from fixation along with an instruction at the fixation spot indicating which target was the direction of the correct saccade. In some blocks, there was a 25% probability that following a variable delay period a second instruction would be given, indicating the previously instructed saccade should be canceled. The delay period was adaptively adjusted to the success and failure of the subject on previous trials: success made the delay period 30ms longer. The experiment attempted to measure the length of time the subject was willing to wait to improve their probability of success. B. Schedule of instruction probabilities. C. Willingness to wait, as measured via latency of saccades for two subjects, one with high saccade vigor (subject 4H, also shown in Fig. 4A), and one which low saccade vigor (subject 16P). The vertical lines denote breaks between blocks. Subject 4H (high vigor) is relatively unwilling to wait, whereas subject 16P (low vigor) is more willing to wait to improve probability of success. From [34].

Neural bases of vigor and subjective utility

Is there an overlap between the neural substrates of movement vigor and subjective utility? Correlates of both variables are present in the cerebral cortex, as well as the basal ganglia.

Neurons in the parietal [58–60] and frontal [61–64] lobe have been implicated in encoding subjective utility. For example, during a free-choice task in which two options are available, lateral intraparietal (LIP) neurons exhibit a higher discharge when the stimulus with the larger economic gain is in their movement response field [59]. In contrast, neurons in the orbitofrontal cortex can encode the subjective value of the stimulus without concern for its spatial location or its movement requirements [65]. Some of these interpretations have been challenged by the observation that LIP activity may reflect attention demands of the stimulus rather than its utility [66]. Regardless, the higher discharge associated with higher subjective utility often translates into a faster rise to threshold during the period of deliberation. As a result, greater activity in these neurons is often followed by greater movement vigor [63,67,68].

Correlates of utility and vigor are also present in the basal ganglia [69]. Encoding of subjective utility has been observed in the striatum [7,70], globus pallidus [71], and substantia nigra pars reticulata [72,73]. Notably, manipulation of specific regions of the basal ganglia including the striatum [74,75], ventral pallidum [76], and substantia nigra pars compacta [42] have produced causal modulation of vigor.

In humans diagnosed with Parkinson’s disease (PD), there is deterioration of the dopaminergic system of the basal ganglia. A cardinal symptom of PD is bradykinesia. Mazzoni et al. [77] proposed that individuals with PD were reluctant to move quickly partly because increased speed required greater expenditure of effort, and they were unwilling to expend this effort to acquire the available reward, i.e., the movement disorder was a result of a disorder in economic evaluation of the options. To test this idea, Manohar et al. [78] designed a task where individuals with PD made saccades toward a target with as little latency as possible. The smaller the reaction-time, the more reward they received. As the prospect of reward increased, individuals with PD responded by making more vigorous saccades, but at a response magnitude that was lower than the control group. This reluctance to adjust vigor appeared to indicate a decreased sensitivity to reward.

Control of saccades provides an example of how cortical regions and the basal ganglia cooperate to influence decisions (via computation of utility) and movements (via control of vigor). The cortical regions and the basal ganglia both converge onto the superior colliculus, which in turn projects to the brainstem saccade generator circuit. The frontal eye field and LIP have excitatory projections to the superior colliculus, and their increased activity indicates attention allocation and movement preparation toward regions of space where reward is found and where movements should be made to maximize utility. Basal ganglia projections to the colliculus are inhibitory, and decreased activity indicates attention allocation and movement preparation. The combined effects of these two types of input likely results in the decision of where to saccade to, and how vigorously to perform that saccade. However, more studies are needed to uncover the shared functionality between these areas (see Outstanding Questions).

Concluding remarks

Our choices reflect our subjective valuation of economic variables such as reward and effort. Remarkably, the same variables affect movement vigor: the eyes and the arm move with a shorter reaction-time and greater speed toward stimuli that promise greater reward. This implies that control of movement and control of decision-making are linked in a common economic framework: movements produce effort expenditure, while decisions lead to rewards. The sum of reward and effort, divided by time, defines the total capture rate [8]. When utility of an option increases, moving faster toward that option is a good policy because doing so increases the capture rate. These results raise the intriguing possibility that study of how individuals move toward an option may reveal their implicit preferences for reward and effort, thereby introducing a real-valued scale with which subjective economic utility can be measured.

Highlights.

• Movement vigor is influenced by the magnitude of the reward to be obtained and the effort at stake, suggesting that control of movements is a reflection of the brain’s economic evaluation of the utility of the outcome.

• Historically, utility has been identified based on an individual’s choices, an approach that is limited in its ability to assign a numeric measure of relative preference. Vigor may provide a new scale with which to quantify a real-valued measure of utility.

• During deliberation between various options, saccade vigor can provide a real-time measure of the ongoing evaluation process, reflecting the current utility of each option.

• The basal ganglia, particularly the dorsal striatum, are implicated in the control of both movement vigor and decision making. Altering function of these regions has immediate effects on vigor.

• Movement slowing in older adults and individuals with PD may be related to an altered economic evaluation of reward and effort.

Glossary

Capture rate

sum of all rewards acquired, minus all efforts expended, divided by total time. Capture rate is computed in terms of energy accumulated from rewards, minus energy expended to acquire those rewards (due to movements and other energetic expenditures), divided by time.

Marginal value theorem

A theory that provides a policy regarding how to make decisions [29], and how to make movements [8], in order to maximize the capture rate. The policy specifies that control of movements and decisions requires a comparison between the local rate of reward and effort, and the global rate experienced and expected in the environment.

Temporal discounting

A reward that can be attained at time t is generally more valuable than the same reward delivered at time t+Δ. Temporal discounting refers to the rate at which the value of reward declines as a function of time delay Δ.

Utility

A measure of how much one values a particular good.

Vigor

The reciprocal of the total time it takes for a subject’s movement to arrive at a goal location (i.e., the reciprocal of the sum of reaction-time and movement-time), normalized for distance.