Dopamine Function and the Efficiency of Human Movement

Abstract

To sustain successful behavior in dynamic environments, active organisms must be able to learn from the consequences of their actions and predict action outcomes. One of the most important discoveries in systems neuroscience over the last 15 years has been about the key role of the neurotransmitter dopamine in mediating such active behavior. Dopamine cell firing was found to encode differences between the expected and obtained outcomes of actions. Although activity of dopamine cells does not specify movements themselves, a recent study in humans has suggested that tonic levels of dopamine in the dorsal striatum may in part enable normal movement by encoding sensitivity to the energy cost of a movement, providing an implicit “motor motivational” signal for movement. We investigated the motivational hypothesis of dopamine by studying motor performance of patients with Parkinson disease who have marked dopamine depletion in the dorsal striatum and compared their performance with that of elderly healthy adults. All participants performed rapid sequential movements to visual targets associated with different risk and different energy costs, countered or assisted by gravity. In conditions of low energy cost, patients performed surprisingly well, similar to prescriptions of an ideal planner and healthy participants. As energy costs increased, however, performance of patients with Parkinson disease dropped markedly below the prescriptions for action by an ideal planner and below performance of healthy elderly participants. The results indicate that the ability for efficient planning depends on the energy cost of action and that the effect of energy cost on action is mediated by dopamine.

INTRODUCTION

Previous work suggested that key distinctions in understanding the role of dopamine for control of movement are associated with the regime of dopamine release (tonic vs. phasic) and with neural structures innervated by dopamine projections (ventral circuits vs. dorsal circuits; Schultz, 2007; Grace, 1991). The tonic (sustained) release of dopamine establishes background levels of the neurotransmitter in both the ventral striatal-prefrontal (“ventral”) and dorsal striatal (“dorsal”) circuits. In contrast, the phasic (transient) release of dopamine provides rapid rise and fall of the level of dopamine, which are thought to encode differences between the expected and obtained reward of an action and is associated with synaptic modification and learning (Glimcher, 2011; Schultz, Dayan, & Montague, 1997).

Understanding how the different neural structures and regimes of dopamine release interact in the control of action has been facilitated by computational studies of action planning. From this perspective, movements are viewed as outcomes of a process that selects from a set of candidate movements, each associated with sensory and motor uncertainty, cost, and reward (e.g., Niv, Daw, & Dayan, 2006; Schultz, 2006). The many parameters of each candidate movement are combined into a single variable: expected utility of movement. The movement with the most desired utility is selected for execution. This unifying approach has been helpful in combining results from physiological and behavioral studies, opening new possibilities for understanding the nature of movement disorders.

The computational studies suggested that the motivational role of tonic dopamine is twofold. In the ventral striatum, dopamine determines how vigorously participants perform repeated responses over time (Niv, Daw, Joel, & Dayan, 2007). In the dorsal striatum, dopamine determines the speed of single movements according to their different energetic costs (Mazzoni, Hristova, & Krakauer, 2007). These effects are conveniently summarized in terms of “motivational sensitivity.” For example, in the state of high motivational sensitivity (associated with high levels of tonic dopamine in the ventral striatum), participants are willing to perform energetically demanding series of actions to obtain a small amount of reward that would not elicit action if the sensitivity were low.

This line of thought found a striking confirmation in a recent study of Parkinson disease (PD), which is characterized, among other things, by low levels of tonic dopamine in the dorsal striatum. Mazzoni et al. (2007) hypothesized that bradykinesia (a pervasive slowness of movement typical of PD) was caused by patients’ reluctance to perform the energetically expensive fast movements, rather than that the slowness was a compensation for the low accuracy of movement associated with parkinsonism. The authors tested this hypothesis in patients with mild PD and in healthy participants, all instructed to move at different fixed speeds. Patients were capable of moving with the required speeds, including the high speed, although they performed the fast movements less often than healthy participants. This result supports the view that PD patients prefer slow movements (rather than they are incapable of fast movements) because of patients’ heightened sensitivity to the energetic cost of movement. Mazzoni et al. (2007) proposed that energetic cost of movement can be thought of as a motivational signal for the motor system, encoded by tonic levels of dopamine in the dorsal striatum. In this view, PD patients move slowly because they have decreased “motor motivation.”

The hypothesis that decreased dopamine function increases sensitivity to the energetic cost of movement makes several seemingly paradoxical predictions. It is expected, for example, that movements by patients suffering from severe parkinsonism (and thus a marked decrease in motor motivation) would nonetheless be highly efficient, perhaps as efficient as movements by healthy participants, when the energetic cost of movement is low. (By “efficient” we mean similar to predictions of an ideal planner, introduced below.) This is because the decreased energy cost of the movement would increase the patients’ motor motivation. Testing this hypothesis is complicated by the fact that the greater motor impairment of such patients creates a large gap in motor competence between patients and healthy participants. Here we overcome this difficulty by using an ideal planner framework. We derive benchmarks of performance for every participant by taking into account their individual precision of movement. We then measure performance of each participant against his or her individual benchmark, which allows us to compare performance across a broad range of motor competence.

We studied motor performance in a challenging motor task under different energetic costs of movement, assisted or countered by gravity. We found that, in conditions of low energy cost, the patients performed surprisingly well, in fact similar to prescriptions of an ideal planner and to healthy elderly participants. As energy costs increased, however, the patients’ performance dropped markedly below the optimal prescriptions and below the performance of healthy elderly participants. The results support the notion that tonic dopamine levels control human sensitivity to the energetic cost of movement. The results suggest, moreover, that dopamine innervation mediates the perceived cost of movement beyond which humans do not or cannot optimize their movements fully.

METHODS

Participants

Eighteen participants took part in the experiments: six PD patients (mean age = 64 years, SD = 7.4 years) with mild to moderate idiopathic PD (at Hoehn and Yahr Stages II and III of the disease; Hoehn & Yahr, 1967), six healthy age-matched control participants (mean age = 64 years, SD = 4 years), and six healthy young adult participants (mean age = 25.8 years, SD = 3.5 years).

All participants had normal or corrected-to-normal vision (20/40), and all gave their written informed consent approved by the institutional review board of the University of California at San Diego. PD patients were tested in the morning after having been off medication for at least 12 hr (Defer, Widner, Marie, Remy, & Levivier, 1999).

Before each experimental session, patients were administered the motor scale of the Unified Parkinson’s Disease Rating Scale (Goetz et al., 1995) as well as the Mini-Mental State Exam (Folstein, Folstein, & McHugh, 1975) and Beck Depression Inventory (Beck, Rush, Shaw, & Emery, 1979). Table 1 presents the clinical characteristics of the PD patients, indicating that all participants were non-demented and nondepressed. Other than PD for the PD patients, no participant had any known neurological or psychiatric disorder.

Table 1.

Clinical Characteristics of PD Patients

N	Age (years)	Handedness	Disease Duration (years)	UPDRSa	H&Y Stage	Medications	MMSE/BDI
1	51	R	9	58	3	L; S; A	29/6
2	77	Lb	11	41	3	L; P	24/4
3	63	R	11	38	2	L; P; S	30/12
4	67	R	7	44	3	P; S; A; C	29/2
5	68	R	9	47	3	L; E; Ro; Ar	29/22
6	62	R	6	29	2	L; Ro; R	30/7

UPDRS = Unified Parkinson Disease Rating Scale; MMSE = Mini-Mental State Examination (out of a maximum score of 30), higher scores indicate more intact cognitive function; BDI = Beck Depression Inventory (out of a maximum score of 63), higher scores indicate a greater degree of depression.

Medications: A = amantadine; Ar = artane (trihexyphenidyl); C = coenzyme-q10; E = entacapone; L = carbidopa/levodopa (regular formulation); P = pramipexole; R = rasagiline; Ro = ropinirole; S = selegiline. Handedness: R = right; L = left.

^aMotor Section of the Unified Parkinson’s Disease Rating Scale. Average score across the 3 days of the experiment (out of a maximum score of 108). Higher scores indicate more severe motor impairment.

^bTarget locations are mirror reversed and touch screen shifted to maintain same relation to screen.

Apparatus

The experiment was controlled by a Dell Optiplex 745 computer using the PyGame library for the Python programming language. Participants were seated in front of a 32 in. LCD touch-sensitive monitor (ET3239L, Elo Touch-systems, Milpitas, CA) in a dimly lit room and used a pen-like stylus to perform pointing movements to the screen. A chin rest was used to stabilize head position and maintain a constant viewing distance of approximately 46 cm in the direction normal to the screen. Monitor slant was individually adjusted to make screen surface normal to every participant’s gaze line. Because the temporal resolution of the touch screen was low (19 ± 6 msec for lifting the stylus), we concentrated on spatial measures of participants’ performance.

Stimulus and Procedure

The stimulus consisted of two colored regions: a green “reward” disk and a red “penalty” disk. After choosing a disk size, the disks had the same radius of 9.3 mm (18.75 pixels), 12.5 mm (25 pixels), or 15.6 mm (31.25 pixels). Disk centers were separated by one disk radius, either along the dock-target axis (yielding the “aligned” stimulus condition) or orthogonal to the dock-target axis (the “nonaligned” condition). Thus, three reward–penalty configurations were created for each stimulus location (the upper and lower locations or “heights,” illustrated in Figure 1A). Only one target disk and one penalty disk were presented, simultaneously, on every trial.

Figure 1.

Stimulus design and the expected gain of action. (A) Stimulus configurations superimposed on the touch screen. On every trial, participants performed a rapid movement from a central location (white disk, “dock”) to a stimulus configuration that consistent of two adjacent disks: green (“target”) and red (“penalty”). The monetary rewards associated with hitting the target and penalty are listed in the inset on top left. Stimulus configurations were classified as “aligned” and “nonaligned,” depending on whether or not the target-penalty axis was aligned with the dock-target axis. (B) Expected gain derived by the ideal planner for the stimulus configuration on top right of panel A. According to the ideal-planner model, the optimal behavior is to aim at the point where the expected gain is maximal: away from the penalty disk and off the center of the target disk. (In B, the origin of coordinates is aligned with the center of target disk.)

Participants initiated every trial by touching a white disk (“dock”) of radius 12.5 mm (25 pixels) at the screen center. A stimulus was immediately displayed at one of two heights: lower left (“lower”) or upper right (“upper”) at the average distance of 25 cm from the dock, illustrated in Figure 1A. Importantly, movements to the lower target were assisted by gravity, whereas those to the upper target were countered by gravity and thus had increased biomechanical (energy) cost (d’Avella, Portone, Fernandez, & Lacquaniti, 2006; Papaxanthis, Pozzo, & Schieppati, 2003; Papaxanthis, Pozzo, & Stapley, 1998). Stimulus height was randomized across trials. Moreover, to prevent participants from adopting stereotypical movements, the absolute location of each target was “jittered” slightly across trials (within an area of 8 mm horizontally and 16 mm vertically of mean target position). Stimuli were displayed for 650–950 msec: The durations were selected individually for every participant in preliminary experiments, based on participant’s movement speed (as explained below).

Participants accrued monetary gains and losses by touching the screen within the target and penalty regions (5-cent gain and 10-cent loss, respectively) while the stimulus was present. Touching the region of overlapping target and penalty yielded the sum of corresponding returns (loss of 5 cents). Touching the screen outside the target and penalty regions yielded a zero return. Time-outs led to the loss of 15 cents each. The cumulative score was continuously displayed in the upper left corner of the screen, updated after every trial. The task was to maximize the cumulative winnings.

Participants received visual and auditory feedback. A small disk (radius = 2.5 mm) marked the location where the stylus touched the screen (“movement endpoint”), presented until the participant touched the dock to initiate the next trial. Distinctive tones informed participants whether they had touched the target or penalty region or timed out (failed to touch the screen during the required RT). When the participant timed out, only an auditory feedback was issued; no visual feedback of endpoint position was given. Once the participant returned to the dock, another target immediately appeared. Thus, participants made rapid, sequential reaching movements from the dock to a target, back to the dock, to the next target, and so on. To reduce fatigue, a message was displayed after every 20 trials, reminding participants to take a short break.

The experiment consisted of three sessions, one on each of three consecutive days. Sessions on the first day were used to select appropriate stimulus duration and size for every participant and to thoroughly familiarize participants with the task, such as to eliminate effects of learning and thus establish a stable performance. On the first 2 days, touching the penalty region yielded no monetary loss, but the penalty disk was presented on every trial; the same way it was presented on Day 3, so participants grew accustomed to stimulus appearance. The sessions of the first 2 days consisted of 300 trials per day, using two different stimulus durations and three different target sizes (150 trials each, 50 of each target size of 9.3, 12.5, or 15.6 mm). On Days 1 and 2, every participant was first tested using the duration of 750 msec, after which the duration was increased or decreased by 100 msec depending on performance. If performance was greater than 75% hit rate, stimulus duration was reduced; if less than 75%, it was increased. To ensure that all participants were able to comfortably perform the task but were still challenged, the combination of target size and duration with 70–90% maximum reward on Day 2 was selected for Day 3. Day 3 had 300–400 movements (depending on fatigue levels) using a fixed target size, fixed duration, and the full penalty. One experimental session lasted for 30 min on average.

Endpoint Variability

Endpoints of rapid movements vary from trial to trial, even when participants are trying to repeatedly reach the same point on the screen (“aim point”). We studied consequences of the endpoint variability for the task illustrated in Figure 1A, modeled after Gepshtein, Seydell, and Trommershäuser (2007).

When endpoint variability is much smaller than the target disk, participants could do well by aiming at the target center. When the variability is large, such that the dispersion of endpoints across trials is comparable with the distance between target and penalty disks, some of the endpoints would fall on the penalty disk. To reduce losses, participants shift their aim points away from the penalty disk, effectively directing movements off the target center. An optimal shift minimizes the risks of hitting the penalty and missing the target (Figure 1B; Trommershäuser, Maloney, & Landy, 2003).

Distributions of endpoints in this task are typically elongated in the direction of movement (Figure 2B; Gepshtein et al., 2007; Gordon, Ghilardi, & Ghez, 1994). Therefore, the shift of the aim point away from the target center is expected to depend on the arrangement of target and penalty disks (Figure 2A). To test whether participants take into account the shapes of their endpoint distributions, we arranged the penalty disks to appear at one of three locations relative to the target disk (Figure 1A). In the “aligned” configurations, the target-penalty direction was approximately aligned with the direction of movement (i.e., with the dock-target direction). In the “nonaligned” configurations, the target-penalty direction was approximately orthogonal to the direction of movement. In the aligned configurations, the extent of endpoint distribution relevant to the task was larger than in the nonaligned configurations. Therefore, an optimal strategy was to shift the aim point away from target center for a larger distance in aligned than nonaligned stimuli (Figure 2A).

Figure 2.

Shift of aim point across stimulus configurations. (A) A schematic illustration of how the shape of endpoint distribution is expected to affect behavior. The ellipses represent the shapes of endpoint distributions: The scatter of endpoints is generally larger in the direction of movement than in other directions. The black squares represent aim points: means of endpoint distributions. Because of the anisotropy of endpoint distributions, the predicted aim points are farther from the target center in the “aligned” than “nonaligned” conditions. (B) Examples of actual endpoint distributions on Day 3 for two representative participants: a PD patient (off medications) on top and an age-matched control participant on the bottom, plotted separately for the aligned and nonaligned stimulus configurations. The black dots represent movement endpoints, and black ellipses represent 95% confidence intervals of mean endpoints. The dock-target axis (roughly aligned with direction of movement) is represented by the green lines. The aim points are farther from the center of the target disks in the aligned than nonaligned conditions for both patients and control participants.

Assuming that the trajectories for upward and downward movements are nearly the same except for their directions (Papaxanthis et al., 1998), from the virial theorem applied over the trajectory it follows that the average expended energy (and thus the average muscle activity) is greater for the upward than downward movements, that is, moving with or against gravity. Two terms matter for comparing the energy required by upward versus downward movements: One term concerns moving the arm, and the other concerns resisting gravity. Because the trajectories for up and down movements are nearly identical, the terms responsible for the energy of moving the arm cancel out. But the change in energy due to gravity is positive for upward movements and negative for downward movements, which is why the difference in the gravity terms does not cancel out, and indeed, it takes more energy to move up than down. Consistent with this expectation, EMG recordings indicate more muscle activity for rapid upward than downward movements (Papaxanthis et al., 2003).

We varied target height, placing targets above and below the dock where reaching movements required different amounts of energy. The larger effort in upward movements relative to downward movements was expected to lead to increased endpoint variability and, consequently, to a larger shift of the aim point away from the upper than lower targets.

Computation of Expected Gain

We compared human behavior with prescriptions of an ideal planner that maximizes expected gain by taking into account participants’ individual motor variability. In the previous studies that used this normative approach, a Gaussian model of participants’ endpoint distributions was used to derive predictions of optimal action (Gepshtein et al., 2007; Trommershäuser, Gepshtein, Maloney, Landy, & Banks, 2005; Trommershäuser et al., 2003). Our present endpoint distributions were inconsistent with the Gaussian model. Using kurtosis excess to test for normality of the endpoint distributions, we found that endpoint distributions in all groups were consistent with normality on Day 2, F > 15, p < .0002, but none on Day 3, F(1, 25) < 1.32, p > .26.

Because endpoint distributions significantly deviated from normal, we computed the expected gain using a robust resampling procedure. For every participant and experimental condition, we estimated the mean and standard error in the participant’s score by resampling endpoints 1000 times with replacement (Efron & Tibshirani, 1993). For each resampling, a score was calculated by accumulating the scores of all resampled endpoints. This resulted in a bootstrapped distribution of score estimates from which the bootstrapped mean and standard error were calculated. The standard error from the bootstrapping was combined with the standard error of the actual data by taking the square root of the sum of the squares (to propagate the two errors) and included them in all analyses. The bootstrapping standard error was approximately 1–10% of the SEM. Next, we assumed that participants chose the aim point constrained by the individual variability in the endpoints (Trommershäuser et al., 2003).

To estimate whether participants may have scored better if they had aimed at a different point, we translated their distribution and recalculated the bootstrapped score estimate. This translation and recalculation was repeated on a fine-meshed grid with the horizontal and vertical spacing of 1 pixel (~1/2 mm) on the touch screen. The resulting distribution of score (Figure 1B) indicated how well participants would have performed if they had chosen a different aim point with the same distribution of endpoints. Importantly, the bootstrapping also propagated the error such that all translations (or shifts) whose scores were within one standard error of the maximum score were considered potential shifts. The “optimal shift” was the center of mass of the potential shifts, and its error was estimated as one half the square root of the potential shifts area.

We validated the resampling procedure by splitting endpoints for every participant and every experimental condition into two sets (from the first and second halves of trials) and then comparing results of resampling performed separately for each set. A Kolmogorov–Smirnov test of the resulting distribution of shifts in each direction showed that the two distributions were the same (p > .8). The test validated the resampling procedure, and it also confirmed our assumption that participants’ behavior was constant for duration of the experiment.

Models of Aiming

We considered several approaches to modeling the shift of the aim point. First, we modeled shift d of the aim point as a linear combination of expected gains and losses from touching the target and penalty regions of the stimulus (Figure 4). We have also considered a more general approach in which total shift d_T is viewed as a combination of two components:

$${\mathbf{d}}_{\mathbf{T}}={\mathbf{d}}_{\mathbf{R}}+{\mathbf{d}}_{\mathbf{B}},$$ (1)

where d_R is the “reward shift” (required to maximize reward in face of motor uncertainty) and d_B is the “biomechanical shift” (required to maximize reward in face of the variable biomechanical cost). These two terms correspond to the terms in the formulation by Trommershäuser et al. (2003), but in our case, we can neither neglect the biomechanical cost nor simplify to the case of normal endpoint distributions. However, we can experimentally isolate each of the two terms as follows.

Figure 4.

Participants’ winnings and shifts of aim points across subject groups and target location. The ordinate is the difference of the observed reward and the reward predicted by the ideal planner. It is the amount of reward participants would gain if their behavior was as predicted by the ideal planner unaffected by biomechanical costs. The abscissa is the distance of the observed aim point from the aim point predicted by the ideal planner, normalized by target radius. The upward and downward pointing triangles correspond to the upper and lower targets, respectively, for each participant in each group. The shaded regions represent standard deviations of losses averaged within subject groups.

First, we isolated the effect of varying reward by computing the difference

$${\mathbf{d}}_{\mathrm{T},-10}-{\mathbf{d}}_{\mathrm{T},0}=({\mathbf{d}}_{\mathrm{R},-10}+{\mathbf{d}}_{\mathrm{B},-10})-({\mathbf{d}}_{\mathrm{R},0}+{\mathbf{d}}_{\mathrm{B},0}),$$ (2)

where subscripts “0” or “−10” indicate the penalty values used on Days 2 and 3 of the experiment. Biomechanical costs were unchanged across days of experiment and across penalty values, and the targets were presented within a few millimeters of the same locations regardless of penalty value, such that d_B,−10 = d_B,0 and the effect of biomechanical cost in the above equation cancelled out, yielding d_{R,−10 −} d_R,0. We therefore evaluated the effect of reward by constructing a statistical model of the difference of aim points between Days 2 and 3 and by asking how the difference depended on Subject Group, Stimulus Alignment, and Target Height (Figure 5A).

Figure 5.

Shifts of aim points across subject groups plotted using the same color convention is as in Figure 2. (A) Difference of aim points between Day 2 (penalty absent) and Day 3 (penalty present). The two days differed in terms of expected gain, but they did not differ in terms of biomechanical cost. Therefore, the difference of shifts across days discards the effect of biomechanical cost. No systematic pattern of shifts is observed across subject groups, indicating that participants’ ability to compute expected gain was comparable across groups and target configurations. (B) Difference between the actual and optimal aim points within Day 3 (penalty present). The optimal aim points were computed disregarding biomechanical cost, which is why the differences of actual and optimal aim points emphasize the effect of biomechanical cost. The results are similar to those in A for the lower target, but for the upper target aim point shifts decrease from PD patients to old control participants to young control participants. Because the penalty was present in the upper and lower targets and because biomechanical cost was larger for the upper than the lower target, the difference in results for the upper and lower targets indicates that both PD and elderly age impair the ability to take into account the expected biomechanical cost of action.

Second, we isolated the effect of varying biomechanical cost by contrasting the observed aim points with the aim points predicted for the upper and lower targets while disregarding biomechanical cost, that is, the optimal shift. Denoting the shifted distribution with subscript “S,” we wrote the difference of observed and shifted aim points as

$${\mathbf{d}}_{\mathrm{T}}-{\mathbf{d}}_{\mathrm{T},\mathrm{S}}=({\mathbf{d}}_{\mathrm{R}}+{\mathbf{d}}_{\mathrm{B}})-({\mathbf{d}}_{\mathrm{R},\mathrm{S}}+{\mathbf{d}}_{\mathrm{B},\mathrm{S}}).$$

Because d_R = d_R,S and d_B,S = 0, this difference isolated the effect of biomechanical cost, d_B. We therefore evaluated the effect of biomechanical cost by constructing a statistical model of the difference of actual and predicted aim points. As before, we asked how the difference depended on Subject Group, Stimulus Alignment, and Stimulus Height. (Results of this analysis are shown in Figure 5B.)

Data Analysis

We analyzed the data using generalized linear models in the nlme package (cran.r-project.org/web/packages/ nlme/) by R Development Core Team (2005). Fixed or random effects were used as described in Pinheiro and Bates (2000), considering both effects of interest and tests of Bayesian Information Criterion (BIC) to systematically identify the best random effects for the linear model. In all cases, the simple subject random effect had the lowest BIC, which is why we used the random effect of subject in the analysis. For analyses involving average values (e.g., distance to target or distance to optimal), we took into account the individual variability by weighting data in proportion to the inverse standard error estimate of the parameter. In the analysis of score versus distance, where individual trials were considered, all trials were used without weighting. F and p statistics were then estimated from the model using a marginal term removal, where the full model was compared with the model with the selected term deleted (Pinheiro & Bates, 2000). Dependent variables were precalculated using a custom C/C++ script.

RESULTS

Effect of Motor Uncertainty on Movement Planning

Results for Days 2 and 3 are summarized in Tables 2 and 3 and in Figure 3, where estimates of aim points averaged within subject groups are plotted relative to centers of target disks, separately for the upper and lower stimulus locations. As we explained in Figure 2, when the penalty is present (i.e., on Day 3), participants were expected to shift the aim point away from target center for a larger distance in the “aligned” than “nonaligned” stimulus configurations (Figure 2A). The results in Figure 3 were consistent with this expectation. To evaluate how the shift of aim point depended on target configuration, we used a statistical model of the distance of individual endpoints from the target center with fixed effects of Subject Group, Alignment, Target Height, and their interactions with a random Subject effect. For the data from Day 3 (when penalty disks had negative values), this analysis revealed that a wider distribution of endpoints along the direction of movement than in the other directions caused participants to shift endpoints 0.13 ± 0.06 target radii (mean ± standard error) farther away from target center in the aligned than nonaligned configurations, F(1, 81) = 5.78, p = .02. These effects depended on the interaction of Subject Group and Target Height, F(2, 81) = 9.91, p = .0001. We also found a marked effect of height for PD participants, F(1, 27) = 14.1, p = .0008, but not for either group of control participants, F < 1.16, p > .3. Notably, PD patients shifted aim points 0.23 ± 0.06 target radii farther away from the upper target than the lower target.

Table 2.

Mean Endpoints for Day 2

Group	Direction	Nonaligned 1		Aligned		Nonaligned 2
		Down	Up	Down	Up	Down	Up
PD	x	8.8 ± 3.2	−18 ± 2.2	11 ± 1.9	−14 ± 1.6	5.1 ± 4	−5.4 ± 2.6
PD	y	0.7 ± 2	−4.9 ± 3.2	3.9 ± 1.9	−14 ± 1.7	7.6 ± 2.5	−14 ± 1.7
OC	x	9.7 ± 2.3	−7.7 ± 2.6	9.4 ± 2.1	−9.4 ± 2.9	3.8 ± 2.2	−1.9 ± 2.8
OC	y	−0.21 ± 2.6	0.62 ± 2	4.1 ± 1.6	−7.2 ± 1.8	4.4 ± 2	−4.1 ± 1.8
YC	x	4.1 ± 2.1	−3.7 ± 2.1	4.8 ± 1.7	−4.2 ± 2.3	−0.16 ± 2.4	−0.42 ± 2.7
YC	y	−1.7 ± 1.9	2 ± 2.6	4.2 ± 1.5	−2 ± 1.4	2.4 ± 2.2	−3.2 ± 1.7

All units are in millimeters, and directions are on the screen with respect to the target center.

Table 3.

Mean Endpoints for Day 3

Group	Direction	Nonaligned 1		Aligned		Nonaligned 2
		Down	Up	Down	Up	Down	Up
PD	x	12 ± 2.2	−16 ± 2.3	14 ± 2	−14 ± 2.3	1.8 ± 2.3	−6.2 ± 1.8
PD	y	−1.3 ± 2.3	−3.7 ± 2.2	4.9 ± 2.3	−15 ± 2.2	9.9 ± 2.5	−16 ± 2
OC	x	11 ± 1.8	−11 ± 1.4	13 ± 1.4	−11 ± 1	0.93 ± 1.9	0.41 ± 1.2
OC	y	−3.2 ± 1.2	3.8 ± 1.5	8.6 ± 1.3	−11 ± 1.3	11 ± 1.6	−10 ± 1.7
YC	x	6.8 ± 0.72	−7.9 ± 0.99	9.4 ± 0.65	−7.2 ± 0.81	−3.3 ± 0.72	3.1 ± 1.5
YC	y	−6.1 ± 1.2	7.5 ± 1	5.6 ± 1.1	−7.9 ± 0.74	8.8 ± 0.93	−7 ± 0.66

All units are in millimeters, and directions are on the screen with respect to the target center.

Figure 3.

Aim points for all subject groups and stimulus configurations for Day 3 (penalty present, top) and Day 2 (penalty absent, bottom). The aim points are shown relative to the center of the target disk (marked by a star), separately for the upper and lower stimulus locations. The gray bands are introduced in the top panels to group data points from Day 2 according to stimulus configuration: aligned (labeled “AL”) and nonaligned (“NAL”). The gray regions of the top panels are reproduced in the bottom panels to help compare the less orderly data from Day 2 with the data from Day 3. The results from Day 3 indicate that PD patients shifted aim points away from targets more for the upper than lower target. For the upper target, the shifts were graded across subject groups, with the largest shifts observed for PD patients. This effect was not observed for the lower target.

We performed similar analyses for Day 2, when the penalty disk had zero value. As expected in this case, the spatial alignment of the zero penalty region relative to the target region had no significant effect on the shift of aim point, F(1, 81) = 0.5, p = .5. But the interaction of Subject Group and Target Height on Day 2, F(2, 81) = 7.49, p = .001, was similar to that on Day 3, dominated by PD patients shifting aim points 0.26 ± 0.06 target radii farther for the upper target, F(1, 27) = 17.21, p = .0003, resulting in the upper target undershooting by PD patients (see Figure 3). This result indicates that the larger shift observed in PD for the upper target was not an effect of the magnitude of penalty.

Normative Model of Score

The radius of the target disk was varied to make performance approximately the same across participants, at 80% success rate. To verify that motor uncertainty and target size were matched, we asked whether the sensitivity of the participants’ score to their deviation from optimal was similar. This sensitivity is represented by the slopes of functions in Figure 4, where the difference of observed and optimal scores (the “loss” of score) is plotted as a function of the distance of observed from optimal aim points. The slopes have the units of reward per distance (cents per target radius, notated as cent/radius). For simplicity, we assumed that effects of alignment were symmetric with respect to direction of movement, and we collapsed them into two levels: aligned or nonaligned. In Figure 4, we plotted the score participants lost by shifting their aim points away from the aim points predicted by the ideal planner. We selected the simplest model according to the BIC with fixed effects of Shift, Shift by Day, Shift by Group, Shift by Alignment, and Shift by Height interactions and a random effect of Subject Group. Overall, an ANOVA on the mixed model showed that, on average, participants lost 1.5 ± 0.4 cent/radius, F(1, 192) = 18.5, p < .0001. There was an additional 0.5 ± 0.1 cent/radius lost for the aligned condition, reflecting the asymmetry of the endpoint distribution, which was longer in the direction of movement. There was also an overall interaction of Shift and Subject Group, F(2, 192) = 12.7, p < .0001.

In the models restricted to PD patients and Old Control participants, individual contrasts showed that PD patients lost 0.8 ± 0.3 cent/radius more than Old Control participants, F(1, 127) = 8.79, p = .004. In turn, Old Control participants lost 0.4 ± 0.2 cent/radius more than Young Control participants, but this effect was not significant, F(1, 127) = 3.36, p = .07.

Isolating Effect of Reward and Biomechanical Cost

In the results shown in Figure 5A, the biomechanical cost was factored out by subtracting the aim points for Day 2 from Day 3 for each participant (Equations 1–2, Methods). The size of this shift of aim point due to the increased penalty isolated changes of the participant’s motor plan due to changes of the variable reward/penalty values, while taking into account their variable biomechanical cost. Little variability was observed across the conditions, and indeed a null linear model, fit without fixed effects, had a BIC of 56.2 lower than the full model, and it provided the best fit. Comparing models sequentially while incorporating terms to the null model showed no significant effects, F < 1.71, p > .16. We therefore concluded that participants’ ability to compute expected gain was comparable across Subject Groups and target configurations.

In the absence of penalty on Day 2, participants’ movements were unconstrained: They could aim at any location inside the target. We therefore used data from Day 3 alone to calculate participant’s shift from the optimal location. This way we isolated the effect of participants’ biomechanical cost in selection of motor plan while taking into account the effects of reward and penalty values. On Day 3, we found a large variability of the magnitudes of the shift from optimal across subject groups and target heights (Figure 5B). The best model incorporated linear terms for Subject Group, Alignment, and Target Height and only the interaction of Subject Group with Target Height (its BIC was 28.5 less than the full model with all interactions and 13.1 less than the null model with no terms except for the intercept). An ANOVA revealed an effect of Subject Group, F(2, 15) = 3.97, p = .04, Alignment, F(2, 85) = 5.22, p = .007, and an interaction of Subject Group and Target Height, F(2, 85) = 6.2, p = .003. These effects are manifested in the striking differences between the plots for upper and lower targets in Figure 5B. That is, effects of biomechanical cost on participants’ behavior were clearly largest for PD patients, next largest for Old Controls, and the smallest for Young Controls for the upper target, but not for the lower target. To account for variability in task difficulty across groups, we fit linear mixed models to each group separately. There was no dependence within any group on either the interaction of Target Height and Alignment or on Alignment alone, F < 2.13, p > .1, and thus, the overall alignment effect was likely an artifact of variability across subject groups. The interaction of group and height in the overall analysis appeared as a dependence on Target Height unique to PD participants, F(1, 25) = 5.06, p = .03, where the effect of biomechanical cost was a shift of aim points by 0.3 ± 0.1 target radii greater for the upper than the lower target.

Finally, we investigated the significant interaction of Subject Group and Target Height in the effect of biomechanical cost by concentrating on the upper and lower targets separately. The lower target showed a significant overall effect of Subject Group, F(2, 15) = 4.21, p = .04, which bears out as individual trends for (a) PD patients shifting aim points 0.2 ± 0.1 target radii farther than Old Control participants and (b) Old Control participants shifting aim points 0.09 ± 0.1 target radii farther than Young Control participants. For the upper target, the effect of the gradient of biomechanical cost became most apparent as (a) a larger shift by PD than Old Control participants of 0.4 ± 0.1 target radii, F(1, 10) = 11.7, p = .007, and (b) a larger shift by Old Control than Young Control participants of 0.22 ± 0.09 target radii, F(1, 10) = 5.77, p = .04. This gradient corresponded to a clear reduction of the relative weight of biomechanical cost versus reward to endpoint selection from Young Controls to Old Controls to PD patients (Figure 5B, top row).

To summarize, this analysis revealed that the effect of biomechanical cost on action planning was consistent with the view that Young Control participants were capable of shifting aim points so as to maximize reward under variable biomechanical cost. This ability was slightly compromised in Old Control participants and severely compromised in PD patients.

DISCUSSION

We studied the ability of healthy participants and PD patients to perform rapid, sequential movements under conditions of risk and uncertainty. Using an ideal planner approach, we modeled optimal performance for every participant while taking into account each participant’s individual motor uncertainty. The model provided an optimal prescription for maximizing expected reward: a benchmark of individual performance for participants having different degrees of motor uncertainty. In this manner, the performance of participants whose motor abilities were starkly different in absolute terms could be compared in terms of how closely their performance approached their individual benchmarks. We found that the performance of PD patients could be as good as the performance of healthy participants when energy cost was minimized, but patients’ performance was significantly reduced when the same task was associated with an increased biomechanical cost of movement.

We varied the biomechanical cost by having participants perform actions assisted or countered by gravity. The effect of gravity on muscular activity in unconstrained, multijoint 3-D reaching movements was studied by d’Avella et al. (2006). These authors recorded EMG from 19 shoulder and arm muscles during rapid, point-to-point movements from a central target to one of eight peripheral targets in the frontal plane. They found that the EMG amplitude of muscle synergies for reaches up and to the right (our upper target) was considerably larger than for reaches down and to the left (our lower target), thus demonstrating the higher energy cost for the upper target.

In this study, PD patients performed as well as the age-matched control participants in conditions of low biomechanical cost, that is, when their actions were assisted by gravity. But under conditions of high biomechanical cost, where actions were executed against gravity, the performance of patients precipitously deteriorated, as if the task difficulty suddenly increased to a level beyond which the same participants were incapable of performing actions as well as they could in conditions of low biomechanical cost.

We examined the participants’ performance not only in absolute terms, but also in terms of proximity to the optimal aim point. The optimal aim point is a benchmark of performance that takes into account individual’s precision of movement, thereby allowing comparisons of performance across a broad range of motor competence. We found that when the movements were assisted by gravity, PD patients performed well, in fact similar to the optimal prescriptions and to healthy elderly participants. When the movements were countered by gravity, the performance of both patients and healthy elderly participants dropped below that of their optimal prescriptions, but the performance of PD patients was significantly worse than that of the elderly control participants, not only in absolute terms but also in terms of distance from their optimal prescriptions.

We also tested healthy young participants to examine the intriguing possibility that aging may produce intermediate effects on motor performance, similar to those of PD but to a much smaller degree. We found that the performance of healthy young participants was similar to that of the optimal planner. Interestingly, the performance of healthy elderly participants under conditions of increased biomechanical cost was intermediate to that of PD patients and healthy young controls. This pattern held both in terms of the absolute performance and in terms of proximity to the optimal prescriptions. This result supports the possibility that aging may have effects similar to those caused by PD, but to a much smaller degree (Buchman, Shulman, Nag, et al., 2012; Ross, Petrovitch, Abbott, et al., 2004; Fearnley & Lees, 1991). Human adults lose about 5% of dopamine-containing cells per decade, such that a gradient of dopamine cell loss holds across the age span, a process markedly accelerated in PD (Fearnley & Lees, 1991). Our results suggest that this gradient of dopamine cell loss is associated with a gradient of decrement in motor performance. It should be noted, however, that it is not yet clear whether aging causes changes in striatal dopamine along the same spectrum as PD (Darbin, 2012; Fearnley & Lees, 1991).

These results allow us to examine hypotheses about the function of neural systems whose malfunction may be responsible for the difference between the performance of patients and healthy participants in our study. PD is currently viewed as a multisystem neurodegenerative disorder, in which patients suffer from deficits in multiple neurotransmitter systems (Lang & Obeso, 2004). However, motor deficits in PD patients are associated directly with the degree of dopamine depletion in the dorsal striatum (Rodriguez-Oroz et al., 2009; Pirker, 2003). Indeed, there is a gradient of dopamine depletion in dorsal versus ventral striatum in PD (Nandhagopal et al., 2009; Morrish, Sawle, & Brooks, 1996). In PD patients with mild to moderate disease severity, the primary dopamine depletion is in the dorsal striatum. Only later in the course of the disease does the degeneration of dopamine pathways affect the ventral striatum. Therefore, our results support the view that sensitivity to biomechanical cost (or energetic cost) of movement is associated with degree of dopamine depletion in the dorsal striatum (cf. Mazzoni et al., 2007).

It is well established that loss of the dopamine-containing cells in the midbrain leads to profound deficits in initiating, controlling, and maintaining movements, as seen in patients with PD (Torres, Heilman, & Poizner, 2011; Wu, Wang, Hallett, Li, & Chan, 2010; Rodriguez-Oroz et al., 2009; Tunik, Feldman, & Poizner, 2007). But the precise role of dopamine in enabling motor behavior is not well understood. As mentioned, theoretical studies suggested that dopamine functions not only to provide reward-related signals but also to directly control active behavior by providing motivation for action or motor vigor in terms of both repeated action selection (Niv et al., 2007; Hallett, 1990) and speed of movement (Mazzoni et al., 2007). In particular, Mazzoni et al. (2007) proposed that the motor system has its own motivational circuit, controlling action vigor, modulated by the participants’ perceived energetic cost of a movement. In this view, the role of dopamine in sensorimotor circuits (which connect the dorsal striatum, thalamus, and sensorimotor cortices) is to provide implicit motivation for movement, analogous to the motivational role dopamine plays in reward-based circuits that connect the ventral striatum with prefrontal cortices. Thus, the slow movements observed in PD result from a lack of motor vigor, such that PD patients prefer slow movements rather than being unable to move fast (Mazzoni et al., 2007).

Consistent with the notion that at least some aspects of PD manifest as patients’ reluctance to make certain movements rather than inability to do so, we found that our patients were capable of highly efficient performance in rapid, sequential movements, nearly as good as that of the ideal planner, and in spite of marked motor deficits. The motor ability of patients degraded in conditions of increased energetic cost, when rapid sequential movements had to be performed against gravity. The deterioration was unlikely to be caused by fatigue because participants were given frequent breaks, each trial was self-initiated, and there was no significant decline in patients’ performance over the course of the session.

Patients’ inability for efficient action against gravity cannot be explained by hypometria: a tendency for small movements leading to systematic “undershooting” independently of the expected risk, reward, or energetic cost of the movement. Compared with the age-matched control participants, PD patients did not undershoot the target when the movements were assisted by gravity for the lower target (Figure 3, right). And for the upper target, the patients were capable of performing the movements required to maximize reward. Indeed, they did so on Day 2 (when the penalty was zero) and on Day 3 (with high penalty) because the absolute position of the target varied across trials. When the target happened to be farther away from the dock, patients reached for points that they would need to (but did not) reach when the target happened to be closer to the dock. Evidently, it was the combined effect of the expected reward and the expected cost of movement that determined behavior, rather than merely the distance to the target.

The comparison of performance on Days 2 and 3 (that differed in terms of expected gain but did not differ in terms of biomechanical cost) supported this view. The difference of aim points across days was not larger for patients than for healthy controls (Figure 5A). But the difference of aim points was larger for patients than for healthy controls within Day 3 (Figure 5B), where the aim points were compared across conditions that differed in terms of biomechanical cost. That is, the patients were able to plan actions similarly to control participants, except they were differentially sensitive to the biomechanical cost.

Using fMRI, Croxson, Walton, O’Reilly, Behrens, and Rushworth (2009) recently have elucidated a neural system in humans critical for evaluation of effort-based decision-making, that is, for evaluating how much effort is worth expending to obtain expected rewards. This system comprised the dopaminergic midbrain, ACC, and ventral striatum, regions that are highly interconnected in primates (Williams & Goldman-Rakic, 1998). Ventral striatal, anterior cingulate, and related mesolimbic pathways are well known to be involved in motivation and reward processing. However, in PD the nigral-striatal pathways typically degenerate well before degeneration of the mesolimbic pathways (Kish, Shannak, & Hornykiewicz, 1988). Because our patients were in the mild to moderate stage of the disease, the primary degeneration in their dopamine pathways would be nigral-striatal rather than mesolimbic. Thus, our results are complementary to those of Croxson et al. (2009) and implicate nigral-striatal pathways in mediating effort-based movement decisions.

Our results are also consistent with the results of Baraduc, Thobois, Gan, Broussolle, and Desmurget (2013), who studied one-dimensional reaching movements by PD patients who had stimulating electrodes surgically implanted in the subthalamic nucleus. The latter study showed that a single optimal control model that minimized total neuromuscular cost (motor effort) could predict the speed of reaching movements by both healthy individuals and PD patients. But the dynamic range of motor signals was found to be smaller for PD patients than healthy individuals, consistent with the view that reduced motor effort leads to reduced speed of movement.

Parush, Tishby, and Bergman (2011) developed a model of action planning in which the reward and cost of action are represented by separate, independent dimensions, rather than making contributions to expected action outcome on the single dimension of reward. Our modeling approach is not inconsistent with this interesting idea. As the data in Figure 5 suggest, human performance across a wide range of motor proficiency could be explained by a model in which reward of action and its biomechanical cost are separable in the sense their combination is additive. That is, in the less demanding conditions where movement was assisted by gravity, behavior of both normal participants and PD patients could be explained by a model in which the effect of biomechanical cost was ignored or given a zero weight. In the more demanding conditions, where movement was countered by gravity, behavior of both normal participants and PD patients could be explained by taking into account the additive perceived cost of movement.

To summarize, we found that the degree of dopamine depletion in the dorsal striatum correlates with sensorimotor performance in face of variable risk, uncertainty, and biomechanical cost of movements. Progressive loss of dopamine in sensorimotor (“dorsal”) BG circuits is associated with the decreasing ability to cope with biomechanical cost, highlighted by the striking fact that performance of PD patients was similar to that of healthy participants in conditions of low biomechanical cost, but their performance deteriorated precipitously as biomechanical costs grew. Our findings support the view that tonic levels of dopamine in the dorsal striatum provide implicit motivation for movement, supporting the motor motivation hypothesis of bradykinesia proposed by Mazzoni et al. (2007). In addition, our findings suggest that the tonic levels of dopamine in the dorsal striatum determine the energetic cost of movement beyond which participants do not or cannot optimize their motor behavior.