Neural correlates of perceptual learning in a sensory-motor but not a sensory cortical area

Abstract

This study aimed to identify neural mechanisms that underlie perceptual learning on a visual discrimination task. We trained two monkeys to decide the direction of visual motion while recording from the middle temporal area (MT), which in trained monkeys represents motion information used to solve the task, and the lateral intraparietal area (LIP), which in trained monkeys represents the transformation of motion information into a saccadic choice. During training, improved behavioural sensitivity to weak motion signals was accompanied by changes in motion-driven responses of neurons in LIP but not MT. The time course and magnitude of the changes in LIP were correlated with the changes in behavioural sensitivity throughout training. Thus, for this task, perceptual learning appears to involve improvements not in how sensory information is represented in the brain but rather how the sensory representation is interpreted to form the decision that guides behaviour.

Training can induce long-lasting improvements in our ability to detect, discriminate or identify sensory stimuli¹. Despite the prevalence of this phenomenon, called perceptual learning, our understanding of the underlying neural plasticity is incomplete. Changes in early sensory areas of cortex have been inferred from psychophysical studies² (but see refs ^{3, 4}) and identified in monkeys trained on auditory⁵ and somatosensory⁶ tasks. However, monkeys trained on visual tasks show only moderate or no change in early visual cortex^7–11. Changes in higher stages of processing, including those that contribute to decision-making and attention, have also been inferred from psychophysical studies^{3, 4, 8, 12–14}. However, such changes have not been identified directly in the brain. We sought to identify experience-dependent changes in visual processing in two different cortical areas, one in extrastriate visual cortex and the other in parietal cortex, and determine their relative contributions to perceptual learning.

We trained monkeys to decide the direction of motion of a random-dot stimulus and indicate their direction decision with an eye movement. Most neurons in the middle temporal area (MT) of extrastriate visual cortex are tuned for the location and direction of moving visual stimuli¹⁵. In trained monkeys, MT responses can be as sensitive to random-dot motion signals as the monkey’s behavioural reports and are weakly predictive of the monkey’s choices on the discrimination task^{16, 17}. Moreover, MT lesions degrade performance¹⁸, and MT microstimulation biases performance on the task¹⁹. Thus, MT provides at least some of the motion information used to form the direction decision.

Neurons in the lateral intraparietal area (LIP) of parietal cortex, which has been implicated in a variety of cognitive and visuomotor functions, including attention, intention, reward anticipation and decision-making^20–23, also respond during performance of the discrimination task. These responses are modulated by the strength and duration of the motion stimulus and can be strongly predictive of the monkey’s saccadic choices^{24, 25}. LIP microstimulation can bias performance on a reaction-time version of the task²⁶. Thus, LIP appears to play a role in transforming motion information into a saccadic choice, although that role might be shared among several oculomotor regions including the frontal eye field and superior colliculus that show similar responses during task performance^{27, 28}.

We recorded the activity of individual MT and LIP neurons while two naïve monkeys were trained on the direction-discrimination task. Behavioural sensitivity to weak motion signals improved continuously during training, long after monkeys had acquired the stimulus/response association. This improvement in behavioural sensitivity corresponded to changes in the responses of neurons in LIP but not MT during motion viewing. The results are consistent with a model in which perceptual improvements result from changes in how sensory evidence is interpreted and used to instruct behaviour, as reflected in LIP, but not in changes to the representation of the evidence itself in MT.

Results

Discrimination threshold improves with training

We trained two rhesus monkeys on a one-interval, two-alternative direction-discrimination task (Fig. 1a; monkey C: 165 sessions over 645 d; monkey Z: 155 sessions over 473 d). Each daily training session began by introducing recording electrodes into MT and/or LIP (Fig. 1b). As in previous studies, quantifying the relationship between behavioural and neuronal performance depended on matching the visual stimulus to the properties of the neuron(s) being recorded^{16, 25}. Within this constraint, we sought to minimize changes in the stimulus configuration across sessions by selecting neurons with consistent tuning properties, including receptive field (RF) location and direction preference of MT neurons and response field location of LIP neurons (Fig. S1). After finding the appropriate neuron(s), we placed the motion stimulus in the MT RF (or in its modal location if no MT neuron was found) and one of the two choice targets in the LIP response field (or in its modal location if no LIP neuron was found). Training occurred only while the responses of at least one MT or LIP neuron were isolated and recorded, allowing us to make direct, session-by-session comparisons of behavioural and neuronal performance as training progressed.

Figure 1.

Task and anatomical localization. a, Direction-discrimination task. The motion stimulus matched the RF location and preferred direction (and its 180° opposite) of the MT neuron being recorded or the modal values from previous sessions if no MT neuron was found. One target was placed in the response field of the LIP neuron being recorded or the modal location from previous sessions if no LIP neuron was found, the other in the opposite visual hemifield. b, Anatomical localization of recording site locations in areas MT (left, cyan) and LIP (right, red) using magnetic resonance imaging (MRI). Top: volume rendering using the AFNI⁴⁹ render plugin showing the 3D orientation of the recording cylinders relative to the head. Middle: partial reconstruction of the cortical surface along with the projection of the recording cylinder using Caret and SureFit⁵⁰ and custom software. The yellow arrow in the left panel points to the location of area MT (red), along the superior temporal sulcus. The yellow arrow in the right panel points to the location of area LIP (brown), along the intraparietal sulcus. Bottom: partial penetration maps of successful recording sites (black points) superimposed on planes of section perpendicular to the long axis of the recording cylinder. MT sites (top) ranged in depth from 6–9 mm from the dura mater. LIP sites (bottom) ranged in depth from 4–7 mm from the dura mater. These images were generated with methods described in R.M. Kalwani, L. Bloy, J. Hulvershorn, M.A. Elliot & J.I. Gold, Soc. Neurosci. Abstr. 454.14, 2005.

Behavioural performance improved steadily for both monkeys with training (Fig. 2). In early sessions we trained monkeys on a simplified version of the task using only the strongest motion stimulus (99% coherence) to reinforce the association between motion direction and saccade target. After performance on this easy condition improved to well above chance (>~75% correct), we introduced more difficult stimuli using randomly interleaved motion coherences and viewing durations (indicated as session 1 in Fig. 2c). Nevertheless, we continued to interleave trials with 99% coherence randomly in each session throughout training (Fig. 2c). For both monkeys, declining error rates on these high-coherence trials as a function of session were fit by a single-exponential function with a lower asymptote of zero (mean±SEM initial value = 0.51±0.02 for monkey C, 0.48±0.01 for monkey Z; time constant = 3.6±0.4 sessions for monkey C, 30.4±1.1 sessions for monkey Z). Thus, both monkeys learned how to perform the task in early sessions and soon could express the visuomotor association for high-coherence stimuli with few or no errors.

Figure 2.

Behaviour. a,b. Behavioural performance (a) and discrimination threshold (b, best fits and 68% CIs) as a function of viewing time (0.3-s-wide bins in 0.15-s intervals) for different motion strengths (see legend) from two representative sessions early (left) and late (right) in training. Discrimination thresholds in b were computed for each time bin using a cumulative Weibull function¹⁶. Solid lines in a and b are behavioural performance and thresholds computed from a time-dependent cumulative Weibull function (Eq. 1) fit to each data set (not binned by viewing duration), respectively. We report error rates at 99.9% coherence (dashed arrows in a, σ in c and d) and discrimination thresholds at 1-s viewing duration from the fits (dashed arrows in b, ● in c and d). c, d, Discrimination threshold (●; note the logarithmic scale on the left ordinate) and error rate at 99.9% coherence (σ; linear scale on the right ordinate) with 68% CIs plotted as a function of training session for the two monkeys. Prior to session 1, monkeys were trained mostly with 99.9% coherence motion. Solid lines are best-fitting single exponential functions. d, Learning rates (best fits and SEM) of discrimination thresholds (● ) and errors at 99.9% coherence (σ ) during training for the two monkeys. The learning rate was computed as the slope of a linear fit to the behavioural data (log discrimination thresholds or errors at 99.9% coherence) within a 41-session wide bin. A negative learning rate implies that the behavioural parameter improved during that particular epoch of training.

In addition to learning the association, the monkeys learned to better discriminate weaker and weaker motion signals at shorter viewing times. Fig. 2a shows performance from two sample sessions, one early and one late in training, for monkey C. For nearly all coherences and viewing times, the percentage of correct responses was greater in the later session. Indeed, throughout training, performance improved significantly for all non-zero coherences for monkey C and for 12.8–99.9% coherence for monkey Z (linear regression of percent correct per coherence with training session, H₀: slope=0, p<0.05).

To further quantify improvements in performance with training, we used a time-dependent cumulative Weibull function (Eq. 1 and solid lines in Fig. 2a,b) to estimate the discrimination threshold, which is the motion strength at which the monkey achieved ~82% correct for a 1-s viewing duration, for each session. This function takes into account associative (high-coherence) errors (λ in Eq. 1) and thus can indicate changes in threshold that are distinct from changes in associative learning or lapses of attention (Fig. S2). Thresholds improved from ~67% to ~15% coherence for monkey C and from ~68% to ~19% coherence for monkey Z, with a time course that was substantially longer than for the associative improvements (Fig. 2c: mean±SEM time constants of single-exponential solid curves = 24.6±0.4 sessions for monkey C and 58.5±2.5 sessions for monkey Z; final value = 14.7±0.1% coherence for monkey C, 19.3±0.4% coherence for monkey Z; Fig. 2d: improvements in high-coherence errors were limited to early sessions, whereas improvements in threshold were apparent throughout most of training). Other behavioural parameters including the shape and time dependence of the psychometric function did not show similar systematic changes with training (Table S1). The goal of this study was to identify changes in the response properties of MT and LIP neurons that accompanied the improvements in discrimination threshold.

MT responses do not change with training

We recorded the responses of individual MT neurons to the motion stimulus both before and during training (Fig. 3; Monkey C: n=50 MT neurons recorded during a passive viewing condition before discrimination training began, n=92 during training; monkey Z: n=47 before, n=60 during training). As illustrated in Fig. 3a, MT responses before and during training showed a pattern of activation similar to that reported in trained monkeys¹⁶: motion in the preferred direction elicited an increase in spike rate and motion in the opposite direction elicited a decrease in spike rate to relatively constant values that depended on motion strength and sustained throughout motion viewing. To quantify these responses and test for changes with training, we fit a simple linear model to the normalized responses of individual neurons. This model describes the neural responses in terms of the baseline firing rate and three stimulus-based terms: a dependence on motion coherence that does not change with viewing time, a dependence on viewing time that does not change with motion coherence and a dependence on the (multiplicative) interaction between coherence and viewing time that is consistent with an accumulation of motion information over time²⁹ (Eq. 3, Fig. 3b and Fig. S3). For both monkeys, only the coherence-alone term was typically >0, consistent with the idea that MT responses provide evidence about motion direction that depends only on stimulus strength and not viewing time. Moreover, none of the three terms changed significantly with training (linear regression versus session, computed separately for each term, each monkey and before and during training, H₀: slope=0, p>0.05 for all conditions).

Figure 3.

MT responses. a, Average activity of MT neurons as a function of viewing time (using 0.1-s-wide time bins with 0.025-s increments) for different motion strengths (see legend) for each neuron’s preferred (solid line) and null (dashed line) motion during different training periods for monkey C. “Pre-training” refers to responses to the motion stimulus measured while the monkey was rewarded for simply fixating a central spot, before being trained on the discrimination task. b, Coherence-, viewing time- and coherence × viewing time-dependence (Eq. 3) of individual MT neurons before and during training for monkeys C (left) and Z (right). Error bars are 68% CIs. c, Relationship between neurometric threshold and choice probability for individual MT neurons during different training periods for monkeys C (■) and Z (▼). Error bars are 68% CIs. Solid lines are linear fits.

We also quantified the motion sensitivity of individual MT neurons by computing a “neurometric” discrimination threshold using the same time-dependent cumulative Weibull function used to estimate behavioural threshold (Eq 1, Fig. S4). Neurometric thresholds were, on average, ~4% better during versus before training, but the differences were not significant (monkey C: geometric mean threshold coherence before training = 25.9%, during training = 21.0%, Mann-Whitney U test p=0.1964; monkey Z: before = 21.1%; during = 18.4%, p=0.4762). Moreover, the neurometric thresholds of neither the whole population nor the 50% most sensitive MT neurons changed systematically as a function of session either before or during training (linear regression, H₀: slope of log threshold versus session=0; before training: monkey C, p=0.7612; monkey Z, p=0.9507, during training: monkey C, whole population, p=0.9498, 50% most sensitive, p=0.9899; monkey Z, whole population, p=0.8870, 50% most sensitive, p=0.9480). Interestingly, this lack of long-term changes across sessions occurred despite short-term changes within sessions, as has been reported previously (geometric mean±SEM ratio of psychometric thresholds in the second 200 versus the first 200 trials = 0.92±0.22 in monkey C and 0.84±0.29 in monkey Z; ratio of neurometric thresholds in MT in the second 200 versus the first 200 trials = 0.87±0.23 in monkey C and 0.98±0.29 in monkey Z)³⁰. Other response properties including the width of directional tuning were unaffected by training (Table S1 and Fig. S3).

There was, however, a slight strengthening of the relationship between the trial-by-trial variability of MT responses and the monkeys’ choices with training. This analysis, called choice probability, has shown that MT neurons predict choice slightly but reliably in fully trained monkeys (~0.55, where 0.5 is chance and 1.0 is perfect; ref 17), which is the relationship expected for elements in a pool of weakly correlated neurons that provide noisy evidence for the decision³¹. Choice probability increased as a function of training session (combined data from both monkeys, mean±SEM choice probability = 0.493±0.011 for the first 30% and 0.549±0.018 for the last 30% of sessions). Moreover, the relationship between choice probability and motion sensitivity in MT changed systematically over the course of training such that more sensitive neurons became more predictive of the monkey’s direction decision as training progressed (Fig. 3c; correlation coefficient between choice probability and neurometric thresholds of MT neurons in the 1^st 1/3 sessions, r=0.02, H₀: r=0, p=0.5600; 2^nd 1/3 sessions, r=−0.23, p=0.1137; 3^rd 1/3 sessions, r=−0.34, p=0.0423).

In principle, the increases in choice probability could arise from an increase in correlated firing among MT neurons. However, a small number of simultaneously recorded pairs of MT neurons provide preliminary evidence that the degree of correlation is similar before (mean±SEM correlation coefficient r=0.16±0.06, n=8) and during (r=0.18±0.04, n=9) training and does not increase over the course of training (linear regression, H₀: slope=0, p=0.5728). Thus, the systematic change in choice probability is likely to reflect an increasingly selective read-out of activity from MT neurons, particularly those most sensitive to the motion stimulus.

LIP responses change with training

We recorded the responses of individual LIP neurons during motion viewing throughout training (monkey C, n=123; monkey Z, n=99 LIP neurons recorded during training). Fig. 4a shows the average spike rates of the population of LIP neurons recorded from monkey Cy aligned to motion onset (left) and saccade onset (right) for different training epochs. Throughout training, on trials in which the monkey selected the saccade target in the neuron’s response field LIP activity tended to increase gradually during motion viewing, stay elevated during the subsequent delay period and then increase just prior to saccade onset. On trials in which the other target was selected, LIP activity tended to decrease gradually during motion viewing and then remain below baseline levels until after the saccade. The peri-saccadic activity tended to be smaller in the first ~15 sessions (average activity from −0.5 to −0.1s before saccade onset=18.6 spikes/s from sessions 1–15 and 31.3 spikes/s from sessions 16–160, t-test p=0.0157 for monkey C; 27.0 spikes/s from sessions 1–15 and 48.6 sp/s from sessions 16–130, p=0.0024 for monkey Z) but then remained relatively stable throughout the rest of training (linear regression, H₀: slope of average activity versus session=0, p=0.1174 for monkey C; p=0.3124 for monkey Z). In contrast, the responses during motion viewing changed substantially throughout training, with the rates of rise and fall becoming steeper and increasingly dependent on motion strength (higher coherences corresponding to steeper slopes).

Figure 4.

LIP responses. a. Average activity of LIP neurons as a function of viewing time (using 0.1-s-wide time bins with 0.025-s increments) for different motion strengths (see legend) for saccades into (solid line) and out of (dashed line) each neuron’s response field during different training periods for monkey C. Only correct trials were included. b. Coherence-, viewing time- and coherence × viewing time- dependence (Eq. 3) of individual LIP neurons before and during training for monkeys C (left) and Z (right). Error bars are 68% CIs. Solid lines are significant linear fits (p<0.05 for H₀: slope=0). c. Coherence-specific effects of training on the rate of rise of LIP activities during motion viewing for monkeys C (top) and Z (bottom). The rate of rise was estimated separately for each coherence using a piecewise-linear function (Eq. 6 with the coherence-dependence term, β₁, set to zero). Points and error bars are the slope and 68% CIs of a linear regression relating this rate of rise to session number (* indicates p<0.05 for H₀: slope=0).

To quantify the effects of training on the coherence-dependent LIP responses, we fit spike rate data from individual neurons to the same linear model we used for the MT responses (Fig. 4b). For both monkeys, the coherence- and time-alone terms tended to be near zero throughout training (although the coherence term decreased slightly to small negative values with training in both monkeys, and the time term increased slightly to small positive values with training in monkey C). A larger effect was seen in the coherence × time interaction term, which began near zero and thus indicated little or no influence of the motion stimulus on the LIP responses at the beginning of training, and progressed steadily to more positive values. Because these neurons were selected based on spatially tuned pre-saccadic activity (see Methods, Fig. S5), this result implies that training both established and then shaped the sensory-driven responses in these sensory-motor neurons.

The changes in sensory-driven LIP responses reflected the improvements in behavioural sensitivity to weak motion signals. The pattern of coherence-specific changes in LIP was similar to the pattern of coherence-specific behavioural improvements: both the rate of rise of stimulus-driven LIP responses and discrimination performance increased as a function of training session for most non-zero coherences, with greater increases for higher coherences (H₀: slope of a linear regression between session number and either average LIP rate of rise or percent correct for each coherence=0, p<0.05 for coherences ≥6.4% for LIP data and ≥3.2% for behavioural data for monkey C; for coherences ≥25.6% for LIP data and ≥12.8% for behavioural data for monkey Z, Fig. 4c). Moreover, LIP activity was correlated with behavioural threshold across sessions after taking into account the high-coherence errors (Fig. 5, left; partial correlation r_th,(k3|λ) = −0.42, H₀: r=0 using Fisher’s Z transformation, p<10⁻⁵ for monkey C; r_th,(k3|λ) =−0.35, p=0.0049 for monkey Z), but the converse was not true, with no correlation between high-coherence errors and LIP activity after taking into account the changes in behavioural threshold (r_λ,(k3|th) =0.0460, p=0.6198 for monkey C; r_λ,(k3|th)= −0.0590, p=0.6011 for monkey Z).

Figure 5.

Relationship between the coherence- and time- dependent LIP responses (k₃, Eq. 3) and various behavioural, motor and motivational parameters. The r-values for the behavioural parameters (left two columns) are the partial correlations between each parameter and k₃ with the effect of the other parameter on k₃ removed. Other r-values are the correlation coefficients between that behavioural parameter and k₃. ♣ indicates a significant correlation between the behavioural parameter and k₃ (p<0.05). * indicates that the behavioural parameter changed significantly as a function of training session (linear regression, p<0.05; see Table S1). Error bars are 68% CIs.

The changes in sensory-driven LIP responses with training also did not appear to reflect changes in other motor (Fig. 5, middle) or motivational parameters (Fig. 5, right) known to modulate LIP activity^{21, 22}. Some of these parameters changed with training, but none were consistently correlated with the changes in LIP responsiveness over the course of training for both monkeys (Fig. 5 and Table S1).

Comparison of behavioural, MT and LIP data during training

In trained monkeys, MT and LIP are thought to play different but complementary roles in forming the direction decision: MT (possibly in tandem with other motion-sensitive areas like the middle superior temporal area, or MST) provides sensory evidence used to form the decision, whereas LIP (possibly in tandem with other sensory-oculomotor areas like the FEF and superior colliculus) accumulates and interprets the sensory evidence to form the categorical judgment that instructs behaviour²³. Our results suggest that the improvements in behavioural sensitivity result from changes not in the sensory representation but rather its interpretation.

To test this idea and more directly compare the MT, LIP and behavioural data, we used a sequence of three nested models. The first model describes the coherence-dependent sensory evidence and was fit to MT data. The second model builds on the first, describing the accumulation over time of the sensory evidence into a decision variable, and was fit to LIP data. The third model builds on the second, describing the monkey’s decisions in terms of the value of the LIP decision variable, and was fit to behavioural data^{29, 32}(Fig. 6a). The advantage of this scheme is that MT, LIP and behavioural data can be fit separately but then compared directly via a term common to the three models describing how the sensory evidence used to form the decision scales with motion coherence (a_MT, a_LIP and a_be in Fig. 6a). For both monkeys, the value of this motion-sensitive term grew as a function of training session in a similar manner for data from behaviour and LIP but not MT (Fig. 6b,c; monkey C: correlation coefficient between MT and behaviour r=0.03, H₀: r=0 using Fisher’s Z transformation, p=0.7874; LIP and behaviour r=0.59, p<10⁻¹²; monkey Z: MT and behaviour r=−0.14, p=0.2911; LIP and behaviour r=0.60, p<10⁻⁹). A straightforward interpretation of this result is that training does not affect the representation of sensory evidence in MT directly but effectively scales its output, providing increasingly sensitive evidence to the decision variable in LIP used to guide behaviour.

Figure 6.

Decision model. a. Schematic of the decision model and example fits to behavioural, MT and LIP data. The decision model assumes that MT represents the coherence-dependent sensory evidence, LIP accumulates over time this sensory evidence into a decision variable, and the monkey’s choice depends on the value of this decision variable. The model allows us to fit separately data from MT, LIP and behaviour but extract a common parameter: the coherence dependence of the sensory information represented in each stage of processing (a in Eq. 4). b. Coherence dependence (best fit values and 68% CIs) computed from behavioural (left axes, black symbols) and neural data (right axes, cyan symbols for MT data, red symbols for LIP data). Solid lines are linear fits (H₀: slope=0, monkey C: behaviour p<10⁻¹⁰, MT p=0.1056, LIP p<10⁻¹³; monkey Z: behaviour p<10⁻¹⁰, MT p=0.6349, LIP p<10⁻¹³). c. Relationship between a computed from behavioural data and neural data. ♣ indicates a significant correlation (p<0.05). Error bars are 68% CIs.

Specificity of learning

Perceptual learning is typically specific to the stimulus attributes used during training, including motion direction in a direction-discrimination task³³. This phenomenon helps to distinguish perceptual learning from cognitive or motor changes and has been used to argue that the underlying changes occur at early stages of sensory processing². For our task, the stimulus configuration used in a given session depended on the characteristics of the MT and/or LIP neuron(s) being recorded and thus varied slightly from session to session (a factor that in of itself can affect the specificity of learning³⁴). We tested for the specificity of learning by analyzing performance relative to the similarity of the current stimulus configuration to configurations used in previous sessions.

For both monkeys, discrimination performance depended on the familiarity of the axis of motion (Fig. 7). We calculated for each session the difference between the motion sensitivity computed from behavioural data (a in Eq. 4) and its 21-session running average. This quantity provides an estimate of discrimination performance relative to its current trend: a positive value implies better-than-average performance, a negative value implies worse-than-average performance. For both monkeys, the value of this quantity was negatively correlated with the absolute z score of motion direction (monkey C: correlation coefficients r=−0.39, H₀: r=0 using Fisher’s Z transformation p<10⁻⁷; monkey Z: r=−0.49, p<10⁻⁹), indicating that performance tended to degrade when an unfamiliar stimulus was used. This specificity for the axis of motion was mirrored in neural activity in LIP (Fig. 7b, red symbols; monkey C: r=−0.37, p=0.0004; monkey Z: r=−0.29, p=0.0204) but not MT (Fig. 7b, cyan symbols; monkey C: r=0.09, p=0.5439; monkey Z: r=0.22, p=0.2063), which is consistent with the idea that perceptual learning can be specific by virtue of what is learned (in this case, the LIP decision variable) and need not involve changes in the sensory representation (direction-selective responses in area MT)³.

Figure 7.

Specificity of learning. a. The coherence dependence of the sensory information (a in Eq. 4) estimated from behavioural performance from sessions 30–50 for monkey Z. The solid line is a 21-session running average. b. The difference between the coherence dependence from a given session and its 21-session running average for behaviour (black), MT (cyan) and LIP (red) responses are plotted against the absolute z score of motion direction for monkeys C (top) and Z (bottom). For a given session, the z score is computed using the distribution of motion directions used prior to that session. Thus, less frequently used motion directions will have larger z scores. Solid lines are linear fits.

Discussion

Our results show that for monkeys trained to discriminate the direction of motion of noisy visual stimuli, improvements in discrimination threshold (i.e., higher sensitivity to weak motion) corresponded to changes in sensory-driven responses in area LIP but not MT. In MT, individual neurons responded similarly to motion stimuli before and throughout training, with, on average, a sensitivity that was similar to that of trained monkeys. However, responses of the most sensitive MT neurons became increasingly predictive of the monkeys’ choices with training. In LIP, neurons tuned for saccadic direction were insensitive to visual motion at the beginning of training but developed responses that grew increasingly strongly with motion strength and viewing time as performance on the task improved. The results suggest that the perceptual improvements corresponded to an increasingly selective read-out of highly sensitive MT neurons by a decision process, represented in LIP, that instructed the behavioural response.

Because the motion sensitivity of MT neurons likely arises from their direct and indirect input from V1³⁵, our results appear to rule out learning-induced changes in V1, as well. This finding is consistent with electrophysiological studies in monkeys that found little or no changes in V1 with perceptual learning^7–9 (although larger effects have been found in V1 of human subjects using fMRI³⁶) and provides additional evidence that the stimulus specificity of perceptual learning does not necessarily imply changes in primary sensory cortex^{3, 4}. Our results also further distinguish early visual cortical areas from primary somatosensory and auditory cortices, which in both monkeys and humans can show dramatic changes with perceptual learning^{5, 6, 37–39}. We cannot rule out the possibility of changes in other parts of extrastriate visual cortex that carry appropriate motion signals, like area MST⁴⁰. Nevertheless, it is striking that we found no changes in MT sensitivity over the course of training despite the established role of MT in task performance^16–19; the increase in MT choice probabilities with training, indicating an increasingly strong correspondence with behaviour; and the fact that our recording and analysis methods could resolve within-session changes in MT sensitivity³⁰ that were smaller in magnitude than the longer-term changes that we would expect based on behaviour.

The changes we report for LIP complement and extend previous findings. Learned sensory-motor associations have been shown to correspond to changes in LIP but not MT^{41, 42}. Consistent with those findings, we found that motion-driven LIP responses appeared when the monkeys learned the association between motion direction and saccadic response. However, unlike previous studies we showed that both behaviour and LIP responses continued to evolve well after the visuomotor association was established, together reflecting improved sensitivity to weak motion. These results support the intriguing possibility that associative and perceptual learning might share common mechanisms^{4, 43}. Learning a stimulus-response association establishes functional connectivity from neurons that represent the sensory stimulus to neurons that control the motor response. Improvements in perceptual sensitivity might then involve refinement of this connectivity to provide a more selective read-out of the most sensitive sensory signals associated with that response. Our results are consistent with such a refinement of direct or indirect ascending projections from MT to LIP⁴⁴, an idea that merits further testing.

We do not know the exact role that LIP plays in task performance throughout training. In trained monkeys, multiple brain areas including LIP, the superior colliculus and parts of the prefrontal cortex including the FEF exhibit similar sensory-motor responses during performance of the discrimination task^{24, 25, 27, 28}. We do not know what, if any, differences exist in the contributions of these brain areas to task performance either during or after training. Moreover, LIP plays a multitude of cognitive and sensory-motor roles that could, in principle, change during training and account for the changes in LIP responses during motion viewing ^20–22. However, we found no evidence for such modulation by numerous oculomotor and motivational parameters. Instead, the sensory-driven LIP responses changed in a manner that was consistent with a decision process that used increasingly sensitive motion evidence to determine the saccadic choice. These decision computations represented in LIP appear to be critical for performance of trained monkeys, an idea supported by a close relationship between LIP responses and reaction times²⁵ and the ability to bias performance using electrical microstimulation of LIP²⁶. Additional work is needed to establish similarly strong links between LIP activity, decision-making and behaviour throughout training.

The results also further support a close relationship between learning and attention¹². Of the learning-induced changes that have been found previously in V1, attentional modulation appears to play a primary role⁸. Attentional modulation might likewise account for the slight overall improvement in MT sensitivity in our data when comparing sessions before and during training or, like in several other studies, within training sessions^{30, 45}. The changes in choice probabilities in MT and motion-driven responses in LIP might also be thought of in terms of improved attention to appropriate features of the motion representation used to form the decision. This idea is parsimonious with the widely reported role of LIP in spatial and feature-based attention^{20, 46} and the relationship between attention and perceptual learning on other tasks¹². It remains to be seen how general a role this kind of mechanism plays in different forms of perceptual learning, which can have much different time courses⁴⁷, attentional demands¹² and magnitudes⁴⁸.

Methods

Behavioural task

The motion stimulus was a random-dot kinematogram described elsewhere^{16, 29}. Motion direction, coherence, and duration (a random time between 0.1–1.4s from an exponential distribution) were varied randomly from trial-to-trial. Correct responses and half of the 0% coherence trials elicited a juice reward of a variable amount (1–4 drops). Incorrect responses were followed by a “time out” period lasting 1–5s. Task difficulty was held relatively constant by adjusting the distributions of coherences and viewing times to give ~70–80% correct per session. This design kept the total reward per session roughly constant and helped to keep the monkeys motivated throughout training. Eye position was monitored using a video-based system (ASL, Bedford, MA) sampled at 240 Hz to enforce fixation during motion viewing and register the saccadic response.

Electrophysiology

Monkeys were prepared for the experiments by surgical implantation of a head-holding device and recording cylinders. MT and LIP were targeted using sterotaxic information and magnetic resonance imaging (Fig. 1b). Neural activity was recorded using quartz-coated platinum-tungsten microelectrodes advanced using two Mini Matrix systems (Thomas Recording, Inc., Giessen, Germany), one per recording cylinder. Spike waveforms were stored and sorted offline (Plexon, Inc., Dallas, TX). All training, surgical and experimental procedures were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals and were approved by the University of Pennsylvania Institutional Animal Care and Use Committee.

We searched for MT neurons with consistent spatial, direction and speed tuning, measured using a 99.9% coherence stimulus (Fig. S1 and S3). We searched for LIP neurons using a delayed-saccade task and selected neurons with spatially tuned delay-period activity²⁵ and response fields in consistent locations (Fig. S1 and S5).

Analysis of behavioural data

We fit behavioural data to a time-dependent cumulative Weibull function in which discrimination threshold is a power function of time:

$$p(C,T)=0.5+(0.5-\lambda )\times (1-e^{-{(\frac{C}{\alpha T^n})}^{\beta }}),$$ [1]

where P is the discrimination performance at motion strength C (in percent coherence) and viewing time T (in seconds), λ is the fraction of errors measured for stimuli of 99% coherence and viewing times > 0.4s, and with fitted parameters α (threshold coherence at 1s), n (time exponent) and β (shape parameter).

Analysis of MT and LIP data

To quantify the effects of motion coherence and viewing time on the responses of MT and LIP, we first normalized the responses of each neuron:

$$\widehat{r}(C,T)=\frac{r(C,T)-r_{BL}}{r_{97.5}-r_{BL}}$$ [2]

where r(C,T) is the difference in responses (in spikes/s) at coherence C and viewing time T to motion towards versus away from the neuron’s preferred direction (for MT) or preferred choice (for LIP), r_BL is the average baseline response 0.2–0s before motion onset and r_97.5 is the 97.5-percentile response at 99.9% coherence preferred motion.

This normalized response (MT: from 0.1–1s; LIP: from the beginning to the end of the ramp activity, which were estimated by fitting a piecewise-linear function (Eq. 6) to the neuron’s responses to 99.9% coherent motion) was then fit to a simple linear model:

$$\widehat{r}(C,T)=k_0+k_{1}C+k_{2}T+k_{3}CT$$ [3]

with parameters that describe the effect of coherence (k₁), viewing time (k₂) and coherence × time interaction (k₃) on the neuron’s response.

In addition, we also computed the sensitivity (“neurometric” discrimination thresholds)¹⁶ and choice probability¹⁷ of each MT neuron. Sensitivity was computed using an ROC analysis of distributions of MT responses separated by motion direction, computed separately for each motion strength and cumulative bins of viewing time from 0–1s in 0.05-s steps and fit to Eq. 1. Choice probability was computed from the distributions of MT responses to 0% coherence stimuli, separated by the monkey’s choices.

Model of the decision variable

We modelled the decision variable as the accumulated difference in activity between noisy pools of motion sensors tuned to the correct and incorrect directions of motion²⁹:

$$\begin{array}{c}\langle S_c\rangle =(R_0+a{C}^m)T^n\\ \langle S_i\rangle =R_0{T}^n\end{array},$$ [4]

where C is fraction coherence, T is viewing time (in seconds), R₀ is the response (in spikes/s) of MT neurons for 0% coherence, 〈 〉 denotes expectation, and a, m and n are fitted parameters. We assume that S_c and S_i are normally distributed with variances that scale with their mean $\varphi =\frac{\mathit{Var}(S_c)}{\langle S_c\rangle }=\frac{\mathit{Var}(S_i)}{\langle S_i\rangle }$, A correct response results when S_c>S_i. If S_c and S_i are independent and normally distributed, then the probability of a correct response, P, is:

$$P(C,T)=\frac{1}{\sqrt{2\pi {\sigma }^2}}\underset{0}{\overset{\infty }{\int }}{e}^{-\frac{{(x-\mu )}^2}{2{\sigma }^2}}dx,$$ [5]

where μ is the expected value of the accumulated difference, 〈S_c − S_i〉 = 〈S_c〉− 〈S_i〉 and σ² is the sum of the variances of S_c and S_i.

We used different versions of the model to estimate the coherence-dependence of the decision variable (the parameter a in Eq. 4) separately for behavioural, MT, and LIP data. For behaviour, we fit Eq. 5 to performance. For MT data, we estimated the linear dependence of preferred–null direction responses averaged over the full viewing duration, consistent with the difference between the time-independent portions of 〈S_c〉 and 〈S_i〉. For LIP data, we fit

$$F(C,T)=\{\begin{array}{ll}\gamma \hfill & ,\text{ if T}<\tau \hfill \\ \gamma +({\beta }_0+{\beta }_{1}C)(T-\tau )\hfill & ,\text{ if T}\ge \tau \hfill \\ {\beta }_2\hfill & ,\text{if }F(C,T)>{\beta }_2\hfill \end{array}$$ [6]

to the difference in activity between leftward and rightward choices, letting β₀, β₁, β_2, τ, and γ be free parameters and reporting β₁ as the coherence-dependent parameter (equivalent to a in the behavioural model). These estimates are expected to be offset from each other by an overall scale factor because of different pooling assumptions: we fit data from individual MT neurons, from individual LIP neurons that were assumed to pool from an unknown number of MT neurons and from behavioural data assumed to reflect both MT and LIP contributions; note the different axes in Fig. 6 for behavioural and neural data. Nevertheless, these fits are useful for comparing how each estimate changes with training.