Towards a whole brain model of Perceptual Learning

Abstract

A hallmark of modern Perceptual Learning (PL) is the extent to which learning is specific to the trained stimuli. Such specificity to orientation, spatial location and even eye of training has been used as psychophysical evidence of the neural basis of learning. This argument that specificity of PL implies regionalization of brain plasticity implicitly assumes that examination of a singular locus of PL is an appropriate approach to understand learning. However, recent research shows that learning effects once thought to be specific depend on subtleties of the training paradigm and that within even a simple training procedure there are multiple aspects of the task and stimuli that are learned simultaneously. Here, we suggest that learning on any task involves a broad network of brain regions undergoing changes in representations, read-out weights, decision rules, attention and feedback processes as well as oculomotor changes. However, importantly, the distribution of learning across the neural system depends upon the details of the training procedure and the characterstics of the individual being trained. We propose that to advance our understanding of PL, the field must move towards understanding how distributed brain processes jointly contribute to behavioral learning effects.

Introduction

Perceptual Learning (PL) refers to changes in our perceptual processes due to experience and is fundamental to perceptual development, formation of perceptual expertise, and rehabilitation after sensory damage. From a scientific perspective, PL represents one of the most studied perceptual phenomena (with the first systematic investigation dating back to the end of XIX century, i.e., Volkman, 1858) and has been observed in virtually all the human senses, with examples in vision [1], hearing [2], touch [3], taste and smell [4,5] providing evidence that experience can improve human performance on essentially any perceptual tasks. This has given rise to numerous approaches aiming to exploit PL in the development of training interventions for different categories of individuals; such as athletes [6,7], medical experts [8], or people with visual deficits; such as amblyopia [9,10], myopia [11,12], presbyopia [13], macular degeneration [14–16], age-related visual decline [17–19], Autism Spectrum Disorder [20,21], dyslexia [22], among others. However, while research provides promise that PL has potential to improve perceptual skills in those seeking expertise or rehabilitation, such interventions are limited by lack of understanding of both the behavioural attributes of PL and the brain plasticity that underlies it. Here we suggest that to understand and maximally exploit PL, it is necessary to know how training with different tasks and in different individuals gives rise to different behavioural and neurological outcomes.

The first key to understanding the field of PL rests in the behavioural aspects of learning. In the present manuscript we predominantely focus on vision research, which has largely dominated the field of study [23]. Most contemporary approaches to PL train on simple tasks that are thought to target perceptual primitives; such as stimulus orientation [24–26], motion [27–29], contrast [30,31], hyperacuity [32], texture discrimination [33–35], acuity [17], crowding [36–38], binocular processing [39,40] or processing speed [41], among others. A common finding is that PL can be exquisitely specific to trained stimulus features such as the spatial location and orientation [25,32,42], and even the eye of training [1,43]; as summarized in Figure 1. While the degree of specificity depends on training conditions and individual differences [24,32,33], stimulus specific learning is often considered necessary to discriminate PL from simple performance improvements due to learning the task structures. Further, the specificity of PL suggests that learning may be a singular process for which learning can simply be characterized.

Figure 1.

Specificity of PL. The figure shows training displays where training at one location, one orientation or with one eye (left panel) might not transfer to the same stimulus at a different location, orientation, or eye, respectively (right panel).

The exquisite specificity of PL has largely defined the field as it has been thought to provide clues into the neural systems underlying learning. A common view suggests that behavioural specificity puts constraints on the candidate neural representations underpinning learning. This representational view of PL led researchers to postulate a correspondence between features of behavioral specificity and the responsiveness of neurons in a given brain area to those features; suggesting that learning is due to plasticity in that brain area. For example location, orientation and ocular specificity at the behavioral level may be due to plasticity in primary visual cortex, which contains neurons selective to orientation, location and eye. An alternative framework suggests that PL reflects improvements in the ability of higher-level areas to read out task-relevant sensory information in the service of task-related decisions [44,45]. From a computational framework, these read-out models provide simple and parsimonious explanations of PL and have served as a counterpoint to the suggestion that the specificity of PL implies regionalization of plasticity in the brain.

These contrasting theories have led to a binary view of PL that has largely dominated the field for the last 40 years, where significant research focus has been directed to proving that PL is either due to changes in representation of a specific brain region, or that it can be better explained by decision processes reading out information from these representations. However, there has been a recent proliferation of alternative theories arguing for multistage models of learning [44,46,47] or in the most radical view that PL is best explained as a form of ‘conceptual’ learning [24]. These models attempt to fill the explanation gap in the binary view of PL, which has been useful in driving research to understand the contributions of different brain processes in learning, but it is insufficient to explain how plasticity in multiple brain areas, and interactions across the neural system, jointly contribute to the observed behavioural changes.

In the present manuscript we review recent research and propose a new whole brain framework to understand PL. The key to this model is to acknowledge that PL is not a singular process, but instead leads to multiple components of learning that combine together to produce changes in performance. We suggest that without a whole brain model of PL one cannot adequately explain how small changes in task structures lead to different behavioural outcomes or characterize the substantial individual differences in learning often found within and across studies. Further progress in the field will ultimately require understanding of how learning experiences can give rise to plasticity across a broad set of brain systems and how this distribution of changes manifests differently through training with different tasks and in different individuals.

The Neural Basis of Perceptual Learning

Numerous studies, using a variety of techniques (e.g. psychophysics, EEG, fMRI, TMS, animal electrophysiology, computational modelling, etc), have attempted to identify specific brain areas as being responsible for specific examples of learning [30,42,48,49]. We review some key findings from these studies and attempt to categorize them as being related to low-level representation changes, higher-level representation changes, changes in read-out, changes in decisions, changes in attention or other factors (see Figure 2). In reading these it is important to consider that similar tasks are used in research that attempts to localize learning to different stages of processing and that these explanations for the neural bases of PL are not mutually exclusive.

Figure 2.

Multiple mechanisms of Perceptual Learning. Learning has been attributed to low-level representations (red), higher level representations (green), Read-out (blue arrow), Attention/Feedback (yellow arrow) and decisions, as well as other processes. Together, these involve the whole brain.

Evidence for low-level representation changes

In the context of the binary view of PL, the holy grail for the representational view of PL has been to provide evidence that primary visual cortex (V1) undergoes plasticity. Supporting evidence comes from neuroimaging studies showing functional activity changes [25,48,50–52] and electrophysiological studies in animals showing receptive field changes in single units [53–55], both consistent with plasticity in V1. For example, Shibata et al [48], used an fMRI online-feedback method to induce activation patterns in early visual cortices (V1/V2) corresponding to those evoked by an oriented Gabor and found that changes in activity in V1 coincided with performance improvements in detecting the trained orientation pattern. Interestingly, PL has recently been shown to affect even earlier, pre-cortical neural loci: Yu et al [30], reported an increase in neural response to low contrast stimuli in the M-layers of the Lateral Geniculate Nucleus (LGN) that correlated with PL of contrast detection. While these and other studies provide compelling evidence that low-level representation areas can exhibit PL–induced plasticity, it must be noted that the proof of concept that plasticity can be observed in V1 is not evidence of V1’s exclusive role in the reported changes in behavioural performance. On the contrary, while Schoups et al [53] found electrophysiological evidence for orientation tuning curve changes in monkey V1 as a result of training, they estimated that these neural changes were sufficient to explain less that 10% of the performance changes.

Evidence for higher-level representation changes

While primary visual cortex has been a particular focus of PL, there are numerous other visual processing areas that each show unique patterns of featural sensitivity (e.g. MT to motion, V4 to curvature and color, FFA to faces, PPA to houses and scenes, etc), and which are also involved in PL. For example, in the case of learning orientation discrimination, observations of PL from animal electrophysiology have been mixed in V1 (e.g. [56]) but robust in V4 [49,54]. Further, many PL studies train with stimulus features such as shapes, objects and faces [57–60] for which plasticity in primary visual cortex would be insufficient to account for learning. For example, Bi and colleagues [60] showed that training on face orientation showed a high degree of specificity for the stimuli used, and stabilized facial representations in the fusiform face area (FFA [61]). PL has been found to lead to changes in response properties in numerous visual representation areas including; motion in V3a [62], MT [63] and LIP [64], orientation in V1 [53], V4 [49,54] and PIT [42], contours in LO [65], faces in FFA [61], among others. Further, these higher areas better account for behaviorual learning with plasticity in V4 explaining 25% or more [49,66]of observed perceptual learning effects of orientation discrimination in monkeys.

Evidence for read-out from representation areas

While there is substantial evidence supporting the idea that brain areas involved in representing visual information can undergo plasticity, computational models demonstrate that plasticity in visual representations is not necessary to explain specificity of PL [67]. The basic insight that these models provide is that even if representations are fixed during learning, changes in read-out weights between perceptual representation and decision areas can parsimoniously account for most behavioural findings of PL. For example, Dosher and colleagues [68] proposed an integrated reweighting theory (IRT) that relies upon a combination of location specific and location invariant representation to account for transfer of learning to new retinal locations. Talluri et al. [46] proposed a simplified version of the IRT that accounts for individual differences and recent evidence of retinal transfer of learning, as reported in studies using double training [32,69], which is a paradigm where after training that yields stimulus specificity, additional practice on an irrelevant task at a different retinal location, unlocked transfer of the intial learning to the untrained retinal location, putting into question the characteristic of location specificity of classic PL [32, 69]. These models make the important point that the behavioural specificity found in PL offers insufficient evidence to conclude that a visual representation has been changed.

Recent physiological studies in monkeys [70] and functional imaging in humans [71] provide fascinating neuroscientific evidence of read-out mechanisms in the brain. Chowdhury et al. [70] used a GABA agonist (muscimol) to reversibly inactivate monkey MT area to show that performance on a coarse-depth discrimination task was dependant upon MT activity. They then trained the animals on a fine-depth discrimination task (for which a prominent role of V4 is hypothesized) and made the surprising discovery that MT inactivation no longer impaired performance on this task. Chang et al. [71] performed a very similar study in humans finding that extracting depth cues in noise was disrupted by transcranial magnetic stimulation (TMS) to posterior parietal cortex (PPC) prior to training on a fine-depth discrimination task, but not after. Similar results were found by Chen, Peng, Zhou, Thompson and Fang [62] who reported that TMS of MT+ disrupted the ability to filter out a noisy motion direction display prior to, but not after, training a motion discrimination task. Showing the opposite direction of effect, Liu and Pack [72] found that inactivation of MT had little effect on motion perception of moving gratings in monkeys until after training the animals on a motion direction discrimination task with random dots. These studies provide direct evidence that PL can change the brain systems that are read-out to solve a given perceptual task.

Evidence for Attention and Feedback

Another body of research proposes that changes in top-down attentional modulation may explain behavioral findings of PL. For example, Poort et al. [73] showed that visual learning in mice increases V1 selectivity for task-relevant stimuli, leading to enhanced stimulus discriminability at the population level. These learning-induced enhancements of stimulus representation in V1 diminished when mice were not engaged in the task, suggesting the involvement of top-down signals in increasing population-level discriminability. Learned changes in low-level response properties may then be a result of top-down attentional modulation affecting sensory processing by enhancing the target signals and/or attenuating responses evoked by task-irrelevant features [74–76]. Further, Itthipuripat and colleagues [77] suggested that PL relies on two different selective attention mechanisms exhibiting different temporal courses: At first, performance improvements are related to attentional gain amplification, while a later stage relies on a noise reduction mechanism. Bays et al. [78] found that PL was associated with more efficient deployment of attention after learning. While these studies show that attention may explain some behavioral findings of PL, other studies control for attention (e.g. [49]), or find learning without attention [79], and suggest that not all findings of PL are easily explained as attentional learning.

Evidence for other explanations of PL

While most models of PL have focused on how representations, read-out, or attention may best explain observed changes in behavior, a number of other models and mechanistic explanations have been proposed. For example, Guidotti et al. [80] showed how PL increased the frequency and similarity of fMRI resting state activity in the cortical regions engaged by the training task. Diaz et al. [81] found that enhancement of the single-trial amplitude of late (decisional) but not early (sensorial) EEG components predicted PL. Chen et al., [82] report that motion direction discrimination training led to a sharpening of the cortical tuning to trained stimuli at the sensory processing stage (visual area V3A), as well as the optimization of the connectivity between this sensory area and a higher level, decision-making region in the Intraparietal Sulcus. Finally, neural plasticity as a product of training can manifest as a change in propensity for a system to learn new perceptual tasks [83]. For example, Kattner et al. [84] showed that training on a series of tasks sharing common components can induce transfer of learning to new task relying on similar components not in the form of immediate performance improvement but rather as increase in learning rate (‘learning to learn’). This is consistent with a recent model by Wang et al. [24] claiming that PL occurs at a ‘conceptual level’ where participants learn abstract rules that can be applied to novel features. At the other extreme, PL can help stabilize eye-movements in individuals with nystagmus [85] or to learn more efficient eye-movement strategies in situations of visual field loss [86,87]. Together these studies suggest that factors such as conceptual frameworks and propensity to learn new tasks, and, at least in clinical populations, oculomotor processes, can all make contributions to behavioural changes found through PL.

Towards a Whole Brain Model of Perceptual Learning

Evidence presented thus far shows that PL gives rise to learning at different levels of visual representations, in read-out from representations, in attentional feedback to representations, in decisions and in ocular-motor systems. If each of these systems can change then it seems unlikely that learning, even in the simplest tasks, is a singular process. Take, for example, a simple visual search to find a target line orientation among a set of distractor lines (Figure 3), Le Dantec and Seitz [88] showed that not only did participants show a classic result of improved search performance specific to the target orientation, PL also exhibited specificity to orientations of the distractor stimuli, the 24 (of 36) possible grid locations where the target appeared during training, and spatial context of the target and distractor stimuli. This demonstrates that multiple components of PL, which likely rely upon different brain systems, can arise together in the optimization of task performance. We suggest that learning any task is likely to rely upon a distribution of plasticity across the brain.

Figure 3.

Multiple stimulus properties are learned together. Figure shows schematic of a visual search task for an oriented target (indicated by red circle) that can appear in one of 24 trained (pink, upper left) or 12 untrained (blue upper left) locations, a trained set of distactor orientations (around the horizontal in the upper right), and spatial context of distractors and target (as seen in the right column). Training has been found to be specific to all of these features [e.g. 87].

Viewing PL as a distributed process may provide a useful framework to interpret recent findings of how the degree of specificity is mediated by attributes of the stimuli and tasks employed, and moderated by the individuals who are trained. For example, training with high-precision stimuli gives rise to more specificity and is hypothesized to lead to greater low-level learning than found through training with low-precision stimuli [32,46,89]. Other stimulus details such as target complexity [90] and stimulus adaptation during training [33] also impact generalization of learning. Further, recent studies show that training with multiple tasks ([29,45,91], but see [31]), directing exogenous attention towards the trained stimuli [92], training with more stimulus categories [93], all promote greater generalization of learning. In addition, characteristics of the individuals being trained moderate the specificity of learning [29,32], such as different levels of baseline performance [41,94], experience with action video games [83,95], personality traits and motivation [96], socio-cognitive profiles [97] and sleep habits [35]. We suggest that all of these factors may contribute to different distributions of learning across the brain systems that can contribute to PL.

Summary

While traditionally PL research has focused on identifying specific learning attributes and associating these with particular brain processes, accumulating evidence suggests that a multitude of brain processes contributes to PL. Further we observe that training on a simple task can give rise to dissociable components of learning that are likely related to plasticity in different brain systems. Complementing this observation is that a wide range of factors associated with the training conditions, as well as the individuals being trained, mediate and moderate the behavioural attributes of training. Together, these findings suggest that the premise that PL is a singular process that can be attributed to a single learning system is insufficient to explain the broad set of findings in the field. We propose that to advance our understanding of PL, new models need to be developed that account for how learning is jointly due to plasticity across multiple brain systems and that the distribution of this learning across the system is likely mediated by different training conditions and moderated by characteristics of the individuals being trained. For example, one can view PL as changing the “weights” of bi-directional connections between the units of a hierarchical network encompassing different visual and non-visual areas. Such a view can explain changes in early areas, intermediate areas, changes in top-down modulations (attention) and read-out. As a start towards this end, recent studies have found that deep neural networks can provide parsimonious fits for PL and provide a framework to see how learning is distributed across layers of the network and even to understand changes at the single unit level [98, 99]. While these models also have their limitations, they do provide some initial evidence of the utility of a whole brain model as route to better understand perceptual learning.

Highlights.

• Perceptual Learning research has identified numerous processes that change with learning.

• Identified mechanisms of Perceptual Learning are not mutually exclusive.

• Learning even a simple task involves changes a multitude of processes.

• Advanced understanding of Perceptual Learning will require a whole brain model of plasticity.

Biographies

**Chang, D.H.F., Mevorach, C., Kourtzi, Z., Welchman, A.E. (2014). Training Transfers the Limits on Perception from Parietal to Ventral Cortex. Current Biology, 20; 24(20): 2445–2450.

Authors used repetitive transcranial magnetic stimulation (rTMS) to show that PL shifts the functional role of cortical regions involved in a visual feature task: Before training, rTMS of the posterior parietal cortex (PPC) disrupted the identification of a target in noise, while rTMS of the lateral occipital (LO) circuit impaired feature difference judgement; after training, target extraction from noise was impaired with rTMS of LO but not of PPC. This result shows that PL generalization is implemented by replacing dynamic processing in the PPC with specific feature templates stored in the ventral cortex.

**Chen, N., Bi, T., Zhou, T., Li, S., Liu, Z., Fang, F. (2015). Sharpened cortical tuning and enhanced cortico-cortical communication contribute to the long-term neural mechanisms of visual motion perceptual learning. Neuroimage 115,17–29.

Authors trained participants on a motion direction discrimination task, showing a performance improvement accompanied by increase in V3a neural selectivity and connectivity between V3a and the intraparietal sulcus (IPS), a decision making cortical region, for the trained motion direction. This improvement was well explained by a linear combination of the selectivity and connectivity increases. These findings suggest that the long-term neural mechanisms of motion PL are implemented by sharpening cortical tuning to trained stimuli at the sensory processing stage, as well as by optimizing the connections between sensory and decision-making cortical regions.

**DeLoss, D., Watanabe, T., Andersen, J. (2015). Improving vision among older adults: Behavioral training to improve sight. Psychological Science, 26, 456–466.

Authors trained elderly participants on an orientation discrimination task with different levels of additive noise. Results showed that after training, elderly participants performed as well as college-aged participants at baseline. Additionally, learning transferred to untrained orientation. These results suggest that different levels of additive visual noise during training seem to induce greater transfer of learning to untrained features, likely because external noise filtering is a general process whose improvement might to benefit visual processing in a number of tasks.

**Diaz, J.A., Queirazza, F., Philiastides, M.G. (2017). Perceptual learning alters post-sensory processing in human decision-making. Nature Human Behaviour 1, 35.

Authors trained participants on a visual categorization task and used EEG multivariate pattern analysis to identify two temporally separated components encoding sensory (‘Early’) and decision (‘Late’) evidence. The single-trial amplitudes of the Late, but not the Early component, were amplified during training, and these enhancements predicted the performance improvements. Authors modeled these improvements with a reinforcement learning mechanism, using a reward prediction error signal to strengthen the read-out of sensory evidence used for the decision. Results showed a robust association between the model’s decision variables and the amplitudes of the Late component that encode decision evidence.

**Hung, S.C., Seitz, A.R. Prolonged training at threshold promotes robust retinotopic specificity in perceptual learning. Journal of Neuroscience 34 (25), 8423-8431.

Authors showed that trial difficulty during training represents a key factor in predicting learning vs generalization. In particular, training participants at thresholds gates learning and induces location specificity. Training with more suprathreshold stimuli (due to multiple short staircases), however, led to a significant transfer of learning between retinal locations. Moreover, the use of multiple staircase training led to location transfer even in the absence of double training.

**Kattner F., Cochrane A., Cox C.R., Gorman T.E., Green C.S. (2017). Perceptual Learning Generalization from Sequential Perceptual Training as a Change in Learning Rate. Current Biology, 27(6), 840-846.

Authors sequentially trained participants on different perceptual tasks, all sharing some higher-level structure or components, and showed that these common elements can be extracted and potentially learned, producing faster learning of new tasks. Generalization of learning thus can manifest itself not only through immediate performance improvement in a transfer task, as previous literature showed, but as an increased learning rate in a new task.

**Shibata, K., Sasaki, Y., Kawato, M., Watanabe, T. (2016). Neuroimaging Evidence for 2 Types of Plasticity in Association with Visual Perceptual Learning. Cereb Cortex. 26(9), 3681–3689.

Authors used neuroimaging techniques, to show evidence for the existence of two types of plasticity, feature- and task-based plasticity. The former refers to refinement of the neural representation of the trained feature, while the latter refers to improvement in task-related processing due to training on a task. Feature-based plasticity occurs in a specific visual area where the trained visual feature is mainly processed, while task-based plasticity occurs in higher-level, associative regions.

**Wang. R., Wang. J., Zhang. J.Y., Xie. X.Y., Yang. Y.X., Luo. S.H., Yu. C., Li. W. (2016). Perceptual Learning at a Conceptual Level. J Neuroscience.36 (7), 2238–2246.

Authors trained participants on an orientation discrimination and a motion direction discrimination tasks, showing that PL can transfer from training stimuli encoded by lower visual areas (luminance gratings and first-order motion signals) to untrained stimuli computed by higher sensory regions (symmetric dot patterns and second-order motion signals, respectively). This result indicates that PL can take place at a ‘conceptual’ level and generalize to stimuli with different physical properties.

**Yu, Q., Zhang, P., Qiu, J., Fang, F. (2016). Perceptual Learning of Contrast Detection in the Human Lateral Geniculate Nucleus. Curr Biol, 26 (23), 3176-3182.

Authors used fMRI techniques to show that PL of contrast detection results in eye- and hemifield-specific performance improvement accompanied by an increase in neural response to low contrast stimuli in the M-layers of LGN, thus suggesting that PL can promote plasticity not only in cortical, but in subcortical regions as well.