Two-stage model in perceptual learning: toward a unified theory

Abstract

Training or exposure to a visual feature leads to a long-term improvement in performance on visual tasks that employ this feature. Such performance improvements and the processes that govern them are called visual perceptual learning (VPL). As an ever greater volume of research accumulates in the field, we have reached a point where a unifying model of VPL should be sought. A new wave of research findings has exposed diverging results along three major directions in VPL: specificity versus generalization of VPL, lower versus higher brain locus of VPL, and task-relevant versus task-irrelevant VPL. In this review, we propose a new theoretical model that suggests the involvement of two different stages in VPL: a low-level, stimulus-driven stage, and a higher-level stage dominated by task demands. If experimentally verified, this model would not only constructively unify the current divergent results in the VPL field, but would also lead to a significantly better understanding of visual plasticity, which may, in turn, lead to interventions to ameliorate diseases affecting vision and other pathological or age-related visual and nonvisual declines.

Introduction

Visual perceptual learning (VPL) is defined as a long-term improvement in visual performance on a task resulting from visual experiences.^1–10 An expert in X-ray analysis, for example, can identify a tumor from the pattern of gray and black spots on an X-ray scan without much difficulty, whereas it is impossible for an untrained person to perform the task. Such feats are possible because the experts are trained through experience.

VPL indicates that, long after many aspects of sensory development have ceased in an early period of our life, repeated exposure or training can improve our visual abilities and cause reorganizations of neural processing in the brain. Therefore, VPL is regarded as an important tool that can help to clarify the mechanisms of adult visual and brain plasticity.^11–14 In addition, an increasing number of studies have found that training on a visual task can significantly strengthen the visual abilities of adult patients with amblyopia and other forms of abnormal vision^15–20 and ameliorate age-related visual declines.^21–23 These recent clinical successes are the result of translating basic VPL research findings into practical and powerful real-world applications.^24–28 Thus, a clearer understanding of the mechanisms of VPL in adults could lead to innovations in clinical intervention as well as advancement in basic science.

Three major directions in VPL research

In light of the steadily increasing body of evidence in the VPL field, a unified model is highly sought. However, a new wave of research findings has caused the field to diverge along three major directions. First, the mechanisms that govern the specificity and generalization of VPL remain controversial. Seminal psychophysical studies reported that VPL is highly specific for stimulus features (e.g., retinal location, orientation, contrast, spatial frequency, motion direction, background texture, and eye) that are trained or exposed during training.^29–44 However, recent studies have indicated that under some conditions VPL can be generalized to untrained features.^45–52 Second, the locus, or brain area/information processing stage, at which VPL occurs has been highly debated. One line of studies has suggested that VPL is associated with changes primarily in visual areas (visual model),^53–67 while the other line of studies has proposed that VPL arises from changes in higher-level cognitive areas that are involved in decision making, or changes in weighting between the visual and cognitive areas (cognitive model).^68–76 Third, it has been controversial whether task-relevant and task-irrelevant VPL (TR-VPL and TI-VPL, respectively) share a common mechanism or reflect distinct mechanisms.⁸ TR-VPL of a feature occurs owing to training on a task of the visual feature and is gated by focused attention to the feature in principle.^33,77–80 On the other hand, TI-VPL is defined as VPL of a feature that is irrelevant to a given task.⁸¹ It has been found that TI-VPL does not necessarily require attention to, and awareness of, the trained feature.^81–83 Some studies have suggested that the same or similar mechanisms^84–86 underlie TI-VPL and TR-VPL, while others have suggested distinct mechanisms.^1,45,87,88

Toward a unifying model of VPL

The findings described above reveal a substantially divergent picture of the current VPL field. Existing assembled models account for only a partial body of evidence. Instead, we propose a constructive approach that builds a unified VPL model from a global perspective. This approach attempts to comprehensively explain the existing divergent findings in the field.

Here, we propose a new theoretical framework, termed a two-stage model, built on the following two simple assumptions. First, there are two different types of learning: stimulus-driven learning (feature-based plasticity) and task-dependent learning (task-based plasticity). Second, feature-based plasticity occurs within visual areas, whereas task-based plasticity occurs in higher-level cognitive areas, where information from visual and nonvisual areas are integrated, or in the connections between the visual and cognitive areas.

The goal of this review is to summarize the aforementioned three major directions in the VPL field and describe how the two-stage model accounts for the divergent findings, and therefore leads to a comprehensive picture of VPL. Systematic examination of the validity of the two-stage model will be needed in future studies.

Specificity versus generalization of VPL

Specificity and generalization have been a point of divergence in the VPL field. A well-established trademark characteristic of VPL is its specificity; improvement in visual performance is largely specific to stimulus features such as retinal location,^33,35,38,42 contrast,^29,44 orientation,^34,36,37,40 spatial frequency,^32,36,40,42 motion direction,^30,31,41,43 or background texture³⁸ that were used during training. In other words, VPL is not generalized to other features. In addition, it has been shown that performance improvement is specific to an eye to which a trained stimulus is presented during training.^32,33,39,40 Such high specificity of VPL has been often regarded as the evidence for refinement of the neural representation of a trained visual feature.

The early general view that VPL occurs specifically for a trained feature has been challenged by recent VPL studies that reported generalization of VPL to untrained features under certain conditions.^45–52,89 These results support the view that VPL is mostly caused by improved task processing, and specificity of VPL is not explained by changes in low-level visual representations, but rather by higher level cognitive factors such as enduring attentional inhibition of the untrained features,^{48,50–52,90} selective reweighting of readout process to find specific visual representations which are the most useful for a trained task,^{46,69,75,76,91} or overfitting the trained neuronal network to accidental features of the specific stimulus that is used during training.^6,92

Lower versus higher brain locus of VPL

One of the controversies in the VPL field concerns the brain locus of VPL, that is, the neural processing stage that is changed in association with VPL. The accumulated findings can be generally framed in terms of one of two opposing models: the visual and cognitive models. Both models have amassed substantial psychophysical, physiological, and computational findings in their respective favors.

As mentioned, a number of psychophysical studies have shown that VPL is often specific to a trained feature^29–44 and eye.^32,33,39,40 These findings suggest that VPL involves changes in visual areas in which feature- and eye-specific information are processed.^93–95 In accordance with this visual model, physiological studies have reported changes in neural responses of visual areas to a trained feature,^54,96,97 even when the trained feature was merely exposed to subjects without any active task on the trained feature.^{53,56–58,60,65} Results of neuroimaging studies have also supported this view.^{55,59,61,62,64,67}

On the other hand, recent psychophysical and computational studies have proposed the cognitive model or connectivity model in which VPL arises from changes in higher level cognitive areas that are involved in decision making (e.g., the intra parietal sulcus, frontal eye fields, and anterior cingulate cortex) or changes in connection weights between the visual and cognitive areas.^{50,51,69,74–76} This argument is supported by studies that showed that VPL occurs in association with neural changes in cognitive areas or connectivity between visual and cognitive areas,^68,70,71,74 rather than only in visual areas.⁷²

TR-VPL versus TI-VPL

Another open problem concerns whether TR-VPL and TI-VPL are subserved by common mechanisms or by separate mechanisms. Since the initial discovery of TI-VPL,⁸¹ a number of studies have been conducted on TI-VPL.^{82,83,88,98–110} However, the mechanism of TI-VPL has yet to be well clarified. Some studies have developed models that focus on accounting for TR-VPL, but not for TI-VPL.^1,45,88 In contrast, other studies have proposed models that assume that TR-VPL and TI-VPL share common mechanisms.^84–86 In spite of this controversy, there have been scarcely any attempts to empirically examine the relationship between the neural mechanisms of TR-VPL and TI-VPL from a global perspective.

Multistage mechanisms of VPL

A potentially useful framework for understanding the aforementioned divergent results consists of multistage learning mechanisms. It has been generally assumed that a single mechanism governs VPL of a given task throughout an entire training period. However, if VPL is associated with changes in mechanisms in multiple stages, the seemingly contradictory findings described above could be regarded as a reflection of changes in different stages. As discussed below, several studies have shown the involvement of at least two qualitatively different stages in VPL.

A pioneering study by Karni and Sagi³⁹ indicated two different stages for TR-VPL of a texture-discrimination task. In an early phase of training on the texture-discrimination task, a rapidly saturating performance improvement occurred. After this initial improvement, no further performance improvement was found during training. However, prominent improvement occurred when the subjects’ performance was tested 8–20 h after the training period. Importantly, while the initial improvement in the early phase was transferable from a trained eye to an untrained eye, the improvement found several hours after the training period was monocular (i.e., no transfer to the untrained eye). These results suggest that the transferred learning component reflected the setting up of a task-related processing while the eye-specific learning component occurred at an earlier stage of visual processing after training was terminated. Thus, at least two qualitatively different stages of learning may be involved in TR-VPL of the texture-discrimination task.

Two different stages of learning were also reported in a study for TR-VPL of a global motion– discrimination task using a random dot–motion stimulus.¹⁰⁹ The random dot–motion stimulus in this study was designed so that the directions of local dots that compose the stimulus do not overlap with a global-motion direction that is defined by the mean direction of the local dots. During relatively early phases of time course of training on the global motion–discrimination task, motion sensitivities increased for the directions in which the local dots moved. However, as training proceeds further, motion sensitivities increased only at or around the trained global-motion direction. Importantly, the motion directions of the local dots did not include the global-motion direction, which was the target in the discrimination task. On the other hand, in a different condition in which the randomdot–motion stimulus was merely presented as irrelevant to a given task, increase of motion sensitivities was found only for the direction of the local dots, but not for the global-motion direction, even after an extended training period. These results indicate that learning gain for the directions of the local dots is obtained irrespective of whether the motion stimulus was TR or TI, and that there is learning gain for a global-motion direction only when the global-motion direction was TR. That is, learning on the local direction is based purely on the stimulus while learning on the global direction is only in response to task demands, since the global-motion direction was not physically presented during training. These results are also in accord with the hypothesis that the two different stages of learning occur at different temporal phases or at different speeds.

A recent neuroimaging study⁶⁷ gave important hints about the multistage mechanisms of VPL in the brain. In an early phase of training on the texture-discrimination task, performance increase was accompanied with activation enhancement of a trained region of the early visual area (V1). However, in a later phase, after performance on the task reached a plateau, V1 activation was reduced to the baseline level, while performance remained high. The observed time course of V1 activation might be a reflection of synaptic potentiation during the early phase of training and synaptic downscaling in the later phase after performance becomes saturated.⁶⁷ This finding suggests a multistage mechanism of VPL in which V1 is involved in the encoding of learning but not in the retention of already improved performance after learning, which might be supported by a mechanism in a different stage than encoding. This could explain why a number of single cell–recording studies have found very little evidence for changes in receptive-field properties in V1.

A recent psychophysical study reported that location specificity of VPL is related to long-term adaptation in early visual areas.⁸⁹ Results of the study showed that location-specific VPL occurs under conditions that involve sensory adaptation. However, VPL at one location is completely generalized to a new and untrained location when sensory adaptation is removed during training. Sensory adaptation, which develops within the visual system,¹¹¹ can occur passively without any task involvement,¹¹² persist for a long time,¹¹³ and selectively reduce performance on an exposed feature.^114–116 Together with the nature of sensory adaptation, the results of the study are in accord with the hypothesis that location specificity results from long-term sensory adaptation that may passively occur in visual areas. The generalization of VPL to untrained features after the removal of the adaptation⁸⁹ also suggests that plasticity related to the task occurs at a higher stage but is constrained by the location-specific signals owing to sensory adaptation. This view is also consistent with results showing that VPL becomes highly specific to a trained feature with a larger number of trials, while more generalization of VPL occurs from shorter training with a relatively small number of trials,¹¹⁷ since the shorter training leads to less sensory adaptation.¹¹⁸

The above-mentioned findings in these four studies suggest that TR-VPL derives from multistage neural mechanisms and that improved performance can be mediated by refinement of a representation of a trained feature and/or improvement in task-related processing. These results also suggest that the one learning type can precede the other, depending on the phase of training and the stimulus and task used during training.

A unified two-stage model

Here, we propose a new theoretical framework, which we call a two-stage model. The two-stage model is inspired by the findings about multistage mechanisms of VPL described above and may account for the aforementioned divergent results in the current VPL field.

The two-stage model

According to the two-stage model, there are two different types of plasticity—feature-based plasticity and task-based plasticity—that occur in different stages in visual information processing. Feature-based plasticity refers to refinement of the processing/representation of visual features (e.g., improved information transmission,¹¹⁹ gain, and bandwidth) that are used during training. This type of plasticity results from simple exposure to the features, irrespective of whether the feature is TR or TI, and results in specificity of VPL to the features (feature constraints). Task-based plasticity refers to improvement in task-related processing and results from active involvement in a trained task during training. Feature-based plasticity occurs within visual areas in which feature-specific visual signals are processed and/or feature representations are made, whereas task-based plasticity occurs in higher level cognitive areas that govern task-related processing.

How does the two-stage model account for the divergent results in the current VPL field?

The aforementioned divergent findings/controversial issues could be understood within the framework of the two-stage model. First, the two-stage model would clarify the relationship between TR-VPL and TI-VPL. TR-VPL involves both refinement of the processing/representation of visual features that were used during training (feature-based plasticity) and improvements in task processing (task-based plasticity) (Fig. 1A). On the other hand, TI-VPL only involves feature-based plasticity (Fig. 1B). In other words, TI-VPL is subserved by the lower-level component of TR-VPL. This view is consistent with both common and divergent aspects of TR-VPL and TI-VPL found in previous studies.^84–86,88

Figure 1.

The two-stage model in visual perceptual learning (VPL). (A) Task-relevant VPL (TR-VPL) reflects two different types of plasticity: feature-based and task-based plasticity. Contributions of feature- and task-based plasticity to overall performance improvement vary according to the phase of training and the task and stimulus used during training. (B) Task-irrelevant VPL (TI-VPL) is subserved only by feature-based plasticity. (C) TR-VPL can be generalized to untrained features if feature constraints are removed.

Second, the two-stage model would account for the current divergent findings about the brain locus of VPL. As described before, the visual model supports changes within visual areas in association with VPL,^53–67 whereas the cognitive model argues that VPL arises from changes in higher-level cognitive areas or changes in connections between the visual and cognitive areas.^68–76 In particular, it is controversial whether TR-VPL involves changes in the visual or cognitive areas, or both.⁸ From the viewpoint of the two-stage model for TR-VPL (Fig. 1A), this lack of clarity derives from two different types of plasticity and their different contributions to improved performance. For a certain task, stimulus, and training phase, improved performance in TR-VPL mainly depends on feature-based plasticity, and the resultant changes are observed in visual areas. In a different condition and phase, improved performance in TR-VPL is mainly based on task-based plasticity, which in turn appears as changes in higher-level cognitive areas or connectivity between the visual and cognitive areas, but not in the visual areas. This view does not accept a simple dichotomy between the visual and cognitive models, but rather supports multifaceted mechanisms for the loci of VPL in the brain. For example, the two-stage model may explain why changes in visual areas are observed in some studies but not in other studies of TR-VPL, because feature-based plasticity does not always occur during an entire period of training and its neural signature may be observable only in a part of the period. This view can account for both the involvement of the visual areas in human studies^{55,59,61,62,64,67} and little evidence of changes in neural activity in the visual areas in monkey single-unit recordings.^72,120 Animal studies have employed much longer training period (months) than human subjects (days) before neurophysiological measurements to examine neural changes caused by VPL.

Third, the specificity of VPL^29–44 can be accounted for as feature constraints that are assumed in feature-based plasticity. The two-stage model (Fig. 1C) predicts that generalization of TR-VPL to untrained features occurs if the feature constraints are removed or not involved during training. As mentioned previously, removal of sensory adaptation leads to generalization of TR-VPL.⁸⁹ This generalization can be regarded as a result of a lack of feature constraints during training. In addition, previous studies showed that generalization of VPL is obtained as a result of training that employs easy task trials.^1,45 It has been suggested that, in this condition, refinement of processing/representation of trained visual features does not occur, while performance increase is mediated mainly by improvement in task-related processing.^1,45 Furthermore, recent studies have reported that performance improvement owing to training on a task of a feature is generalized to an untrained location when the training is followed by an additional training on another irrelevant task at the untrained location (so-called double-training paradigm).⁵⁰ Although precise nature of the effects of the double-training paradigm is still under debate,^105,121 the reported generalization effect may be attributed to removal of feature constraints as a result of the additional training.

Relationships with existing models

What is the key difference distinguishing the two-stage model from existing models in the field? As described, the visual, cognitive, and connectivity models assume that TR-VPL involves changes in the visual areas, higher cognitive areas, and connection weights between the visual and cognitive areas. In the two-stage model, a key difference from these models is that TR-VPL occurs as a result of at least two different types of plasticity (feature and task-based plasticity) at different cortical stages. The reverse-hierarchy theory^1,87 proposed that TR-VPL is derived from changes in higher and lower cortical stages. There are two important differences between the reverse-hierarchy theory and the two-stage model. First, the reverse-hierarchy theory considers only TR-VPL while, as described above, the two-stage model covers both TR-VPL and TI-VPL within an integrated framework. Second, while the reverse-hierarchy theory assumes that learning at a lower cortical stage follows learning at higher cortical stage, the two-stage model retains flexibility in temporal order of different types of plasticity. These types of plasticity can occur in order or at the same time, depending on the task and stimulus. Indeed, task-based and transferrable learning may occur first in some cases,³⁹ while learning based on an exposed TI feature occurs first in other cases.¹⁰⁹

Conclusions and future directions

As described above, the two-stage model would constructively clarify the incompatibilities in the VPL field and synthesize current divergent findings into a global and comprehensive shape.

The assumptions of the two-stage model need to be empirically validated in future studies. The first touchstone is to test whether the two different types of plasticity (feature-based and task-based plasticity) involved in TR-VPL are experimentally dissociable. Feature-based plasticity refers to changes in the visual representation of a trained feature in visual areas, while task-based plasticity refers to changes in task processing in higher level cognitive areas. Thus, neural changes based on feature-based plasticity may be observed as changes in neural responses to the trained feature in the corresponding visual areas in association with TR-VPL, irrespective of whether subjects engage in a trained task on a feature or are merely presented with a feature that is irrelevant to a given task. On the other hand, neural changes due to task-based plasticity may be observed in higher cognitive areas or in connectivity between the visual and cognitive areas only when subjects engage in the trained task on the trained feature.

The second touchstone is to test whether the two different phases in TR-VPL are experimentally dissociable. The two-stage model predicts that contributions of the feature- and task-based plasticity to overall performance improvement vary according to a temporal phase of training and a task and stimulus used during training. The previous psychophysical studies suggested that performance improvement can occur on the basis of visual and task-related processing in different phases of training.^39,109 Thus, it is possible that in a certain phase of training neural correlates of improved performance are found in the visual areas, while in a different phase neural changes are observed in the cognitive areas or in the connectivity between the visual and cognitive areas.

The framework of the two-stage model poses important questions/predictions that have the potential to comprehensively connect different findings that have been investigated separately and to clarify relationships between them. For example, it would be beneficial to reconsider the roles of response feedback and reward from the multistage perspective. Response feedback provides subjects with the information as to whether their response to a stimulus was correct.^122–124 Some types of TR-VPL require response feedback¹²⁵ but others do not.^122,126 On the other hand, reward can be given to subjects irrespective of whether the subjects engaged in a task on a trained feature.⁸² We propose that task-based plasticity follows both supervised and reinforcement learning rules,^40,73,76 while feature-based plasticity, in which task-related improvement is not involved, is driven by unsupervised learning modulated by reinforcement signals.^{63,82,83,85,106,126,127} Thus, it is likely that both response feedback and reward facilitate task-based plasticity, whereas only reward influences feature-based plasticity.

Another example is the relationship between TI-VPL and perceptual-deterioration effects on a visual feature that occur as a result of intensive training or overexposure to the feature.^{114–116,118,128,129} It has been suggested that the deterioration effect is the result of saturation of synaptic strength and the resultant breakdown of visual information processing in early and local neuronal networks when too many task trials/stimuli are given.^6,118 Importantly, it has been reported that TI-VPL often occurs when an exposed feature is not suppressed by attention.^108,130 In addition, a fast version of TI-VPL has been found to occur with as little as a single trial of exposure to a feature.¹³¹ These characteristics of TI-VPL and the perceptual-deterioration effect suggest that feature-based plasticity is sensitive to the strength and frequency of visual signals that are exposed during training.

In this review, we focused on VPL of primitive features, such as motion direction, orientation, and background texture. The learning of more complex features that may involve affective or semantic processing, categorization learning, and other types of learning and memory have emergent distinguishing characteristics of their own, but may also have fundamentally common aspects of mechanisms shared by that of the learning of primitive features. Thus, it is possible that the unified two-stage model leads to a better understanding of the mechanisms unique to the learning of primitive features and also of a general mechanism of learning and memory. Indeed, previous studies on VPL of primitive visual features have shown them to be highly related to the learning of changes of objects,^132,133 categorization learning,¹³⁴ word learning,^135,136 speech learning,¹³⁷ learning in general,¹³⁸ sleep,^115,139 memory,¹⁴⁰ psychiatric diseases,¹⁴¹ and attention.¹⁴²

A fuller understanding of VPL also has implications for clinical applications. The research history of VPL has proved that an understanding of the mechanisms of VPL is vital to the development and application of clinical interventions for patients with weak or degraded vision.^15–23 Thus, the proposed two-stage model, if experimentally verified, would not only resolve the major controversial issues in the current VPL field, but would also clarify the comprehensive mechanisms of VPL, which would help us to develop more practical and powerful clinical applications.