Perceptual learning reflects external noise filtering and internal noise reduction through channel reweighting

Abstract

To investigate the nature of plasticity in the adult visual system, perceptual learning was measured in a peripheral orientation discrimination task with systematically varying amounts of external (environmental) noise. The signal contrasts required to achieve threshold were reduced by a factor or two or more after training at all levels of external noise. The strong quantitative regularities revealed by this novel paradigm ruled out changes in multiplicative internal noise, changes in transducer nonlinearites, and simple attentional tradeoffs. Instead, the regularities specify the mechanisms of perceptual learning at the behavioral level as a combination of external noise exclusion and stimulus enhancement via additive internal noise reduction. The findings also constrain the neural architecture of perceptual learning. Plasticity in the weights between basic visual channels and decision is sufficient to account for perceptual learning without requiring the retuning of visual mechanisms.

Perceptual learning specific to retinal position (1, 2), orientation (3–8), or scale (9–13) has been claimed to reflect plasticity in the adult early visual system (1, 2, 11, 12). But what is the nature of this plasticity? The answer to this question stands at the interface between visual neuroscience and human performance. Given the existence of multiple orientation and spatial frequency channels in the visual system (14), one important question is whether perceptual learning “fine tunes” individual channels or reflects plasticity in the weighting of channel activity. To approach this question, visual perceptual learning in the presence of systematically varying amounts of environmental (random external) noise was measured in adult humans. Perceptual learning combined with the external noise manipulation (15–18) allows the identification of the underlying mechanisms of learning at the overall system level as external noise exclusion, additive internal noise reduction (effecting stimulus enhancement), and/or multiplicative internal noise reduction. Perceptual learning in a peripheral orientation discrimination task was a result of improvements in both external noise exclusion and stimulus enhancement. In terms of basic visual mechanisms, coupled changes in external noise exclusion and stimulus enhancement may reflect plasticity in the weighting or selection of task-relevant spatial frequency channels.

MATERIALS AND METHODS

Observers discriminated the orientation of peripheral Gabor patches embedded in visual noise while performing a central task (Fig. 1a). A fixation point in the center of the display was followed by the rapid display of a central task stimulus at fixation and a perceptual task stimulus peripherally in the lower right quadrant (Fig. 1a). In the central task, designed to demand fixation of the eye, the observer decided whether a string of small characters (1/33 ms) contained either an S or a 5. Perceptual learning of orientation discrimination was measured in the lower right quadrant of the visual display. The signal was a Gabor stimulus (windowed sine wave with 2.3 cycle/degree center frequency) tilted either 12 degrees top to the right or left. Accuracy feedback for both tasks was presented on every trial. The background luminance was set to 71 cd/m²; maximum achievable luminance for the display was 144 cd/m². On each trial, random external pixel noise was chosen from a Gaussian distribution, with one of eight levels of external noise contrast defined by standard deviations ranging from 0 (noiseless) to 33% of the maximum achievable contrast. Examples of the eight levels of external noise, combined with the signal Gabor are illustrated in Fig. 1d. Contrasts required to achieve two different thresholds (18) were estimated with staircase methods (19) (Fig. 1 b and c). A staircase manipulates signal contrast to track a specific target proportion correct: A 3/1 staircase requires three successive correct responses to reduce contrast and one error to increase contrast, and tracks 79.3%, corresponding to 1.634 d′; a 2/1 staircase tracks 70.7%, corresponding to a 1.089 d′. Each session included 100 trials per 3/1 staircase and 80 trials per 2/1 staircase (≈20 reversals per staircase). Ten sessions of 1,440 trials each were performed on different days by each of four observers, for a total of 14,400 trials per observer, or 57,600 trials in the experiment.

Figure 1.

Procedure and results of the perceptual learning experiment. (a) Layout of the displays. Threshold estimation by a 3/1 (b) and 2/1 staircase (c) (19). (d) Contrast thresholds for orientation discrimination were estimated at each of eight levels of environmental (external) noise. Contrasts at threshold are shown averaged over observers and pairs of days. Contrast thresholds for the more stringent criterion (e) and the less stringent criterion (f). Smooth curves are fits of the Perceptual Template Model (PTM) (Fig. 2). Perceptual learning at the two criterion levels identifies the improvements as a hybrid of external noise exclusion and signal enhancement.

RESULTS

Contrast thresholds (visual signal contrast of the Gabor required to achieve a specified accuracy) at the two different criterion performance levels are shown in Fig. 1 e and f. Fig. 1e reports the thresholds for the 79.3% accuracy criterion whereas Fig. 1f reports the thresholds for the 70.3% accuracy criterion averaged over 2-day sets and observers. Reaching the higher criterion requires higher signal contrasts. Thus, corresponding curves in Fig. 1e are above those in Fig. 1f. Within each curve of each panel, higher levels of signal contrast are necessary to compensate for the damaging effect of higher levels of external noise. The dependence of threshold on the contrast of external noise has a standard shape (20, 21). At high levels of external noise (the high noise limb of the function), signal contrast increases directly with the external noise level. At low levels of external noise (the low noise limb of the function), signal contrast is nearly independent of external noise.

Conventionally, the high noise limb reflects performance limited by external noise, whereas the low noise limb is associated with limitations due to inefficiencies in the observer’s visual system, expressed as equivalent internal noise (20, 21). Orientation discrimination performance at both criterion accuracies improved substantially with practice at all levels of external noise, as seen in the lowered thresholds over days (Fig. 1 e and f). The (geometric) mean threshold ratio of initial performance (days 1–2) and final performance (days 9–10) was ≈2.8. As is standard in perceptual learning (1–7), the fastest changes are observed early in training. This is a characteristic of many learning models (22). These data exhibited two extraordinarily strong quantitative regularities. First, at a particular criterion accuracy, the ratio of any two practice curves was approximately constant over external noise, corresponding to the observation that the shape of the threshold versus external noise functions was similar over levels of practice in the log contrast threshold scale: each practice curve is a vertically shifted copy of approximately the same shape. The second observed regularity is in the ratio of the contrast thresholds at the higher (Fig. 1e) to the lower (Fig. 1f) threshold criterion at any practice level, which was approximately constant over widely varying levels of external noise: the two sets of curves are shifted vertically relative to one another on the log contrast threshold scale. These relations did not differ significantly with noise level or over days (P > 0.10), a finding that will allow us to eliminate certain mechanisms of perceptual learning (18). The observation that perceptual learning over days improves performance in both high noise, where external noise is the limiting factor and low noise, where internal processing inefficiencies are the limiting factor, implies a mixture of mechanisms or loci of improvement. Improvements in the high noise limb are naturally associated with external noise exclusion and those in the low noise limb with stimulus enhancement through internal noise reduction. However, a model is necessary to fully test and quantify this interpretation and to rule out alternative interpretations. Luckily, the observation that the performance functions for the two criterion levels maintain a ratio relationship to one another independent of day of practice and external noise level provides strong constraints on an explicit model. We evaluate the perceptual learning data by using a model (Fig. 2) that describes the observer as an input–output system. The observer model (15–18 and see Fig. 2a) characterizes the perceptual system as (i) a perceptual template or filter tuned to the signal-valued stimulus (23); (ii) an optional nonlinear transducer function; (iii) independent multiplicative noise; (iv) independent additive noise; and (v) a statistical decision rule. The perceptual template or filter describes the observer’s overall filtering characteristics in a particular task and reflects a weighted combination of inputs from basic visual mechanisms. Multiplicative and additive internal noise characterize processing inefficiencies in visual system (24). Limitations in performance reflecting internal inefficiencies can be quantified in terms of the amount of internal noise needed to produce an equivalent performance limitation. Additive internal noise reflects the existence of an absolute threshold in the absence of external noise. The nonlinear transducer function reflects nonlinearities early in visual system (25). Multiplicative internal noise describes the consequences of contrast gain control systems (26). The observer model predicts the form of threshold versus contrast of external noise functions based on a fundamental analysis of signal to noise properties:

The d′ is a measure of the threshold accuracy specified by the experimenter; N_ext² is the contrast power of the external noise; β is the gain on a signal stimulus; c is the signal contrast; N_a² and N_m² are additive and multiplicative equivalent internal noise that quantify processing inefficiencies; and γ describes visual system nonlinearity. Solving for the contrast thresholds c (in the log form) yields the model curves (Fig. 2b-d). (See Appendix A for a brief derivation).

Figure 2.

Signature patterns of change in performance during perceptual learning associated with signal enhancement, external noise exclusion, and internal multiplicative noise reduction. (a) The Perceptual Template Model (15–18). (b) Practice enhances the stimulus as processed by the perceptual template, leading to improvement in performance only in the region where performance is limited by internal noise (A_a reduction). (c) Practice improves external noise exclusion by changing the perceptual template, resulting in improved thresholds at high levels of external noise (A_f reduction). (e) Practice reduces internal multiplicative noise, yielding improvements in performance over the range of external noise (A_m reduction).

Three distinct mechanisms or loci of perceptual learning can be distinguished (see Fig. 2a). One mechanism narrows the overall filter of the observer; the other two mechanisms reduce multiplicative or additive internal noise. These three potential mechanisms or loci of perceptual learning are quantified by attentuating multipliers A (0 < A < 1): A_f for external noise exclusion through filter narrowing and A_m and A_a for multiplicative and additive noise reduction. All of the As reflect reductions relative to a baseline (A = 1). The equation incorporating these multipliers is:

These three mechanisms yield signature patterns of improvement when external noise is manipulated (15–18): stimulus enhancement through additive noise reduction, corresponding to reduction in A_a (Fig. 2b), external noise exclusion, corresponding to a reduction in A_f (Fig. 2c), and internal multiplicative noise reduction, corresponding to a reduction in A_m (Fig. 2d). The smooth curves in Fig. 1e-f are the best fitting model for the data. Parameters were estimated by using matlab (Math Works, Natwick, MA) routines to minimize the least squared error in the log threshold contrasts. The log approximately equates the standard error over large ranges in contrast thresholds, corresponding to weighted least squares, an equivalent to the maximum likelihood solution for continuous data.

These data rule out multiplicative noise reduction and changes in nonlinearity. In the observer model, the ratio of performance for the two threshold levels is purely a function of multiplicative noise and nonlinearity γ (18). The observation that the ratio of the two threshold levels is essentially constant over both external noise level and day of practice rules out changes in both nonlinearity and multiplicative noise. Changes in multiplicative noise over days would have revealed larger changes in threshold for the higher than the lower criterion staircases (27). The value of the ratio for the two threshold levels (in particular, that this ratio departs from the ratio of d′s corresponding to the 79.3% and 70.7% accuracies criteria) (18) requires a nonlinearity γ of ≈1.4 that is constant over training days.

At the overall system level, perceptual learning reflects a combination of improved stimulus enhancement and external noise exclusion, with values of A_a and A_f after training of 0.38 and 0.31. These As reflect reductions relative to baseline, measured here in the first two days of practice. Corresponding results obtained for each observer individually. Filtering of external noise (A_f) improves performance in the high external noise region (Fig. 2c); stimulus enhancement via additive internal noise reduction (A_a) improves performance in the low external noise regions (Fig. 2b).

DISCUSSION

The strong regularities revealed by the data from perceptual learning in external noise, and the strong distinctions, which follow from those regularities, are quite novel. Previous studies (1–13) of visual tasks for adult observers evaluated perceptual learning in the absence of environmental noise, corresponding to the single zero external noise points in our data (1–12). Evaluation of performance over levels of external noise both extends the conditions of perceptual learning and provides key tests of three distinct mechanisms of perceptual learning that any model must address. The external noise approach is quite different from other work, which investigates related questions by an analysis of stimulus and task transfer (1–3, 10, 28, 29). Transfer data support qualitative inferences about generality or specificity of learning. The external noise paradigm and model provide quantitative tests, which support direct identification of the mechanisms of learning. The finding that perceptual learning reflected coupled improvements in threshold contrast at both high and low levels of external noise (Fig. 1e-f) contrasts sharply with previous observations of the effects of attentionally cuing a spatial location in practiced task performance. Tradeoffs in attention between competing tasks at two locations in space have resulted in performance differences restricted to low levels of external noise, corresponding to reduction in additive internal noise alone (15). A different set of attentional manipulations exclusively impact on performance at high levels of external noise and reflect external noise exclusion alone (30). These previously documented attentional effects have revealed single mechanisms (Fig. 2 b and c). The observation of coupled effects at low and high external noise in perceptual learning is neither trivial nor obligatory. Attentional effects on performance also have been of quite modest size in relation to the impact of perceptual learning reported here (15–18, 27, 30). Observed attentional effects in practiced observers yielded improvements of ≈15–25%, compared to the >250% effects of perceptual learning reported here.

What is the significance of the coupled improvements in external noise exclusion and internal additive noise reduction (stimulus enhancement) in perceptual learning? These coupled improvements at the observer level have a natural interpretation at the level of basic visual channel(s) (14). The spatial frequency bandwidth (at one-half height) of the perceptual filter can be calculated from the model, assuming that the perceptual filter for our task has a Gaussian shape centered on the center frequency of the Gabor stimulus. The spatial frequency bandwidth of the perceptual filters at the beginning of training was approximately two octaves (0.9–3.7 cpd) and at the end of training, was approximately one octave (1.6–3.0 cpd), which corresponds closely with previously reported single channel bandwidths (14). The bandwidth early in practice spans several channels and late in practice narrows to a single, task-relevant, channel.

We suggest that perceptual learning primarily serves to select or strengthen the appropriate channel and prune or reduce inputs from irrelevant channels. The connections between the most closely tuned visual channel and a learned categorization structure are maintained or strengthened, while input from other channels is reduced or eliminated (Fig. 3). A similar interpretation has been used to account for the improvements in single stimulus relative to mixed stimulus blocks in uncertainty experiments (14). Reducing the weights on irrelevant channels reduces the contributions of external noise and additive internal noise§. Perceptual learning, then, reflects plasticity in the relative activity of different basic visual channels, which contribute to categorization. At some level of neural representation, this might appear as a reorganization in the weighting of the critical channel in cortical maps (31–34). Retuning of individual channels cannot be generally ruled out as a possible mechanism of learning, but learning in this task required only the reweighting of channel activities.

Figure 3.

Schematic of early visual system channels and the changes with perceptual learning. A Gabor patch in noise is processed by visual channels tuned to different spatial frequencies and orientations (14). Channels tuned to a range of spatial frequencies and their corresponding bandpass stimuli are illustrated. Each channel has additive and multiplicative noise as well as nonlinearities. Early in training, several channels are active and connected to a decision structure. After training, the most closely tuned channel has the most active inputs to a decision module, whereas that of other channels has been reduced.

This reweighting of connections between basic visual channel outputs and a learned categorization structure is compatible with a wide range of standard proposed learning mechanisms (35–37). Although we did not investigate the importance of feedback, trial by trial feedback was in fact available in this task. Hence, any of a large class of neural network models would be consistent with these data (36, 37). The model illustrated in Fig. 3 has a very simple structure; more complex tasks might require a more complex (hidden unit) structure to support learning. The fact that learning approached asymptotic performance after ≈7,000 trials may pose a constraint on the learning model (22).

Optimal channel selection is inherently specific to retinal location, spatial frequency, and orientation (1–8). The observations of specificity of learning to retinal location or stimulus, which have been used in the past to argue for adult plasticity in visual system (1–3, 10), are consistent with reweighting of inputs from retinally specific channels. Further, perceptual learning based on channel selection and learned categorization weights may leave performance on other tasks unaffected, whereas alteration of the tuning of the channels themselves would necessarily affect all tasks relying on those channels. The learning of channel selection would allow simultaneous incompatible perceptual learning (38) whereas channel retuning might not. Another example of perceptual learning that appears to be inconsistent with retuning basic visual mechanisms is the ability of observers to learn and then switch rapidly between different prism adaptation states; this flexibility also requires the coexistence of incompatible learned categorization structures (39).

Appendix

The perceptual template model (15–18) (Fig. 2a) predicts observer performance based on fundamental signal to noise relations. For a signal of physical contrast amplitude c, the signal strength S = βc, where β is the gain through the perceptual filter for a signal valued stimulus. Performance also depends on the limiting noise. Overall noise variance N_total² is the sum of three independent noises, external noise, multiplicative noise, and additive noise: N_total² = N_ext² + N_mult² + N_a². The magnitude of multiplicative noise is a direct function of the total contrast power of the signal and external noise: N_mult² = N_m²(β²c² + N_ext²). With transduction nonlinearity (||.||^γ), the signal term becomes S = (βc)^γ, and the squared noise term becomes N_total² = N_ext^2γ + N_m²(β^2γc^2γ + N_ext^2γ) + N_a². Performance is described by accuracy statistic d′:

Rearranging and solving for contrast at a given threshold criterion d′_τ, c_τ:

Taking the log,

The log form corresponds to the model curves shown in Fig. 2.