Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2010 Mar 12.
Published in final edited form as: Neuron. 2009 Mar 12;61(5):700–707. doi: 10.1016/j.neuron.2009.01.016

Rewards Evoke Learning of Unconsciously Processed Visual Stimuli in Adult Humans

Aaron R Seitz 1,2,*, Dongho Kim 1,*, Takeo Watanabe 1
PMCID: PMC2683263  NIHMSID: NIHMS99493  PMID: 19285467

Summary

The study of human learning is complicated by the myriad of processing elements involved in conducting any behavioural task. In the case of visual perceptual learning there has been significant controversy regarding the task-processes that guide the formation of this learning, however, there is a developing consensus that top-down, task-related factors, are required for such learning to take place. Here we challenge this idea by use of a novel procedure in which human participants, who were deprived of food and water, passively viewed visual stimuli while receiving occasional drops of water as rewards. Visual orientation stimuli, which were temporally paired with the liquid-rewards, were viewed monocularly and rendered imperceptible by continuously flashing contour rich patterns to the other eye. Results show that visual learning can be formed in human adults through stimulus-reward pairing in the absence of a task and without awareness of the stimulus presentation or reward contingencies.

Introduction

A difficulty in understanding the processes of skill learning is that skills are typically learned in the context of behavioural training paradigms that involve a variety of perceptual, decisional and motor processes. In the field of visual perceptual learning, researchers are often concerned with studying aspects of plasticity that are thought to take place within the early visual system, but observations along these lines are typically confounded by the contribution of high-level factors in this learning. A simple reason for this can be found in a typical definition of perceptual learning as “performance improvements on a perceptual task after training”. This “training” almost uniformly involves performing a task that is the same or resembles that used to evaluating the learning. Given this constraint there have been great debates regarding the processes that gate the process of perceptual learning.

A prevailing hypothesis was that for a feature to be learned, focused attention must be directed to that feature. Along these lines, a number of studies have found perceptual learning for task-relevant features (i.e. features that must be attended to successfully perform the task), whereas exposed task-irrelevant features (i.e. features that convey no useful information to that task) showed no or only a very limited amount of sensitivity change (Ahissar and Hochstein, 1993; Schoups et al., 2001; Shiu and Pashler, 1992). These results have led to the conclusion that perceptual learning will not occur to stimuli that are task-irrelevant and unattended.

However, recent research of task irrelevant learning (TIPL) has demonstrated that task-irrelevant stimulus features can be learned when they are presented in temporal conjunction with task-relevant features (Ludwig and Skrandies, 2002; Nishina et al., 2007; Seitz et al., 2005a; Seitz and Watanabe, 2005; Seitz et al., 2005b; Seitz et al., 2005c, 2006; Seitz and Watanabe, 2003; Watanabe et al., 2001; Watanabe et al., 2002). These studies demonstrate that directed attention is not necessary for perceptual learning to occur and have argued that perceptual learning is gated by reinforcement processes (for reviews see Seitz and Watanabe 2005; Seitz and Dinse 2007).

However a commonality of studies of attentional learning and those of TIPL is that recently a seeming consensus has been reached in the conclusion that learning is gated by top-down, task-related factors (Ahissar and Hochstein, 1993; Dosher and Lu, 1998; Dupuis-Roy and Gosselin, 2007; Li et al., 2004; Polley et al., 2006; Seitz et al., 2005a; Seitz and Watanabe, 2005; Shiu and Pashler, 1992). For example, even in the case of TIPL, learning occurs for stimuli that are correlated with task-targets (Seitz and Watanabe, 2003) and then learning does not occur when targets are not fully processed (such as in the attentional blink) (Seitz et al., 2005a). Similarly, studies of auditory learning have concluded that reward-based learning also requires high-level gating (Polley et al., 2006). The main difference between TIPL and attentional learning frameworks is that in the case of attentional learning, high-level gating process select which features are to be learned whereas in TIPL gating selects when learning should occur (Seitz and Watanabe, 2005).

These studies raise the question of whether visual skill learning requires an active goal directed process or whether learning can occur automatically without any task, stimulus awareness, or goal directed behaviour. While this may seem like an extreme question, it is worth considering other forms of learning such as that of classical conditioning in which learning can take place through pairing stimuli with reinforcers and that this type of learning requires no awareness (Morris et al., 1998; Pessiglione et al., 2008). The model of perceptual learning proposed by Seitz and Watanabe suggests that successful performance of a task works as an internal reward and triggers neuromodulatory learning signals such as acetylcholine, norepinephrine, and dopamine, which gate learning and thus restrict sensory plasticity (Seitz and Watanabe, 2005). The learning signals provide a likely mechanisms of learning since they are triggered by important events in behaviour and are released relatively diffusively throughout the brain (Dalley et al., 2001; Schultz, 2000). However, studies show that these same learning signals are released outside the context of a task including at times of unpredicted rewards and in the context of classical conditioning (Dayan and Balleine, 2002; O’Doherty et al., 2006; Pearce and Bouton, 2001; Schultz et al., 1997; Thiel et al., 2002; Yu and Dayan, 2005). Accordingly, one might predict that visual learning should also for stimuli that are temporally paired with rewards even outside the context of a task.

Here we take the task out of perceptual learning by use of a classical conditioning procedure in which human subjects, who were deprived of food and water, passively viewed visual stimuli while receiving occasional drops of water as rewards (Dorris and Glimcher, 2004; Lauwereyns et al., 2002; Leon and Shadlen, 1999). The advantage of this procedure is that it allows us to address the mechanisms that gate visual learning as well to reduce the contribution of decision-stage and response learning effects. In a first experiment, subjects passively viewed visual orientation stimuli, which were temporally paired with the liquid-rewards. In a second experiment, these orientation stimuli were viewed monocularly and rendered imperceptible by continuously flashing contour rich patterns to the other eye (Tsuchiya and Koch, 2005). Results show that visual learning can be formed in human adults through stimulus-reward pairing in the absence of a task and without awareness of the stimulus presentation or reward contingencies. Furthermore, these learning effects were specific to the eye to which the stimuli were presented; a hallmark of early visual processing. These results demonstrate that visual learning in humans can be driven by reward signals and that this learning can take place in an automatic way and can affect early stages of visual processing.

Results

The basic design of this study is typical of procedures used to study visual learning. Human-participants were first tested on their sensitivity to the oriented sinusoidal gratings (the stimulus for which we are evaluating learning), followed by a “training stage” in which a specific angle-of-orientation was “trained”, and finally a second testing session measured possible changes in sensitivity for the “trained” and control (i.e. untrained) orientations. Changes in performance between the test sessions that are specific to the trained orientation are the signature of visual learning (Fahle, 2005). The key innovation in the present study is that the participants were given no task during training. Instead, they were instructed to passively view the computer monitor, maintain gaze on a central fixation spot, and to enjoy the occasional drop of water that was delivered through a tube that was placed in their mouths (Figure 1). The “trained-orientation” refers to the orientation that was paired in a consistent temporal relationship (i.e. preceded and partially overlapped) with the reward (i.e. the drop of water) and the “control-orientation” was a second orientation that was presented equally often as the trained-orientation, but was never temporally paired with reward. To ensure that the drops of water were in fact rewarding, participants were required to restrain from eating or drinking for five hours prior to each experimental session. The use of this procedure allowed us to take the “task” out of perceptual learning and to examine the specific hypothesis that reward-related learning signals are sufficient to cause improvements in visual sensitivity for visual stimuli paired with rewards.

Figure 1.

Figure 1

Open in a new tab

General procedure and results of Experiment 1. a, Participants received occasional drops of water as rewards while passively viewing visual orientation stimuli (20% signal; 2 c/deg; 4 deg diameter) without any task (see Supplementary Figure 1 for task schematics). b, Results for trained-orientation, blue curve indicates psychometric function from the first sensitivity test and the red curve that of the second test; shaded regions indicate standard error. A clear learning effect is evident by the separation of the curves. c, Results for untrained orientation indicate no reliable change in performance.

In the first experiment, participants underwent nine days of this training procedure (see Methods). Given that there was no training task, participants’ performance is evaluated through a comparison of the ability for participants to discriminate the trained and control orientations in the sensitivity tests conducted before and after training. To make discrimination challenging the orientation patterns were degraded with spatial noise and we generated a psychometric function (Figure 1b) that describes participants’ discrimination performance as a function of the signal-to-noise ratio (SN). With this type of graph, learning can be determined by whether the psychometric function changes between the sensitivity tests. Such a learning effect can be seen in Figure 1b, where significant learning was found for the trained-orientation that was paired with reward (F(1,3)=41.1, p<0.001, two-way repeated measures ANOVA). However, no significant learning effect was observed for the control-orientation (Figure 1c, right; F(1,3)=2.3, p=0.23); results are similar for d′ and reaction time (Supplementary Fig. 3).

These results demonstrate that visual learning can occur through reward-pairing in the absence of a task and are consistent with the hypothesis that visual learning occurs automatically based upon stimulus-reward contingencies of which the subjects were not aware. However, the results do not rule out the possibility that learning was due to participants directing more attention to the trained than the control orientations. During the training phase, the orientation stimuli were presented at the SN level of .2 that was subtle, but for which the orientation stimuli can be reliably discriminated. Participants may have determined the orientation of the trained and control orientations and discovered the stimulus-reward contingencies. Given that directed-attention is well recognized as a factor that drives visual learning (Ahissar and Hochstein, 1993; Seitz and Watanabe, 2005; Seitz and Dinse, 2007) we designed a new experiment in which orientation-stimuli were made imperceptible to ensure that participants could not discover the stimulus-reward contingencies.

In this second experiment, we used the technique of continuous flash suppression (CFS) (Tsuchiya and Koch, 2005) to render the orientation stimuli imperceptible throughout the training and the orientation stimuli were only presented in a single eye (trained-eye). CFS is a robust type of binocular rivalry in which a series of bright, contour-rich patterns are continuously flashed to one eye while a less salient image is presented to the other eye (Figure 2). In this circumstance, where the inputs to the two eyes do not match, the eye receiving the stronger input (here the contour rich patterns) dominates perception and suppresses the perception of the image presented to the eye receiving the weaker input. Thus by using CFS we could eliminate participants’ awareness of the stimuli (and thereby the stimulus-reward contingencies) during training while largely preserving the responsiveness of visual areas to the orientation stimuli (Leopold and Logothetis, 1996; Logothetis et al., 1996). In Experiment 2, participants underwent twenty-days of training and a comparison of performance between the first and last sensitivity tests revealed a similar pattern of learning to that found in the first experiment. A significant effect of learning was found for the trained-orientation (Figure 3a; F(1,3)=18.6, p<0.01). However, no learning was observed for control-orientation (Figure 3b; F(1,3)=0.30, p=0.62); see Supplementary Fig. 5 for d′ and RT data. These results confirm that perceptual learning can be driven by reward signals, even in the absence of awareness of the learned stimuli and stimulus-reward contingencies.

Figure 2.

Figure 2

Open in a new tab

Training procedure for Experiment 2. Alternating blocks of 15s duration CFS stimuli to were presented to each eye. The CFS stimuli consisted of a sequence of full-screen textured pattern images that were presented at a rate of 10 Hz. For the trained-eye a sequence of 2Hz noise and orientation patterns were presented while CFS was shown to the other eye. For the untrained-eye, a grey screen was presented while CFS was presented to the trained-eye. After every 5 minutes, participants took a 3 minutes break. These sequences repeated 8 times in each of the 20 training sessions.

Figure 3.

Figure 3

Open in a new tab

Results of Experiment 2. a, Results for trained-orientation, blue curve indicates psychometric function from the first sensitivity test and the red curve that of the final test; shaded regions indicate standard error. A clear learning effect is evident by the separation of the curves. b, Results for untrained orientation indicate no reliable change in performance. c, d, Results for untrained eye showed no reliable change in performance for either the trained (c) or control (d) orientations.

To evaluate whether participants may have been aware of the presentation of the orientation stimuli or of the stimulus-reward contingencies during training, we conducted a test of stimulus awareness in a separate session after the final sensitivity test was complete. In this test, participants viewed the same stimulus sequence that had been used in the training stage and were asked to report the orientation of the stimuli presented to the suppressed eye. The results showed that participants failed to respond to the vast majority of the stimulus presentations, and, that when participants did respond, there were no significant differences between the number of correct (11.9 ± 6.0 %) and incorrect (10.1 ± 5.0 %) responses. This post-test should be a conservative estimate of suppression given that perceptual switches under condition of binocular rivalry occur with increasing frequency with long-term exposure (Suzuki and Grabowecky, 2007). However, we ran another awareness test in a group of 8 naïve subjects who hadn’t undergone training. In this awareness test, we presented rewards in some trials and in other trials presented only noise patterns, subjects were asked to report when they observed orientation stimuli. Results showed no differences of responses patterns between trials containing an Orientation + Reward (4.9 ± 4.5%), Orientation + no-Reward (5.2 ± 4.8%), Reward + no-Orientation (5.1 ± 5.0%), or no-Reward+ no-Orientation (5.1 ± 4.5%). Furthermore, in debriefings, subjects from the training sessions showed no indication of having noticed the orientation stimuli or the stimulus-reward contingency during the training sessions. These results indicate that the CFS procedure was effecting in preventing subjects from being aware of the orientation stimuli during the training phase of the experiment.

An important aspect of this study is that participants were trained with a stimulus that was only presented to one of their eyes (Figure 3c, d). This enabled us to examine whether learning in the trained-eye transferred to benefits in sensitivity to the same stimuli presented to the untrained eye. The specificity of the learning effects can provide important clues into what stages of visual processing underlie the observed learning effects; ocular specific learning has been argued to be evidence of plasticity of early, monocular, stages of visual processing (Fahle et al., 1995; Karni and Sagi, 1991; Lu et al., 2005). Analysis of the sensitivity tests for the untrained-eye showed that there were no significant performance changes for either the trained orientation (F(1,3)=3.4, p=0.16) or the control orientation (F(1,3)=0.34, p=0.60) and no effects in either d′ or reaction time (Supplementary Fig. 6). Given that there was a slight trend of a learning effect for the trained direction in the untrained eye, we calculated an index of interocular transfer (100* [untrained-eye learning]/[trained-eye learning]) to evaluate more directly the degree of ocular specificity of the learning. This index showed that for the four subjects (53.9%, −34.4%, 23.5%, and 28.3%) of the learning effect for the trained-orientation in the trained-eye transferred to the untrained-eye; note that the negative value in subject 2 indicates that performance in the untrained eye was worse in the post-test than in the pre-test. The fact that the learning effect is largely specific to both the stimulus orientation and to the eye of training is indicative of plasticity involving an early, monocular, stage of visual processing (Nishida et al., 1994; Paradiso et al., 1989; von der Heydt et al., 1984), although some learning may be occurring at later processing stages as well (Ahissar and Hochstein 1997).

While these experiments are suggestive that a reward process is involved in this learning, it is possible that the results are due to mechanisms other than reward. To test this we ran two new experiments. In one experiment, we asked subjects to rate the pleasantness (on a scale −2, very unpleasant, to 2, very pleasant) of the water delivered during the experiment in one session after they had been deprived of food and water and in another session in which they had not been deprived (order of sessions counterbalanced across subjects). For the fourteen subjects who participated in this test, we found that subjects rated water to be pleasant (1.4 +/− 0.23) after deprivation but not without deprivation (0.07 +/− 0.16), when asked at the end of an hour-long session that resembled the training task from Experiment 2; this difference in favourability between the deprived and nondeprived conditions was significant (p<0.01, two-tailed paired t-test). In a second control experiment, we had 8 new subjects participate in replication of Experiment 2, however these subjects were not deprived of food and water. Given that we found water not to be rewarding without deprivation, we expected not to find perceptual learning to result from this experiment. As hypothesized we failed to find learning for the orientation that was paired with water in the absence of deprivation (F(1,7)=2.2, p=0.20). While we acknowledge that with a null effect we cannot prove that no learning occurred, it is notable that we failed to find learning even with twice the power (i.e. 8 subjects rather than 4) of Experiments 1 and 2, although we cannot say that no learning occurred in the non-deprived condition, at least the results indicate that a higher reward value (such as water with deprivation) induces stronger learning. These results demonstrate that water was rewarding to subjects after deprivation and that learning does not occur to the same extent in a condition where the water delivery was not rewarding. Together, they confirm our hypothesis that the learning is indeed related to reward.

Another question regarding the reported results is how does reward impact the processing of the grating stimuli during training? Did the water delivery “reward” the presentation of the gratings or did it interrupt the effectiveness of the masks that immediately follow the gratings. To discriminate between these possibilities we ran an additional control experiment in which a group of twenty new subjects were water deprived and then tested on grating orientation discrimination under 2 (interleaved) trial types (Reward and NoReward; in Reward trials a drop of water was given 400ms after the presentation of the grating). One group of ten subjects participated in the Mask trials, where subjects reported the orientation of the gratings when the gratings were preceded and followed by a noise-mask (as was done in the tests of Experiments 1 and 2). The other group of ten subjects participated in the NoMask condition in which a gray screen was presented instead of the noise-mask that normally followed the grating presentation. Performance was highly similar between the four conditions (Mask-NoReward= 80.2% +/− 1.3; NoMask-NoReward = 80.4% +/− 1.5; Mask-Reward = 78.7% +/− 2.7; NoMask-Reward = 77.4% +/− 2.3). An ANOVA showed no main effects of Reward (F(1,9)=0.42, p=0.53) or of Masking (F(1,9)=0.71, p=0.41) nor an interaction between them (F(1,9)=0.90, p=0.36). These results indicate that there was no significant effect for the mask. This may be because the grating stimuli were presented for 500ms before the mask onset, which allowed plenty of time for them to be processed before the onset of the mask.

Discussion

Our results challenge current theories of perceptual learning that presuppose that learning requires “top-down” gating processes such as attention (Ahissar and Hochstein, 1993) or task-related reinforcement signals (Herzog and Fahle, 1997; Petrov et al., 2006; Roelfsema and van Ooyen, 2005; Seitz et al., 2005a; Seitz and Watanabe, 2005). We show that stimulus-reward pairing is sufficient to cause learning even in the absence of awareness of the learned stimuli or stimulus-reward contingencies. The results are consistent a process that gates learning that may originate from subcortical reward systems (Schultz et al., 1997; Yu and Dayan, 2005). We suggest that visual skill learning is generally an automatic (ie unconscious) process and that goal-directed factors such as directed attention serve mostly to bias how learning takes place rather than actually gating the learning process. We acknowledge that this conjecture is contentious and further studies will be required to more fully test this.

While the methods used in this experiment are similar to those used in conditioning, it is important to realize that the aspect of learning that is being described here is different from that discussed in the typical studies of conditioning. In contrast to classical conditioning, learning in perceptual processes does not measure a stimulus-response association but instead measures a change in sensitivity to the stimulus. It is thus important to verify that the observed learning effects are resultant from perpetual learning rather than a learning of decision-strategies or response patterns. While our procedure was designed to minimize the contribution of task-related learning due to the training task (we had no training task), it is still important to account for the possibility that the observed learning effects are resultant from a learned bias. For example, we have previously demonstrated that a component of TIPL can be explained by a learned perceptual bias, which could be considered to be a conditioned visual response to preferentially see the trained stimulus (Seitz et al., 2005b). We argue that this is unlikely explanation of the present results given that our d′ analysis (see Supplementary Figs. 3, 5) shows that after training subjects are more sensitive in discriminating between the two orientations (this rules out a response bias to choose a trained orientation or a simple perceptual bias to be more likely to see the trained orientation). Furthermore, the fact that the increase of sensitivity was accompanied by an increase in accuracy specific for the rewarded orientation is consistent with our suggestion that the learning effect is specific to the trained orientation. Also, the fact learning in Experiment 2 is ocular specific further demonstrates that these results are not simply due to a bias of the trained orientation, which should be equally evident for both eyes. We also note that a bias to report the trained orientation is in line with a previous study of TIPL in which we found both bias and sensitivity changes for the trained stimulus, however, that the bias was best explained as a perceptual effect and was consistent with a conditioned response to “see” the paired stimulus even when none is presented (Seitz et al., 2005). Thus while we cannot rule out some contribution of bias, all together, it is very reasonable to conclude that our results reflect that there is plasticity in the visual processing of the trained orientation.

The fact that we found a high degree of ocular specificity in the masking experiment is not surprising given the design of this experiment. The CFS procedure is specifically designed so that the activity of most binocular neurons should be dominated by the contour rich patterns rather than the orientation patterns. Given this, little correlation is expected between the activity of binocular neurons and the delivery of the liquid reinforcers. On the other hand, cells that are monocular will remain responsive to the perceptually suppressed orientation stimuli and show a more consistent pattern of responses in correlation with reward delivery. While it is intriguing to conclude that the learning effect is taking place in V1, the fact that some degree of ocular specificity remains in higher visual areas (although at lower incidence; Uka et al., 2000) makes this conclusion premature. However, physiological studies conducted with a similar procedure in awake behaving macaques indicates that the learning effect is taking place at V4 or earlier (Franko et al., 2006, 2007). Also, it is hard to predict the degree of ocular specificity that would have been expected without suppression. However, our argument that early stage, ocular specific, neurons are impacted by reinforcement signals remains valid.

While we discuss the observed learning effects as resulting from a reinforcement process, we use this term loosely and we note that reward, and more specifically dopamine, is only one candidate mechanism that may underlie the learning. We have previously discussed that a variety of neuromodulators (such as acetylcholine, norepinephrine, and dopamine, etc) have the general properties that we expect in a learning signal; they are released relatively diffusely throughout the brain in correlation with behaviourally relevant events (Dayan and Balleine, 2002; O’Doherty et al., 2006; Pearce and Bouton, 2001; Schultz et al., 1997; Seitz and Watanabe, 2005; Thiel et al., 2002; Yu and Dayan, 2005). Additionally, while we rule-out the possibility that attention is directed to the learned stimuli during training, we acknowledge that the observed learning effects are consistent with types of attention that are not stimulus directed (Fan et al., 2002; Posner and Petersen, 1990). Within this framework there can be a commonality between the learning signals hypothesized to underlie perceptual learning and those thought to underlie some aspects of attention (Seitz and Watanabe, 2005). Our reward-learning technique is a novel methodology for studying human visual learning and helps to overcome limitations of previous studies that have relied on explanations of learning based upon complicated, and not well understood, psychological processes (Ahissar and Hochstein, 1993; Herzog and Fahle, 1999; Seitz and Watanabe, 2005; Seitz and Dinse, 2007). The benefit of a liquid reward is that it yields a well-characterized physiological response (Mitz, 2005; Schultz, 2006), and allows for a close comparison with physiological studies in primates, which rely upon liquid-rewards. Furthermore, an increasing number of studies are characterizing effects of liquid rewards/punishment on human brain processing (Gottfried et al., 2003; O’Doherty et al., 2006; O’Doherty et al., 2002). The research presented here provides important clarity regarding the mechanisms that guide learning in the adult brain and introduces a methodology by which processing components involved in learning can be more easily dissociated from the myriad of processing elements involved in conducting a behavioural task.

Methods

Participants

A total of 8 participants (aged between 19 and 35) were used in the experiments; 4 participants (2 male and 2 female) in Experiment 1 and 4 participants (2 male and 2 female) in Experiment 2. The experiments were conducted in accordance with the IRB approved by the Committee on Human Research of Boston University and the Declaration of Helsinki.

Experiment 1

Apparatus

The stimuli were presented using Psychophysics Toolbox (Brainard, 1997; Pelli, 1997) for MATLAB® (The MathWorks, Natick, MA) on a Macintosh G5 computer. The stimuli appeared on a 19″ CRT monitor with a resolution of 1024 by 768 pixels and a refresh rate of 85 Hz. The viewing distance was 25 cm. A chin rest was used to maintain the participants’ head position. The participants used a computer keyboard to make responses. Water was delivered using a ValveLink®8.2 system made by Automate Scientific, Inc.

Stimuli

Oriented sinusoidal gratings (112.5° or 22.5°; 2 cycle/deg; 4° diameter; luminance ~50 ± 50 cd/m2) were presented at the centre of the screen and were spatially masked by noise. In the sensitivity tests, the signal-to-noise ratio (SN) varied from trial-to-trial (.05, .07, .09, .11, .13, .15, .17, .2; see Supplementary Fig. 1 for example of stimuli). In the training-sessions a constant SN of .2 was used. The variable SN stimuli were created by randomly choosing the SN proportion of pixels from the orientation image and 1-SN proportion of pixels from a noise image. The screen area outside of the 4° stimulus region was all from the noise image (see Figure 1). The noise was generated from a sinusoidal luminance distribution. In this way, the statistics of the luminance distributions were preserved between the orientation-stimulus and the background and there were no texture elements that could distinguish the orientation patch from the noise field when the SN of the orientation-stimulus was brought to 0.

Sensitivity Tests

The first test stage was conducted at least one day before the beginning of the training stage, and the second test was conducted at least one day after the end of the training stage (see Figure 2a for schematic). Participants gazed upon the small fixation point on the centre of the screen and viewed a stream of noise images that changed at 2Hz. Every 4000-ms the fixation point changed from red to green to indicate the presence of an orientation stimulus to which the participants had 2000 ms to respond with a key-press which of the 112.5° or 22.5° orientations was presented. The two orientations and the eight SN levels were pseudorandomly interleaved with 56 repetitions per condition; yielding a 1 hour-long session with 896 trials.

Training Sessions

Participants were asked to restrain from eating or drinking prior to each of the nine training-sessions. In these sessions they were given no task other than to maintain fixation at a red dot in the centre of the screen. The stimulus on the screen changed at a rate of 2Hz, and, at random intervals, a sinusoidal grating (SN = .2) was presented for 500 ms (see Figure 1 for schematic). If the grating was of the trained-orientation (randomly chosen to be either 112.5° or 22.5° for each participant), ~1 ml of water was delivered to the participants through a tube that was positioned in their mouth; no water was delivered for the control-orientation. The water delivery overlapped the last 100ms of the presentation of the trained-orientation. This timing was chosen so that reward could lead to learning both because it was predicted by the trained-orientation and also so that reward-related processing would be concurrent with stimulus processing of the trained-orientation. The stimulus sequence was random with the exception that the minimal time between orientation presentations was 3000ms. Each training session contained of ~350 presentations of each orientation (exact number varied because stimulus sequence was random in this experiment), lasted 1 hour, and participants consumed between 300–350ml of water.

Experiment 2

Apparatus

The apparatus was similar as in Experiment 1 with the exception that participants viewed stimuli from two monitors through a haploscope. Two 19″ CRT monitors with resolution of 1024 by 768 pixels and a refresh rate of 120 Hz were used. The viewing distance was 82 cm.

Stimuli

The orientation stimuli were similar to those of Experiment 1, however some changes were made to optimize the stimuli for use with CFS. Sinusoidal gratings and noise images were presented at only 10% contrast (luminance 44 ± 7 cd/m2) and the stimulus field subtended a total of 8° diameter with the rest of the screen a uniform grey; to prevent hard edges at the stimulus boundary, the outer 4–8° annulus, consisting of pure noise, was blurred with a Gaussian profile (σ=0.5°). In tests the signal-to-noise ratio (SN) varied from trial-to-trial (.03, .05, .07, .1, .13, .16, .2). In the training-sessions a constant SN of .2 was used.

Sensitivity Tests

These sessions were the same as in Experiment 1, with the following exceptions. A total of 3 sensitivity tests were conducted, the first test stage before the training-stage, the second after 10-days of training and the third after twenty days of training. For each 4000-ms trial the orientation and noise stimulus sequence was presented to one eye and a grey screen was presented to the other eye. The two orientations, the eight SN levels, and the two eyes-of-presentation were pseudorandomly interleaved with 28 repetitions per condition; yielding a 1 hour-long session with 896 trials.

Training Sessions

The training session was similar to Experiment 1, but was adapted to promote CFS conditions that would ensure that the orientation-stimuli were suppressed during the entire duration of the 20-day training period. To accomplish this, we presented alternating blocks of 15s duration CFS stimuli to each eye. The CFS stimuli consisted of a sequence of full-screen textured pattern images that were presented at a rate of 10 Hz (see Figure 3a). The texture pattern consisted of 300 randomly placed, physically overlapping, rectangles that varied in size with dimensions between 0.5° and 5°, had random angles of orientation, and consisted of saturated colours (0 or 100 cd/m2). In addition to the textured-patterns, spatially-sparse, coloured, pixel-noise covered a total area of 50% of the screen; we found that the addition of this noise was necessary to reliably suppress the high-frequency content of the training-stimuli. For the trained-eye a sequence of 2Hz noise and orientation patterns were presented while CFS was shown to the other eye. For the untrained-eye, a grey screen was presented while CFS was presented to the trained-eye. After every 5 minutes, participants took a 3 minute break. These sequences repeated 8 times per day. In each day of training, 160 trials of each of the trained and untrained orientations were presented and participants drank ~150 ml of water.

Awareness Tests

The awareness test was conducted on a day following the final sensitivity test. In these sessions participants viewed the identical stimulus sequence as was shown during the training phase, and they were asked to report if they saw orientation stimuli similar to those that presented during the sensitivity tests. After conducting this session, participants were asked to fill out a survey querying them on whether they noticed orientation stimuli or reward-contingencies during the training sessions, and if so, which orientation was paired with reward.

Acknowledgments

We would like to thank Rufin Vogels and John O’Doherty for helpful discussions regarding this work as well as Ladan Shams and Pablo Bartfield for comments on the manuscript and Jonathan Dobres for his artistic contributions. This study was funded by grants from NIH (R01 EY015980 and R21 EY017737), NSF (BCS-0345746, BCS-0549036, and BCS-PR04-137 Center of Excellence for Learning in Education, Science, and Technology), and the Human Frontier Science Program Organization (RGP18/2004).

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Bibliography

  1. Ahissar M, Hochstein S. Attentional control of early perceptual learning. Proc Natl Acad Sci U S A. 1993;90:5718–5722. doi: 10.1073/pnas.90.12.5718. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Brainard DH. The Psychophysics Toolbox. Spat Vis. 1997;10:433–436. [PubMed] [Google Scholar]
  3. Dalley JW, McGaughy J, O’Connell MT, Cardinal RN, Levita L, Robbins TW. Distinct changes in cortical acetylcholine and noradrenaline efflux during contingent and noncontingent performance of a visual attentional task. J Neurosci. 2001;21:4908–4914. doi: 10.1523/JNEUROSCI.21-13-04908.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Dayan P, Balleine BW. Reward, motivation, and reinforcement learning. Neuron. 2002;36:285–298. doi: 10.1016/s0896-6273(02)00963-7. [DOI] [PubMed] [Google Scholar]
  5. Dorris MC, Glimcher PW. Activity in posterior parietal cortex is correlated with the relative subjective desirability of action. Neuron. 2004;44:365–378. doi: 10.1016/j.neuron.2004.09.009. [DOI] [PubMed] [Google Scholar]
  6. Dosher BA, Lu ZL. Perceptual learning reflects external noise filtering and internal noise reduction through channel reweighting. Proc Natl Acad Sci U S A. 1998;95:13988–13993. doi: 10.1073/pnas.95.23.13988. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Dupuis-Roy N, Gosselin F. Perceptual learning without signal. Vision Res. 2007;47:349–356. doi: 10.1016/j.visres.2006.10.016. [DOI] [PubMed] [Google Scholar]
  8. Fahle M. Perceptual learning: specificity versus generalization. Curr Opin Neurobiol. 2005;15:154–160. doi: 10.1016/j.conb.2005.03.010. [DOI] [PubMed] [Google Scholar]
  9. Fahle M, Edelman S, Poggio T. Fast perceptual learning in hyperacuity. Vision Res. 1995;35:3003–3013. doi: 10.1016/0042-6989(95)00044-z. [DOI] [PubMed] [Google Scholar]
  10. Fan J, McCandliss BD, Sommer T, Raz A, Posner MI. Testing the efficiency and independence of attentional networks. J Cogn Neurosci. 2002;14:340–347. doi: 10.1162/089892902317361886. [DOI] [PubMed] [Google Scholar]
  11. Franko E, Seitz A, Volgels R. Effect of stimulus-reinforcement pairing on the local field potentials in macaque visual cortex. Society for Neuroscience. 2006 Program #640.646/N615. [Google Scholar]
  12. Franko E, Seitz A, Volgels R. Effect of stimulus-reinforcement pairing on the local field potentials for suprathreshold, ipsilateral stimuli in macaque visual cortex. Society for Neuroscience. 2007 Program #280.214/FF227. [Google Scholar]
  13. Gottfried JA, O’Doherty J, Dolan RJ. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science. 2003;301:1104–1107. doi: 10.1126/science.1087919. [DOI] [PubMed] [Google Scholar]
  14. Herzog MH, Fahle M. The role of feedback in learning a vernier discrimination task. Vision Res. 1997;37:2133–2141. doi: 10.1016/s0042-6989(97)00043-6. [DOI] [PubMed] [Google Scholar]
  15. Herzog MH, Fahle M. Effects of biased feedback on learning and deciding in a vernier discrimination task. Vision Res. 1999;39:4232–4243. doi: 10.1016/s0042-6989(99)00138-8. [DOI] [PubMed] [Google Scholar]
  16. Karni A, Sagi D. Where practice makes perfect in texture discrimination: evidence for primary visual cortex plasticity. Proc Natl Acad Sci U S A. 1991;88:4966–4970. doi: 10.1073/pnas.88.11.4966. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Lauwereyns J, Takikawa Y, Kawagoe R, Kobayashi S, Koizumi M, Coe B, Sakagami M, Hikosaka O. Feature-based anticipation of cues that predict reward in monkey caudate nucleus. Neuron. 2002;33:463–473. doi: 10.1016/s0896-6273(02)00571-8. [DOI] [PubMed] [Google Scholar]
  18. Leon MI, Shadlen MN. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron. 1999;24:415–425. doi: 10.1016/s0896-6273(00)80854-5. [DOI] [PubMed] [Google Scholar]
  19. Leopold DA, Logothetis NK. Activity changes in early visual cortex reflect monkeys’ percepts during binocular rivalry. Nature. 1996;379:549–553. doi: 10.1038/379549a0. [DOI] [PubMed] [Google Scholar]
  20. Li W, Piech V, Gilbert CD. Perceptual learning and top-down influences in primary visual cortex. Nat Neurosci. 2004;7:651–657. doi: 10.1038/nn1255. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Logothetis NK, Leopold DA, Sheinberg DL. What is rivalling during binocular rivalry? Nature. 1996;380:621–624. doi: 10.1038/380621a0. [DOI] [PubMed] [Google Scholar]
  22. Lu ZL, Chu W, Dosher BA, Lee S. Independent perceptual learning in monocular and binocular motion systems. Proc Natl Acad Sci U S A. 2005;102:5624–5629. doi: 10.1073/pnas.0501387102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Ludwig I, Skrandies W. Human perceptual learning in the peripheral visual field: sensory thresholds and neurophysiological correlates. Biological psychology. 2002;59:187–206. doi: 10.1016/s0301-0511(02)00009-1. [DOI] [PubMed] [Google Scholar]
  24. Mitz AR. A liquid-delivery device that provides precise reward control for neurophysiological and behavioral experiments. Journal of neuroscience methods. 2005;148:19–25. doi: 10.1016/j.jneumeth.2005.07.012. [DOI] [PubMed] [Google Scholar]
  25. Morris JS, Ohman A, Dolan RJ. Conscious and unconscious emotional learning in the human amygdala. Nature. 1998;393:467–470. doi: 10.1038/30976. [DOI] [PubMed] [Google Scholar]
  26. Nishida S, Ashida H, Sato T. Complete interocular transfer of motion aftereffect with flickering test. Vision Res. 1994;34:2707–2716. doi: 10.1016/0042-6989(94)90227-5. [DOI] [PubMed] [Google Scholar]
  27. Nishina S, Seitz A, Kawato M, Watanabe T. Effect of spatial distance to the task stimulus on task-irrelevant perceptual learning of static Gabors. Journal of Vision. 2007 doi: 10.1167/7.13.2. in press. [DOI] [PubMed] [Google Scholar]
  28. O’Doherty JP, Buchanan TW, Seymour B, Dolan RJ. Predictive neural coding of reward preference involves dissociable responses in human ventral midbrain and ventral striatum. Neuron. 2006;49:157–166. doi: 10.1016/j.neuron.2005.11.014. [DOI] [PubMed] [Google Scholar]
  29. O’Doherty JP, Deichmann R, Critchley HD, Dolan RJ. Neural responses during anticipation of a primary taste reward. Neuron. 2002;33:815–826. doi: 10.1016/s0896-6273(02)00603-7. [DOI] [PubMed] [Google Scholar]
  30. Paradiso MA, Shimojo S, Nakayama K. Subjective contours, tilt aftereffects, and visual cortical organization. Vision Res. 1989;29:1205–1213. doi: 10.1016/0042-6989(89)90066-7. [DOI] [PubMed] [Google Scholar]
  31. Pearce JM, Bouton ME. Theories of associative learning in animals. Annu Rev Psychol. 2001;52:111–139. doi: 10.1146/annurev.psych.52.1.111. [DOI] [PubMed] [Google Scholar]
  32. Pelli DG. The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat Vis. 1997;10:437–442. [PubMed] [Google Scholar]
  33. Pessiglione M, Petrovic P, Daunizeau J, Palminteri S, Dolan RJ, Frith CD. Subliminal instrumental conditioning demonstrated in the human brain. Neuron. 2008;59:561–567. doi: 10.1016/j.neuron.2008.07.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Petrov A, Dosher A, Lu Z. Perceptual learning without feedback in non-stationary contexts: data and model. Vision Res. 2006 doi: 10.1016/j.visres.2006.03.022. in press. [DOI] [PubMed] [Google Scholar]
  35. Polley DB, Steinberg EE, Merzenich MM. Perceptual learning directs auditory cortical map reorganization through top-down influences. J Neurosci. 2006;26:4970–4982. doi: 10.1523/JNEUROSCI.3771-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Posner MI, Petersen SE. The attention system of the human brain. Annu Rev Neurosci. 1990;13:25–42. doi: 10.1146/annurev.ne.13.030190.000325. [DOI] [PubMed] [Google Scholar]
  37. Roelfsema PR, van Ooyen A. Attention-gated reinforcement learning of internal representations for classification. Neural Comput. 2005;17:2176–2214. doi: 10.1162/0899766054615699. [DOI] [PubMed] [Google Scholar]
  38. Schoups A, Vogels R, Qian N, Orban G. Practising orientation identification improves orientation coding in V1 neurons. Nature. 2001;412:549–53. doi: 10.1038/35087601. [DOI] [PubMed] [Google Scholar]
  39. Schultz W. Multiple reward signals in the brain. Nat Rev Neurosci. 2000;1:199–207. doi: 10.1038/35044563. [DOI] [PubMed] [Google Scholar]
  40. Schultz W. Behavioral theories and the neurophysiology of reward. Annu Rev Psychol. 2006;57:87–115. doi: 10.1146/annurev.psych.56.091103.070229. [DOI] [PubMed] [Google Scholar]
  41. Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. doi: 10.1126/science.275.5306.1593. [DOI] [PubMed] [Google Scholar]
  42. Seitz A, Lefebvre C, Watanabe T, Jolicoeur P. Requirement for high-level processing in subliminal learning. Curr Biol. 2005a;15:R753–755. doi: 10.1016/j.cub.2005.09.009. [DOI] [PubMed] [Google Scholar]
  43. Seitz A, Watanabe T. A unified model for perceptual learning. Trends Cogn Sci. 2005;9:329–334. doi: 10.1016/j.tics.2005.05.010. [DOI] [PubMed] [Google Scholar]
  44. Seitz AR, Dinse HR. A common framework for perceptual learning. Curr Opin Neurobiol. 2007 doi: 10.1016/j.conb.2007.02.004. [DOI] [PubMed] [Google Scholar]
  45. Seitz AR, Nanez JE, Holloway SR, Koyama S, Watanabe T. Seeing what is not there shows the costs of perceptual learning. Proc Natl Acad Sci U S A. 2005b;102:9080–9085. doi: 10.1073/pnas.0501026102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Seitz AR, Nanez JE, Holloway SR, Watanabe T. Visual experience can substantially alter critical flicker fusion thresholds. Hum Psychopharmacol. 2005c;20:55–60. doi: 10.1002/hup.661. [DOI] [PubMed] [Google Scholar]
  47. Seitz AR, Nanez JE, Holloway SR, Watanabe T. Perceptual learning of motion leads to faster flicker perception. PLoS ONE. 2006;1:e28. doi: 10.1371/journal.pone.0000028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Seitz AR, Watanabe T. Psychophysics: Is subliminal learning really passive? Nature. 2003;422:36. doi: 10.1038/422036a. [DOI] [PubMed] [Google Scholar]
  49. Shiu LP, Pashler H. Improvement in line orientation discrimination is retinally local but dependent on cognitive set. Percept Psychophys. 1992;52:582–588. doi: 10.3758/bf03206720. [DOI] [PubMed] [Google Scholar]
  50. Suzuki S, Grabowecky M. Long-term speeding in perceptual switches mediated by attention-dependent plasticity in cortical visual processing. Neuron. 2007;56:741–753. doi: 10.1016/j.neuron.2007.09.028. [DOI] [PubMed] [Google Scholar]
  51. Thiel CM, Friston KJ, Dolan RJ. Cholinergic modulation of experience-dependent plasticity in human auditory cortex. Neuron. 2002;35:567–574. doi: 10.1016/s0896-6273(02)00801-2. [DOI] [PubMed] [Google Scholar]
  52. Tsuchiya N, Koch C. Continuous flash suppression reduces negative afterimages. Nat Neurosci. 2005;8:1096–1101. doi: 10.1038/nn1500. [DOI] [PubMed] [Google Scholar]
  53. von der Heydt R, Peterhans E, Baumgartner G. Illusory contours and cortical neuron responses. Science. 1984;224:1260–1262. doi: 10.1126/science.6539501. [DOI] [PubMed] [Google Scholar]
  54. Watanabe T, Nanez JE, Sasaki Y. Perceptual learning without perception. Nature. 2001;413:844–848. doi: 10.1038/35101601. [DOI] [PubMed] [Google Scholar]
  55. Watanabe T, Nanez JE, Sr, Koyama S, Mukai I, Liederman J, Sasaki Y. Greater plasticity in lower-level than higher-level visual motion processing in a passive perceptual learning task. Nat Neurosci. 2002;5:1003–1009. doi: 10.1038/nn915. [DOI] [PubMed] [Google Scholar]
  56. Yu AJ, Dayan P. Uncertainty, neuromodulation, and attention. Neuron. 2005;46:681–692. doi: 10.1016/j.neuron.2005.04.026. [DOI] [PubMed] [Google Scholar]

RESOURCES