Learning to Recognize Visual Objects With Microstimulation in Inferior Temporal Cortex

Abstract

The malleability of object representations by experience is essential for adaptive behavior. It has been hypothesized that neurons in inferior temporal cortex (IT) in monkeys are pivotal in visual association learning, evidenced by experiments revealing changes in neural selectivity following visual learning, as well as by lesion studies, wherein functional inactivation of IT impairs learning. A critical question remaining to be answered is whether IT neuronal activity is sufficient for learning. To address this question directly, we conducted experiments combining visual classification learning with microstimulation in IT. We assessed the effects of IT microstimulation during learning in cases where the stimulation was exclusively informative, conditionally informative, and informative but not necessary for the classification task. The results show that localized microstimulation in IT can be used to establish visual classification learning, and the same stimulation applied during learning can predictably bias judgments on subsequent recognition. The effect of induced activity can be explained neither by direct stimulation-motor association nor by simple detection of cortical stimulation. We also found that the learning effects are specific to IT stimulation as they are not observed by microstimulation in an adjacent auditory area. Our results add the evidence that the differential activity in IT during visual association learning is sufficient for establishing new associations. The results suggest that experimentally manipulated activity patterns within IT can be effectively combined with ongoing visually induced activity during the formation of new associations.

INTRODUCTION

The plasticity of object recognition processes induced by learning or experience exploits a fundamental adaptive strategy of biological organisms. It has been hypothesized that neurons in inferior temporal cortex (IT) in monkeys, one of highest visual areas in the ventral visual pathway, are pivotal in various aspects of visual object recognition (Fujita 2002; Logothetis and Sheinberg 1996; Miyashita 1993; Tanaka 1996), including face recognition (Afraz et al. 2006; Sheinberg and Logothetis 1997), visual knowledge (Messinger et al. 2005), and visual association learning and memory (Sakai and Miyashita 1991). IT neurons selectively respond to specific visual object features and their combinations (Desimone et al. 1984; Gross et al. 1972; Sheinberg and Logothetis 2001; Tanaka et al. 1991) and are activated both by feedforward inputs by way of earlier visual areas (Rocha-Miranda et al. 1975) and feedback from medial temporal structures and prefrontal cortex (Naya et al. 2001; Tomita et al. 1999).

More specifically evidence that IT is critical for visual association learning comes both from experiments revealing changes in neural selectivity following learning such as visual-visual association learning (Erickson and Desimone 1999; Freedman et al. 2006; Messinger et al. 2001; Sakai and Miyashita 1991), visual-motor association learning (Baker et al. 2002; Kobatake et al. 1998; Sigala and Logothetis 2002, and visual-reward association learning (Mogami and Tanaka 2006) as well as lesion studies, wherein functional inactivation of IT impairs visual association learning demonstrated by a disruption of previously learned visual discriminations and an increased number of trials to reach criterion when learning new visual discriminations (Brown and Schafer 1888; Buffalo et al. 1998; Chow 1961; Horel 1996; Huxlin et al. 2000; Iwai and Mishkin 1969; Kluver and Bucy 1997; Voytko 1986; Wilson and DeBauche 1981) or for discrimination reversals (Bolster and Crowne 1979; Manning 1972). Whereas many correlational and inactivation studies implicate IT in visual association learning, the sufficiency of specific patterns of IT activity in learning new associations is not well understood. To address this issue directly, we examined the behavioral effects of manipulating activity of IT neurons during learning by microstimulation.

We first asked if controlled stimulation of IT is sufficient for learning new visual classifications by applying electrical stimulation in visually responsive regions of IT during a perceptually ambiguous classification task (Fig. 1, A and B). If the differential activity evoked by electrical stimulation in IT when coupled with otherwise identical evoked visual stimulation can support discrimination learning, we can assert that differential activity in IT can be sufficient for learning new perceptual classifications. We then tested if this learning is a direct association between the electrical stimulation and the learned response by introducing electrical stimulation alone trials. If the monkeys, after establishing learning, elicit same motor response without accompanying the visual stimulus, the learning can be explained as a direct association between the electrical stimulation and learned response. If not, it indicates an interaction between visual evoked activities and electrical stimulation. However, even in the latter case, it is possible the subjects detect electrical stimulation itself using any kind of visual stimulation as a go signal or a temporal cue to detect electrical stimulation. To address this possibility, we tested the visual stimulus specificity of learning by introducing new ambiguous stimulus after learning of ambiguous stimulus. If the learning transfers to a novel ambiguous stimulus, it would support the idea that the visual stimulus generically cues a response to detected electrical stimulation. If not, it indicates more complex interactions between visual stimulation and electrical stimulation.

FIG. 1.

Electrical stimulation in inferior temporal cortex (IT) to disambiguate visual patterns. A: single trial in the basic discrimination task consisted of the presentation of a fixation square followed by a visual stimulus for ≤1 s, the monkey's response (left or right button press), and a feedback signal (tone, plus juice reward on correct trials). B: for each session in this experiment, 3 novel images of painted Ukrainian eggs were selected and mapped to button responses as shown. Two of the 3 images were unique and assigned to the left and right buttons (distinct). The 3rd image was assigned to both the left and right buttons and could only be distinguished by accounting for the presence or absence of the electrical stimulation (ambiguous). During the experiment, the 4 stimuli were randomly interleaved on separated trials. C: learning curves for 3 sessions (top and middle rows are from microstimulation sessions, bottom row is from an auditory stimulation session), comparing performance for trials containing the 2 distinct stimuli (left) and the 2 visually ambiguous stimuli (right). Learning in the distinct condition required ∼10 stimulus repetitions to achieve 80% correct performance. In the ambiguous condition, learning proceeded more slowly. The side assigned for ambiguous plus electrical stimulation is denoted on the top of plots. In 1 experiment (middle row), we introduced a 2nd ambiguous set (indicated by a gray vertical line) after learning of the 1st set was complete. The response mapping for the second ambiguous plus stimulation pair was reversed. In this case, the monkey showed a chance level performance for initial trials and proceeded to learn the new mapping without interference from or interfering with performance on the already learned stimuli. D: comparing the effects of electrical microstimulation with auditory stimulation. Bars show average performance for the last 48 trials in conditions where stimuli could be distinguished based on visual differences (Visual/Visual), on the presence or absence of electrical microstimulation (Visual/IT microstimulation), and on the presence or absence of auditory signals (Visual/Auditory). Each circle represents performance from each experiment. Filled circle, session showed significant learning effects; open circle, no significant learning. Performance in the electrical microstimulation condition was clearly superior to that in the auditory condition, showing that integration of disambiguating information does not always occur. Error bars denote SD.

To further examine the interactive or integrative relationship between visual and electrically evoked activities during learning, we examined the learning where the microstimulation was conditionally informative. In this paradigm, the ambiguous set comprised identical visual pattern and electrical biphasic pulses delivered in visually selective regions of IT (Fig. 3A), but the relative timing between visual and electrical stimulation was different for each class. In one class, the electrical stimulation was delivered just prior to the visual stimulation and the other class during the visual stimulation. In this way, it would not be possible to rely on the detection of electrical stimulation as its sole means of discrimination.

FIG. 3.

Sensitivity to relative timing between electrical and visual stimulation. A: the paradigm was similar to that shown in Fig. 1 except that all visually ambiguous trials included a 200-ms train of electrical stimulation (200 Hz, 200 μs width biphasic pulses). New stimuli and stimulation sites were selected for each experiment session. The relative timing between the onset of the electrical and visual stimulus determined the proper button mapping for each ambiguous pair. B: proportion correct for the final 48 trials is shown for both the visual distinct control stimuli (left 2 columns) and the visual/electrical pairs (middle and right columns). With current levels set to 30 μA (circles), learning was evident in 4/12 experiments (filled symbols). At 60 μA (squares), we observed significantly above chance performance in 10/13 experiments (filled symbols).

Given our focus on learning, we must consider the possibility that any coincident cue could serve the same purpose as IT stimulation. We conducted two control experiments to examine how specific IT microstimulation is for visual learning. We first asked if substituting the electrical microstimulation with suprathreshold auditory stimulus would be equally effective at supporting discrimination learning. We next compared the learning effect for stimulation in IT and an adjacent auditory area. We used an alternate training schedule to ensure the same opportunity for learning for both stimulation sites (Fig. 4A). If the learning of brain stimulation is equally effective in any sensory area, we would expect equivalent training performance irrespective of site.

FIG. 4.

Areal specificity of learning effects. A, left: depth profile of local field potentials (LFPs). Illustration of the estimated coronal brain section superimposed on LFPs evoked by auditory (green line) and visual (red line) stimuli. Right: an alternate day training schedule was applied to microstimulation learning in a learning naïve monkey to compare learning rates for stimulation in visual and auditory responsive areas. B, left: averaged learning curve from 10 IT stimulation experiments. Performance (y axis) for distinct condition (unfilled circles) and ambiguous condition (filled red circles) is plotted against number of stimulus repetitions. Right: average learning curve from ten auditory area stimulation experiments. Performance (y axis) for distinct condition (unfilled circles) and ambiguous condition (filled green circles) is plotted against number of stimulus repetitions. Error bars denote SD. C, left: averaged LFP from 10 IT stimulation sites. Right: averaged LFP from 10 auditory area stimulation sites. Red line shows response for visual stimuli. Green line shows response for auditory stimuli. x axis represents time after onset of visual and auditory stimulus. Y axis represents amplitude of LFP.

Finally, we explored the generality of the effects of microstimulation during learning. In this paradigm, we designed a task where sensitivity to the electrical stimulation would be informative, but not required, for recognition. In a test phase after learning was established, we accessed the effects of microstimulation on performance for learned stimuli and also for related stimuli that had never been seen before (created using a simple image blending technique). If the electrical stimulation was integrated with patterns of visual activity during learning, we would expect a predictable bias in the response to the novel stimuli. Further, if the pattern of microstimulation was integrated with particular stimuli during learning, we would expect the predictable bias to be observed only for those stimuli paired with microstimulation during learning. The results show that artificially manipulated activity of IT by microstimulation does seem to be integrated with visually induced activity during learning and can in turn be sufficient for systematically biasing choices on subsequent recognition tasks.

METHODS

Subjects and apparatus

Experiments were performed on three rhesus monkeys (8–11 kg). During all training and testing, the monkeys' heads were restrained using a single-piece titanium headholder that had been previously implanted using titanium screws, custom-built ball-and-socket recording chamber was implanted on one hemisphere of each animal (centered at AP +17 mm, ML +20 mm) to provide vertical access to the temporal cortex. All surgical procedures and daily care were performed in accordance with the National Research Council Guide for the Care and Use of Laboratory Animals and were approved by the Brown University IACUC.

Procedure

Monkeys were trained to sit in a primate chair and to press one of two buttons (left or right) in response to a visually presented object (Hemera photo objects, Quebec) presented on a 22-in CRT (Iiyama Vision Master Pro 510, 1,024 × 768 pixels, 100-Hz refresh, viewing distance of 119 cm). A trial began with the presentation of a fixation spot (0.2° square). Eye position was monitored by a high-speed infrared camera system (EyeLink II, SR Research, Osgoode, Kanata, Ontario, Canada). Once the monkeys fixated the spot, the spot was turned off and was followed by a visual pattern (after either 200 ms or a random delay in the relative timing task described in the following text). Monkeys were allowed to respond anytime after the onset of the visual pattern. Every correct response was rewarded with a drop of juice. We trained the monkeys extensively for the visual classification task, but we did not explicitly train with either the electrical stimulation or auditory stimulation serving as disambiguating cues before the experiment.

Stimuli were presented by a dual processor x86 PC running a custom-written OpenGL-based stimulation program under Windows XP. Behavioral control for the experiments was maintained by a network of interconnected PCs running the QNX real-time OS (QSSL), which controlled timing, synchronization, and data storage (see Sheinberg and Logothetis 2001 for further details).

Behavioral tasks

DISAMBIGUATION OF IDENTICAL VISUAL PATTERNS THROUGH MICROSTIMULATION.

Three novel images were prepared for each daily session. Two of the three images were differentially assigned to a left and right button response. These two visually distinct images served as a control for the classification involving the third ambiguous image, which was assigned to both the left and right buttons. The button assignments for this ambiguous image were based solely on the presence or absence of electrical stimulation delivered by the microelectrode into IT during the presentation of the visual image. The duration of both visual and electrical stimulation was ≤1 s; if the monkey responded earlier than 1 s, both visual and electrical stimulation were turned off at the time of response. These four trial conditions were pseudorandomly generated within each session. The correct button mapping for the ambiguous trials was randomly assigned from session to session. In one experiment, to test visual stimulus specificity of learning, we introduced new ambiguous stimulus after learning of ambiguous stimulus. For new ambiguous trials, the correct button mapping was switched.

As a control experiment, we investigated whether auditory stimulation would also be effective in disambiguating identical visual patterns. For these experiments, three novel images were prepared for each daily session. The button assignments for the ambiguous image were based solely on the presence or absence of a particular sound. Sounds were delivered by speakers situated to the left and right of the visual display. Onsets of sound and visual stimuli were synchronized. A sound file for a given session was taken from a collection (1,001 sound effects, Sony, Tokyo, Japan) of real-world sounds (e.g., bees warm, balloon blowout, etc.). Sound level was 70–74 dB SPL at the location of the monkey's ear.

RELATIVE TIMING TASK.

To determine whether learning could occur under conditions where the detection of the microstimulation alone was not adequate to support classification learning, we designed a relative timing task. In this conditionally informative condition, three novel images were again prepared for each daily session. Visual patterns were presented for 200 ms. The ambiguous visual stimulus was always paired with a fixed 200-ms train of biphasic electrical stimulation. On half of these trials, the stimulation was delivered starting 200 ms before the visual stimulus appeared, and on the other half, the electrical stimulus started 75 ms after the appearance of the visual pattern. The button assignments for this ambiguous image were solely based on relative timing of electrical stimulation to onset of the same visual pattern. In addition, the absolute timing of the electrical/visual pattern (the delay from the offset of fixation spot to the onset of stimulation) was randomized so that the timing of either the visual or electrical pulse alone did not determine the correct response.

IT SPECIFICITY FOR LEARNING.

To examine the specificity of the learning effects to IT cortex, we used a monkey familiar with the general visual classification task but which had not been trained to disambiguate visual stimuli by microstimulation. We compared the learning effect in IT with that in an adjacent auditory area. Our vertical approach to IT cortex enabled us to obtain the depth profile of local field potentials (LFPs) along the recording track for auditory and visual stimuli (Fig. 4A). After the stimulation site for each session was determined by any cells' spike activity and clear visual or auditory selectivity in LFP, we introduced learning with microstimulation. Based on the electrode trajectory and depth, we estimated that the mean distance between the sites for auditory area stimulation and IT stimulation was 6 mm, and the auditory responsive site was always shallower than the visually responsive site. Based on stereotaxic coordinates, the estimated auditory responsive site was located in the proximity of cytoarchitectonic areas TPO or TAa. The task design was exactly same as described in the preceding text (Disambiguation of identical visual patterns through microstimulation), but the stimulation/recording electrode was positioned in IT cortex on half of the sessions and in an adjacent auditory area on the other half of the sessions. We equalized the number of sessions including stimulation in visual IT and auditory association cortex. The number of repetitions for each condition (distinct and ambiguous) was set as 400. We started auditory stimulation on the first day, followed by IT stimulation the next day. This alternate day training schedule was repeated 10 times (10 attempts for each area). In one auditory area experiment, the monkey stopped working before completing 400 trials (288 trials), so the following IT stimulation was matched in length. In two IT experiments, the monkey stopped working before 400 trials were completed (284 and 366 trials), but we did not truncate the following auditory training sessions (providing as much opportunity as possible for learning to occur in this condition).

GENERALIZATION TO NOVEL STIMULI.

In this task, four novel images were prepared for each daily session (Fig. 5A). These images were initially learned over a period of 400 trials with each appearing 100 times in pseudorandom order. This allowed the animals to always achieve >85% accuracy for these stimuli. During the initial learning phase, one of the four images was consistently paired with electrical stimulation. The assignment of the side for electrical stimulation (plus visual stimulus) was randomly chosen from session to session. After the learning phase, we introduced novel images by blending pairs of stimuli. The blending was implemented by manipulating the transparency (alpha channel) of two images. To determine blend proportions that would result in balanced left and right responses for each stimulus pair, we then ran a test block containing eight trials each of left/right blends mixed using the following proportions: 1.0/0.0, 0.70/.30, 0.55/.45, 0.45/.55, 0.30/.70, and 0.0/1.0. In this block, the correct choice was set to be left for 1.0/0.0, 0.70/.30, and 0.55/.45 and right for 0.45/.55, 0.30/.70, and 0.0/1.0. Responses were tallied on-line, and the sorted data were fit with a logistic function of the form R(x) = 1/(1 + A·e^−Bx), where x denotes the proportion right stimulus in the mixture, R the proportion of right responses, and A and B were best-fit parameters obtained to minimize squared error. From this fit, three intermediate mixtures estimated to produce 30, 50, and 70% right responses were used in addition to the unambiguous stimuli to test the effects of microstimulation. The ambiguous stimuli were rewarded on 75% of the test trials, irrespective of the monkeys' choices.

FIG. 5.

The effects of IT microstimulation during visual classification learning when microstimulation is informative, but not required, for classification. A: in this task, 4 new images of stamps were selected before each test session. Random colored noise was added to each stimulus to introduce initial variability during the discrimination task. Two stimuli were shown upright, and 2 were rotated 45°. Microstimulation was included on all trials containing 1 of the 2 upright stimuli (randomly assigned to either the left or right in each session). Thus during learning, 1 of 4 patterns was always experienced with the microstimulation present. Inset: the recorded analog return signal from the stimulating electrode, which verified the timing and amplitude of the microstimulation. The rotated stamps served as control stimuli in each experiment. B: performance plotted as a function of number of stimulus repetitions shows that for this task, the additional microstimulation (filled circles) did not significantly affect learning rates of the patterns assigned to the stimulation condition compared with performance for unstimulated stimuli (open circles). By 100 stimulus repetitions per stimulus, performance was between 90 and 100% for both trial types.

Recording and stimulation protocols

During each daily session, a tungsten microelectrode (impedance: 250 kΩ to 1.0 MΩ; FHC, Bowdoinham, ME) was lowered into the cortex of the anterior inferior temporal lobe (see Subjects and apparatus) by hydraulic microdrive (Kopf Instruments, Tujunga, CA) to a depth where multiunit visual activity was clearly evident in on-line response histograms aligned to the presentation of visual objects during a viewing-only task. Based on stereotaxic coordinates and the pattern of white/gray matter transitions along each recording track, the estimated recording and stimulation site was located in the proximity of cytoarchitectonic area TE (lateral to ventral area 36 and anterior to the more posterior area TEO). Recorded neural signals were passed through a head stage (Model HZP, Grass Technologies, West Warwick, RI) and then split and filtered (−6 dB filtered between 0.3 and 300 Hz for LFPs and between 100 Hz and 6 kHz for spike activity) by a main amplifier system (Model 15L Grass Technologies). For these experiments, we targeted visually responsive sites but did not search for particular neuronal selectivity. Once a site was selected, the amplifier head stage was switched from record mode to stimulate mode for the duration of the experiment. A chronically placed stainless steel guide tube situated ∼14 mm above IT served as the return path for the stimulation, and a l kΩ resistor was placed in series with the stimulating electrode. We used a battery-powered bioamplifier (DAM50, World Precision Instruments, Sarasota, FL) with output attached to a data acquisition computer to provide a permanent record of the current delivered. Stimulation was delivered by analog stimulus isolator (Model 2200, A-M Systems, Sequim, WA) driven by a programmable arbitrary waveform generator (33220A, Agilent Technologies, Palo Alto, CA).

Microstimulation parameters for the disambiguation of visual patterns experiment were 200-Hz, 200-μs pulse width, 30-μA, biphasic pulses starting 50-ms poststimulus onset for either 1,050 ms or until the monkey responded after which time the monkey pressed the button both the visual and electrical stimulation were removed. In relative timing experiment, we used biphasic pulses of either 30 or 60 μA (in separate sessions). In the experiment for IT specificity for learning, we used biphasic pulses of 75 μA both for IT and auditory stimulation. In the generalization experiments, we used biphasic pulses of 30 μA. The duration of the microstimulation train was shortened in the relative timing experiment and for the generalization experiment to 200 ms, but the pulse width and frequency were kept constant. For all of the generalization experiments, the electrical pulse train lasted from 75 to 275 ms poststimulus onset.

Data analysis

To assess the effect of microstimulation on learning, we evaluated the performance on the final 48 trials for each condition (distinct and ambiguous), the significance of which was tested using a binomial test where the null hypothesis was that left or right responses were equally likely to occur.

To quantify the shift of choice performance by microstimulation, we collapsed the data from all three intermediate mixtures at each daily session. The stimulation effect was defined as the difference between mean choice rates for the stimulus which had been paired with electrical stimulation during learning when presented at intermediate levels with and without electrical stimulation. The significance of the effect was evaluated for each monkey by Wilcoxon signed-rank test. To obtain psychometric functions and to evaluate the effects of stimulation across the collection of experimental sites, we also use a logistic regression analysis, which models the log-odds-ratio of response to the side paired with stimulation as a linear sum of the contributions from several sources (Salzman et al. 1992). We first considered a model that includes interaction term

(1)

where P corresponds to the probability of choosing the side for which the stimulus was paired with stimulation during learning, stim represents a categorical variable which takes the value 0 for nonstimulated trials and 1 for stimulated trials, and mixprop corresponds to the mix proportion for blending images, with 0 corresponding to the stimulus not paired with stimulation during learning and 1 corresponding to stimulus paired with stimulation during learning. β₀, β₁, β₂, and β₃, are the coefficients to be fit by the regression. None of our conditions showed a significant interaction. Thus we used a following simpler model for the analysis

(2)

Significance of the maximum likelihood estimator was tested with the likelihood ratio test. Because stimulation points are asymmetric in terms of response side, we only included intermediate levels for the regression analysis to avoid introducing artifacts.

For reaction time analyses, we measured the time between onset of visual stimulus presentation and the button press. We pooled data from all three intermediate mixtures from all daily test sessions. The significance of the effect was evaluated for each monkey by two-way ANOVA (stimulation and chosen stimulus).

For eye-movement analysis, saccades were extracted from off-line eye records using a velocity-based algorithm written in C, yielding a start and end position for every saccade on each trial. The parameters of this algorithm were set to detect saccades down to ∼0.4° in amplitude. As a quantitative index for eye trajectory separation between the conditions (different targets or stimulated vs. unstimulated), we computed receiver operating characteristic (ROC) curves from the horizontal initial saccade endpoints for each trial and estimated the area under these curves (as described in Sheinberg et al. 2006).

All analysis was done using custom written software embedded in a Tcl-based environment (http://www.tcl.tk/) with software hooks to IMSL library functions (Absoft, Rochester Hills, MI), Matlab (The MathWorks, Natick, Ma) and R (www.r-project.org).

RESULTS

Disambiguation of visual patterns through microstimulation

In our first experiment, we tested the sufficiency of IT microstimulation for visual classification learning by training two rhesus monkeys to perform the visual classification task illustrated in Fig. 1, A and B. On each daily session, we first located a visually responsive site in anterior IT (see methods) and then selected three novel images of Ukrainian painted eggs (Fig. 1B) for use in the visual classification task. Two of the three images were differentially assigned to a left and right button response. These two visually distinct images served as a control for the classification involving the third image, which was assigned to both the left and right buttons. The button assignments for this ambiguous image were based solely on the presence or absence of electrical stimulation delivered by the microelectrode into IT during the presentation of the visual image. Electrical stimulation was present on half of the ambiguous trials. The correct button mapping for the ambiguous trials was randomly assigned from session to session. The results of two single experiments are shown in Fig. 1C (top and middle rows), demonstrating that the added electrical stimulation could be used to distinguish between otherwise identical stimuli.

We repeated this procedure 24 times (12 for monkey S, 12 for monkey J) randomly assigning the stimulated condition to either the left or right side. Performance for the final three blocks of trials (final 48 trials) was analyzed, and in 21/24 experiments, the animals showed significant evidence of learning (P < 0.05, binomial test, Fig. 1D). We found no correlation between the total number of trials (377 ± 133 trials, mean ± SD) and the final performance (r = −0.32; P = 0.13, Pearson's correlation). The average performance for the ambiguous condition in these experiments was 79% (monkey S: 82%, monkey J: 76%), whereas average percent correct performance in the distinct condition was 98% (monkey S: 98%, monkey J: 97%). Over the course of these experiments, we found no evidence for “learning to learn” as rates for acquisition did not improve in later sessions compared with earlier ones (monkey S: r = −0.22; P = 0.49, monkey J: r = 0.03; P = 0.91, Pearson's correlation).

Induced eye movements could confound our interpretation of the ambiguous condition if the microstimulation in IT directly causes a direct systematic eye-movement pattern leading to systematic changes in the retinal projection of the ambiguous images. To address this issue, we analyzed initial saccades elicited 50–500 ms after stimulus onset for both stimulus and unstimulated trials. Figure 2 illustrates initial saccade trajectory and histograms of saccade endpoints in the distinct (no microstimulation) and ambiguous (including microstimulation) conditions. The monkeys were allowed to make saccades during the classification task, and we observed a biased, bimodal distribution of small amplitude saccades in both the distinct (right) and ambiguous (left) conditions. In the distinct condition, the saccades were clearly segregated by the stimulus presented. This separation of first saccade endpoint predicted the monkeys' choice behavior as we previously reported (Sheinberg et al. 2006) in a similar classification task. In the ambiguous condition, the eye separation was not obvious even when we observed clear evidence of learning as shown in the plot under the eye trajectory panel. To quantitatively evaluate the eye separation, we calculated the area under the ROC curve for endpoint positions in the horizontal axis. As shown in Fig. 2B, there were occasional high ROC values; we found no correlation between the eye separation and the behavioral performance in monkey S (r = −0.006; P = 0.95, Pearson's correlation) but a significant correlation in monkey J (r = 0.56; P < 0.001, Pearson's correlation). In monkey J, however, the data from early blocks (filled red circle in Fig. 2B) showed no significant correlation between eye movements and performance (r = 0.37; P = 0.2, Pearson's correlation), whereas data from later blocks consisting of the same number of data points revealed a significant correlation (r = 0.58; P = 0.05, Pearson's correlation). We thus conclude that although eye movements may be biased during a two-choice discrimination task (in the presence and absence of microstimulation), this bias is best explained as a result of response planning associated with learning as opposed to a source of information that could potentially guide learning.

FIG. 2.

Initial saccades in the distinct and ambiguous conditions. A: initial saccade trajectory and histograms of saccade endpoints from individual sessions. Positive value in horizontal and vertical eye displacement indicates displacement to the right and up, respectively. Top row: data from monkey S. For a distinct visual stimulus pair (top left panel), saccades for one stimulus, which was assigned to left button pressing (trajectory and histogram shown in black), and another stimulus, which was assigned to right button pressing (trajectory and histogram shown in green), showed a segregated trajectory. Stimuli used for each condition is inset left above of the trajectory panel. For an ambiguous pair (top right panel), saccades showed similar overall profile but no clear separation between unstimulated condition, which was assigned to left button pressing (trajectory and histogram shown in black), and stimulated condition, which was assigned to right button pressing (trajectory and histogram shown in red). Stimuli used for each condition is inset left above of the trajectory panel. A scatter plot placed under the each eye trajectory panel represents behavioral learning curve from the same session. Another example session from monkey J is shown on bottom row. In this case, stimulated condition (trajectory and histogram shown in red) was assigned to left button pressing. B: eye separation and behavioral performance in the ambiguous condition. Population data from monkey S (12 sessions) and monkey J (12 sessions) are shown in left and right panels. Ordinate (for eye separation) shows area under the curve with receiver operating characteristic (ROC) analysis for endpoint positions in horizontal axis. Abscissa (for behavioral performance) shows mean correct rate at the period that the eye separation is calculated. Each point represents data from 24 trial repetitions. Symbols represent learning phase (periods within each session). Red filled circle indicates initial 24 repetitions in the beginning of each session (early blocks). Pink unfilled square indicates last 24 repetitions of each session (late blocks). Unfilled circle indicates middle phase between initials and lasts (middle blocks).

Because success in the ambiguous discrimination required the animals to differentiate between the visual and the visual plus electrical stimulation conditions, we wondered if the electrical stimulation alone would be adequate for eliciting a manual response. In fact, past studies have reported that electrical stimulation in many cortical areas, including IT, could act as a conditioned stimulus in a simple reward avoidance paradigm (Doty 1965; Doty et al. 1964). More recently, in a two-alternative forced-choice detection task, monkeys were explicitly trained to detect intervals of electrical microstimulation in multiple visual areas including IT (Murphey and Maunsell 2007). Note that in our paradigm, there was never explicit training with microstimulation alone. To test the adequacy of the electrical stimulation in isolation for eliciting behavioral responses, we included a small number of electrical-stimulation-only trials (4/32) during five sessions in which the ambiguous trial performance reached a level of 80% (4 times with monkey S, once with monkey J). On these trials, the microstimulation was delivered with no accompanying visual stimulus while keeping other parameters of the experiment the same. On each of these trials, both monkeys waited the full 5-s timeout period without making a response; something we never observed on trials containing a visual stimulus.

It is important to reiterate that in our task design, microstimulation is not the only antecedent to a particular response. In other words, because the monkeys learned the distinct and ambiguous conditions simultaneously, we know that they must have based their recognition choices on more than just the presence or absence of microstimulation, because none of the distinct trials included stimulation, yet half of these trials were mapped to the same hand as the ambiguous plus stimulation trials. We further investigated the image specificity of the microstimulation learning by introducing a second ambiguous stimulus pair after completion of first learning set in one session (Fig. 1C, middle row). For the new visual stimulus, the ambiguous plus stimulation condition was mapped to the side opposite that learned for the first ambiguous classification. Consistent with the idea that the microstimulation interacts with representations of the presented visual pattern and does not stand alone, we found that the monkey learned this new ambiguous condition without interference from, and without compromising performance on, the already learned pair. Instead, the monkey initially responded at chance, as would be expected for a novel ambiguous pair, and learned the new mapping at a rate very similar to that seen for the initial pair.

To further test the integrative nature of the combined visual-electrical stimulation during learning, we trained one monkey to perform a relative timing task (Fig. 3A ). In this experiment the ambiguous visual stimulus was always paired with a fixed 200-ms train of biphasic pulses. On half of these trials, the stimulation was delivered starting 200 ms before the visual stimulus appeared and on the other half, the electrical stimulus started 75 ms after the appearance of the visual pattern. In addition, the absolute timing of the electrical/visual pattern was randomized so that the timing of either the visual or electrical pulse alone did not determine the correct response. In this paradigm, the monkey could not base his decision solely on the presence of the electrical microstimulation but instead had to rely on information about the relative timing between the visual and electrical signals. As before, three unique stimuli were chosen for each experimental session with the two visually distinct stimuli serving as a control. Using a pulse amplitude of 30 μA, the monkey showed significant evidence of learning (P < 0.05, binomial test) by the last block of 48 trials in the relative timing condition in 4/12 experiments (33%). We then ran a separate series of experiments, doubling the microstimulation current to 60 μA, and found that in 10/13 experiments (77%), the monkey showed significant evidence of learning (Fig. 3B). These experiments show that the timing of the electrical stimulation relative to the associated visual pattern can be used to discriminate between events and that the simple detection of the electrical stimulation is not the only information that can be used to solve the discrimination tasks we have employed.

Electrical microstimulation delivered directly into visually responsive areas of the temporal cortex can thus be used to differentiate otherwise identical visual stimuli. Perhaps any coincident sensory cue could serve the same purpose. Was our stimulation focused in IT in any way special? One way we addressed this question was to ask if substituting the electrical microstimulation with suprathreshold auditory stimulation would be equally effective at supporting discrimination learning. We used the same paradigm as in the preceding text, except that the third ambiguous stimulus was paired on half of the trials with an amplified acoustic waveform (70–74 dB SPL) selected from a collection of real-world sounds. The acoustic cue was synchronized with the visual stimulus. To our surprise, the same monkeys that had learned to distinguish between the ambiguous stimulus pair using temporal cortical microstimulation showed no evidence of learning in this paradigm using the sound as the differentiating stimulus (Fig. 1C, bottom row). We repeated this experiment 14 times (6 for monkey S, 8 for monkey T). Neither monkey showed any evidence of learning in this task. Averaged performance was 49% for monkey S, 49% for monkey T (Fig. 1D). We verified that the monkeys could hear the sounds using a simple orienting task. We played a sound through one the two stereo speakers situated to the left and right of the video display in the absence of any visual stimulus. With no prior training, the monkeys directed their initial saccade following the onset of the sound toward the active speaker on 15/19 trials (79%) for monkey S, on 16/20 trials (80%) for monkey T, a statistically improbable event (P < 0.01, binomial test) if the sound played no role in guiding the redirection of gaze. It is important to note that while we trained the monkey extensively for the visual classification task, we did not explicitly train with either the electrical stimulation or auditory stimulation serving as disambiguating cues before the experiment. The number of repetitions for each daily session in the electrical and auditory stimulation conditions was comparable (361 ± 177 trials for electrical stimulation, 341 ± 190 trials for auditory stimulation, means ± SD) for monkey S. For monkey T, the number of ambiguous trials for each session was set limited to 400 (284, 288, 336 for 3 of 10 electrical stimulation, 400 for the rest of electrical stimulation, and 8/8 auditory stimulation). The clear contrast between two tasks does not seem attributable to our training strategy with the disambiguating cues. Instead these results indicate that in the context of our learning paradigm, electrical microstimulation of IT can be integrated into the visual task with relative ease, whereas informative cues from another sensory modality cannot.

To examine if the learning effects is specific to IT stimulation, we compared the effects of microstimulation in IT and in an adjacent auditory association area within the temporal lobe. For this experiment, we used a monkey that had not previously been trained to disambiguate visual stimuli by microstimulation. To fairly compare the IT and auditory area stimulation, we made the number of attempts equal (400 trials for ambiguous conditions). We started auditory stimulation on the first day followed by IT stimulation the next day (Fig. 4A). On each session, the stimulation site was determined by multiunit spiking activity and clear visual or auditory selectivity in LFP (Fig. 4C). We repeated this alternate day training schedule 10 times for each position. We observed a significant learning effect in most sessions (8 of 10) when the stimulation was delivered in IT (Fig. 4B, left). In contrast, we observed only one significant learning effect (69%) of 10 experiments for auditory stimulation (Fig. 4B, right). The average performance was 68 ± 12% (mean ± SD) for IT stimulation and 52 ± 6% for auditory area stimulation. The difference of two populations was significant (P < 0.001; Wilcoxon rank sum test). Thus the effect of microstimulation in our learning paradigm showed clear specificity in terms of target cortical stimulation area.

Generalization to novel stimuli

We next explored the generality of the effect. In particular, we designed a task where sensitivity to the electrical stimulation would be informative, but not required, for recognition. Also, using a simple image blending technique, we asked if microstimulation would bias responses to specific patterns that had never been seen before. For this experiment, we selected four images of unfamiliar postage stamps before each daily session (Fig. 5A), and we used these in the left/right classification paradigm described in the preceding text. Two monkeys each participated in 12 different microstimulation experiments. One monkey was the subject in the first experiment, whereas the other had never participated in any microstimulation study. Stimulation sites were again selected on the basis of general visual response obtained during a visual fixation task, and the stamp stimuli were only introduced after the site had been chosen.

During learning, the four stamps were each shown 100 times, presented in pseudorandom order. In this learning phase, one of the four images was consistently paired with a 200-ms train of stimulation pulses (Fig. 5A, inset). To assess the effect of the paired microstimulation on initial learning, we compared performance for the paired and unpaired objects over the course of training. For the set of 24 experiments, we found no significant differences between these conditions (Fig. 5B) and average performance in the both stimulated and unstimulated conditions exceeded 90% by the end of training.

Once the initial stimuli were learned, we introduced stimulus mixtures composed of intermediate blends of pairs of the learned stimuli. In this condition, image blends were shown intermixed with the learned objects, but the ambiguous stimuli were rewarded on 75% of the test trials, irrespective of the monkeys' choices. To ensure that the blending procedure would yield reasonable behavioral results, both monkeys had been familiarized with the blending paradigm prior to testing in the microstimulation conditions. Behavioral data from an example session with no microstimulation are shown in Fig. 6A, illustrating the relatively sharp categorical boundaries between the image mixtures.

FIG. 6.

The effect of electrical microstimulation on the classification choices for novel stimuli composed of image mixtures. A: psychophysical performance from a single behavioral experiment shows the systematic effect of varying the stimulus content from 100% left stimulus to 100% right stimulus. After a learning phase consisting of 100 repetitions for each of the individual stamps, stimulus blends were created by mixing the 2 stamps from the same orientation condition (upright or rotated) in various proportions. A brief pretest with fixed blend ratios (see methods) was administered, and the data were fit to a sigmoid function to estimate the midpoint for response selection (which was not always at the 50% blend level). Three levels of mixing around this subjective midpoint were then used for the main test condition. The abscissa for these plots represents the proportion of the right stimulus at each point, and the ordinate shows the proportion of right responses chosen. At the extremes, the animal's choices were close to perfect and choices between the 2 extremes varied smoothly and systematically as a function of blend proportion. B, left: comparison of stimulated (filled circles) and unstimulated (open circles) trials shows a systematic shift in response proportion upward in the direction of the response associated with the stimulus paired with microstimulation during learning. The microstimulation induced shift was measured as the average response difference between the stimulated and unstimulated conditions in the ambiguous region of the response curve (data appeared on gray background). Right: effect of microstimulation on trials from the same experimental session containing mixtures of images neither of which had been been previously been associated with microstimulation. There is transfer of the effect of microstimulation (for this experiment, microstimulation biased choices to favor the right hand response), but the magnitude of this shift was systematically smaller than that observed for the visual mixtures that included the previously associated image (see Fig. 7).

In the critical microstimulation condition, as an example shown in Fig. 6B, we observed consistent effects of stimulation for the ambiguous image mixtures (Fig. 6B, left). For these trials, the addition of the microstimulation pattern that had been paired with one of the two images during learning biased choices in the direction of that object's learned response during testing. We found, however, no significant effects of microstimulation on the recognition of well known visually unambiguous items (data not shown). We also found that the effect of the stimulation for the rotated ambiguous image mixtures, neither of which was ever associated with microstimulation (bottom pair of stimuli, Fig. 5A), was significantly smaller than that observed for the ambiguous mixtures which included as one of the blend images the stimulus that had previously been associated with microstimulation (Fig. 6B, right).

To quantify the stimulation effect across the set of 24 experiments, we compared the monkeys' response proportions to the same visual patterns with and without stimulation for each experiment (open and closed circles within the gray region of Fig. 6B). The logistic regression analysis revealed a significant contribution of the stimulation for both monkeys. The coefficient for stimulation was 0.67 (P < 0.001, likelihood ratio test) for monkey S (Fig. 7A, left top), 0.31 (P < 0.001, likelihood ratio test) for monkey T (Fig. 7A, left bottom). The result from each experiment of this comparison (Fig. 7B) also showed a choice bias induced by the microstimulation in the direction of the stimulus that had been paired with the microstimulation during the learning phase. This bias was statistically significant for both animals (monkey S: P < 0.05, monkey T: P < 0.05, Wilcoxon signed-rank test). We also examined the effect of the microstimulation on the monkeys' reaction times (Fig. 8A). For both animals, we found that microstimulation speeded responses for button choices corresponding to the side of the paired image but slowed responses when the unpaired image was selected (monkey S: stimulation × choice interaction P < 0.001, monkey T: stimulation × choice interaction P < 0.01; 2-way ANOVA), post hoc analysis revealed simple main effects of stimulation for both choices (monkey S: P < 0.01 for paired image choice, P < 0.05 for unpaired image choice, monkey T: P < 0.01 for paired image choice, P < 0.05 for unpaired image choice). The reaction time data thus suggest that the microstimulation systematically influenced the time required to reach a decision in addition to altering the likelihood of making a particular choice.

FIG. 7.

Stimulation induced response shifts and a test of stimulus specificity. A: population plots and psychometric functions from monkey S (12 experiments, top row) and monkey T (12 experiments, bottom row). • and ○, responses with and without electrical stimulation, respectively. — and - - -, fitted psychometric functions for trials with and without electrical stimulation. The psychometric function is obtained by logistic regression analysis for the choices made in response to the stimuli mixture containing the image previously been associated with microstimulation (left) and for mixtures wherein neither of the source images had been previously associated with microstimulation (right). B: 24 experiments were conducted in 2 monkeys comparing the response to ambiguous stimulus blends (Fig. 6) with and without stimulation. Effect size (abscissa) is measured in terms of response bias with positive indicating more responses in the direction of the unblended stimulus that had been paired with stimulation during learning. On average, both monkeys showed a significant shift in the expected direction with a mean amplitude across the set of experiments of 11%. C: the specificity of the stimulation effect was assessed by comparing the shift in response bias caused by microstimulation between the blends of upright stamps (which always included the stimulus that had been learned with microstimulation) and the rotated blends. The mean effect size across the 24 experiments decreased by 68% in the transfer condition (P < 0.001, Wilcoxon signed-rank test). Thus the efficacy of the stimulation did depend in part on the interaction between the visual patterns presented and the presence or absence of microstimulation. Error bars denote standard errors.

FIG. 8.

Effects of IT microstimulation on reaction times. A: pooled reaction times for ambiguous trials for the response to the stimuli mixture from 1 of which had been previously been associated with microstimulation show that microstimulation affected choice times. Choice reaction times were speeded for stimulated trials in which the monkeys chose the predicted (microstimulation associated) stimulus and were slowed by microstimulation on trials in which the animals chose the unpredicted stimulus (choice × stimulation interaction, monkey S; P < 0.001, monkey T; P < 0.01). Error bars denote 95% confidence intervals. B: pooled reaction times for ambiguous trials in response to the mixtures containing images neither of which had been previously been associated with microstimulation. These data do not show the systematic bias observed for the associated mixtures (choice × stimulation interaction, monkey S; P > 0.2, monkey T; P > 0.1).

To test for the stimulus specificity of learning effects, we examined trials with ambiguous blends between two different images, neither of which had been previously been associated with microstimulation (bottom pair of stimuli, Fig. 5A). These stimuli were interleaved with the first pair during training and testing. Regression analysis revealed a small but significant stimulation effect in monkey S (coefficient for the stimulation factor = 0.2, P < 0.05, Fig. 7A, right top) and no stimulation effect in monkey T (coefficient for the stimulation factor = 0.09, P > 0.3, Fig. 7A, right bottom). We found that the stimulation effect was less than one-third of that observed for stimulus blends containing the originally associated item (11.0% for associated pairs, 3.5% for unassociated pairs, Fig. 7C), a statistically significant difference (P < 0.001, Wilcoxon signed-rank test). We also found no clear systematic influence on the monkeys' reaction time for the unassociated blends (Fig. 8B, monkey S: stimulation × choice interaction P > 0.2, monkey T: stimulation × choice interaction P > 0.1; 2-way ANOVA). It thus appears that while there may be limited transfer of the induced bias toward a particular response, this bias is stronger for patterns sharing visual features with the originally stimulated item. This specificity suggests that the effects of the microstimulation in IT directly interacted with stimulus induced activity and did not provide independent signals affecting choice behavior.

DISCUSSION

We investigated the effects of IT microstimulation during learning in cases where the stimulation was exclusively informative, conditionally informative, and informative but not necessary for the classification task. We first demonstrated that monkeys can use stimulation as a cue for discrimination in an exclusively informative design. We explored the integrative or interactive nature between visual and electrical stimulation in the conditions with electrical stimulation alone and re-learning. We then demonstrated that the simple detection of the presence of microstimulation is not adequate to explain the learning we observed as relative timing between electrical and visual stimulation was also shown to be learnable. Using a paradigm where stimulation was informative but not necessary for the classification, we were able to create novel mixture stimuli to assess the transfer of stimulation learning during recognition of novel images. In this task, we found a systematic effect of learning in subsequent recognition and the effect was stimulus selective. These results are consistent with the idea that the activity of IT cell populations plays an important role in the formation of new associations between visual objects and other central representations.

Microstimulation during visual association learning

While our experimental protocol shares some elements in common with previous intracortical microstimulation studies (Romo et al. 2000; Salzman et al. 1992), there are fundamental differences in the approach. Previous microstimulation studies in both primary (Tehovnik et al. 2002) and extrastriate visual cortex (Bisley et al. 2001; Celebrini and Newsome 1995; Ditterich et al. 2003; Gold and Shadlen 2000; Liu and Newsome 2005; Salzman et al. 1992) as well as somatosensory area S1 (Romo et al. 2000) were predicated on the idea that preexisting functional assemblies could be specifically targeted by microstimulation. In our study, we did not target regions exhibiting particular selectivity and tailor the stimuli to suit this selectivity. Instead we explored the possibility that as a putative visual association area, IT contains pools of neuronal assemblies that can be tapped to link unfamiliar visual patterns with new associations. The goal was not, therefore, to reconstruct an internal image of the stimulus but rather to create a distributed activity profile that is both repeatable and learnable.

Our results add to the evidence that the differential activity in IT during visual association learning is sufficient for establishing new associations for complex visual patterns, a result that complements many correlational studies revealing changes in neural selectivity following visual learning (Baker et al. 2002; Erickson and Desimone 1999; Freedman et al. 2006; Kobatake et al. 1998; Messinger et al. 2001; Sakai and Miyashita 1991; Sigala and Logothetis 2002) as well as reports that functional inactivation of IT impairs learning (Horel 1996; Iwai and Mishkin 1969; Merigan and Saunders 2004; Voytko 1986).

We note some critical differences between our experiments and previous studies focusing on brain stimulation learning in monkeys (Doty 1965; Doty et al. 1956, 1964). These authors trained monkeys to associate electrical stimulation with a conditioned stimulus of either food-reward or shock-avoidance with the conditioned response of either nose, chin, or tongue pressing a plate or a manual lever press. These studies reported that following extensive paired conditioning between the unconditioned stimulus (food or mild shock) and electrical stimulation, stimulation by itself could elicit a conditioned response, whereas electrical stimulation in the absence of conditioning never elicited a conditioned response. These studies indicated that electrical stimulation in many cortical areas, including IT, could be used as a conditioned stimulus. In our visual classification experiments, we found that IT stimulation was more effective than nearby stimulation in an auditory sensitive cortical area. Doty et al. (1964) observed quick and strong generalization after establishing the initial “neocortical conditioned stimulus.” In contrast, for our experiments the learning induced by microstimulation was specifically related to training in a single session, and we observed no evidence of learning to learn in the current paradigm. Instead our results suggest that a local perturbation of IT can be integrated with visual association learning for particular images. In Doty's studies, the monkeys were generally trained to associate a visual cue with the unconditioned response and then transferred this training to electrical stimulation—something we never did. Finally, most of Doty's experiments were conducting using surface electrodes and currents much larger than those employed in the current study. Thus differences in both the task design and stimulation method make specific comparisons with Doty's classic studies difficult.

More recently, it has been shown that microstimulation in many visual areas, including IT, can be detected after the initial training of detection of peripheral low-contrast visual stimuli (Murphey and Maunsell 2007). In this study, the stimulation methods were similar to the current study. The current threshold for the detection was smaller than in our study. Although it is not clear how learning to detecting low-contrast visual stimuli transfers to learning to detect microstimulation, clear task differences distinguish our study from that of Murphy and Maunsell (2007).

The current study also explored the integrative and interactive relationship between visual and electrical stimulation. Evidence for integration between both types of stimulation was demonstrated by the failure to elicit a response by microstimulation itself, the lack of transfer for re-learning (Fig. 1C), and successful learning in the relative timing condition (Fig. 3B). The stimulus selective and association selective response bias observed in the generalization paradigm (Figs. 7C and 8) further support the existence of an interaction between the visual and electrical stimulation. However, the degree of this interaction should be considered carefully because it could not be directly assessed from the relative timing experiment and the fact that we did observe limited response bias for nonassociated pairs in one of two monkeys in the generalization paradigm. We note that the limited transfer in the generalization can be explained by the high degree of similarity between the stimulation paired stimuli and unpaired stimuli. Examining the degree of the association transfer by manipulating the degree of visual similarity between the stimulation paired and unpaired stimuli may be a good test for assessing whether the transfer is perceptual or motor in nature.

Perception elicited by microstimulation in IT

One interpretation of the establishment of microstimulation induced learning is that microstimulation of IT during the learning causes subjective experiences to be learned. It is known that local microstimulation of primary visual cortex causes the experience of visual phosphenes in humans (Brindley et al. 1972) and can result in goal directed eye movements in monkeys, suggesting they too may see localized patches of light (Bradley et al. 2005; Tehovnik et al. 2005). Although studies showing the tight link between the activity of IT cells and explicit visual perception seem to support this subjective experience view (Sheinberg and Logothetis 1997, 2001), the subjective effects of electrical microstimulation in higher visual areas are not easily characterized (Doty 1969; Murphey and Maunsell 2007). Recently Afraz and colleagues (2006) showed that the microstimulation of “face-selective” cell clusters within IT cortex in macaques could bias performance in a face/nonface categorization task. Their results support the link between the activity of face selective clusters and face perception. In the Afraz et al. (2006) study, microstimulation centered in large face selective clusters was more effective at biasing choices in favor of faces, and, interestingly, they did not observe any choice bias when they stimulated nonface regions. Our recording and stimulation sites covered a large area within IT. Cells at all these sites showed significant response to nonface objects. We chose not to test with face stimuli because we were particularly interested in whether stimulation could affect visual discrimination behavior for arbitrary stimulus patterns that the monkeys had never seen before. Given the overall small percentage IT containing dense face clusters (Tsao et al. 2006), it is assumed that many of our stimulation sites were centered outside these areas. It is possible that topological clustering of cells selective for nonface properties (Fujita et al. 1992; Tsao et al. 2006) contributed to the efficacy of our paradigms, although we did not attempt to fully characterize the visual selectivity of the stimulated areas in the current experiment. In any case, our results demonstrate that microstimulation of areas not particularly chosen for face selectiveness can still predictably bias subjects' performance in categorization tasks following relatively brief training in a single experimental session. Indeed the magnitude of our average response shift for novel image blends in the predicted direction was virtually identical in magnitude (∼11%) to that observed by Afraz et al. (2006) for faces.

The series of cases reported by Penfield and colleagues (Penfield and Perot 1963; Penfield and Rasmussen 1950) conducted during the surgical treatment of human patients with intractable epilepsy suggest that larger-scale stimulation of regions of damaged areas within the temporal lobes can elicit vivid memories, hallucinations, or impressions of familiarity. They found, however, that these experiences were never reported in response to stimulation of healthy (“silent”) cortex, and thus their relation to the kinds of microstimulation used in the present experiments is not clear.

Another interpretation is that the learning occurred without explicit perception. Behavioral studies indicate that IT is involved in implicit visual discrimination tasks (Mishkin 1982; Phillips et al. 1988) as well as explicit recognition tasks (Gaffan and Murray 1992; Horel et al. 1987). In a human case, visual classification learning could be established even with extensive damage to lower and mid-level visual areas but intact higher visual areas. In that case, the patient had great difficulty perceiving the stimuli despite being capable of categorizing them (Rosenthal and Behrmann 2006).

Candidate neural substrates underlying learning

The current study shows that intracortical microstimulation of IT is sufficient for disambiguating visual patterns during visual association paradigms. The results, however, do not necessarily establish that the learning takes place in IT. Rather they indicate that microstimulation interacts with visual signals in IT. These results do not imply that IT is the only area involved. One natural extension of the current study is to apply the same basic protocol in different brain areas to study their relative contributions to visual association learning.

Furthermore, the exploration of learning with artificially controlled activity in the current paradigm opens the possibility of direct access to potential candidate sites for plasticity perhaps bypassing normal physiological routes. Possible candidates underlying the artificial activity-induced learning include the interactive connection among frontal and temporal cortices (Miller et al. 1996; Ungerleider et al. 1989), temporal-limbic circuitry (Webster et al. 1991; Wirth et al. 2003), and the temporal-striatal loop (Brown et al. 1995; Fernandez-Ruiz et al. 2001; Yeterian and Pandya 1995). It is also possible that microstimulation-induced plastic changes in local circuits within IT. One previous study, for example, reported long-term changes in synaptic efficacy within IT induced by high-frequency electrical stimulation with similar parameters used in current study (Murayama et al. 1997). We do not, however, have any direct evidence that our protocol induced any lasting changes in cell responses within IT. Given the relative ease of establishing the learning and the manipulability of behavioral and physiological conditions, subsequent studies on plastic neuronal changes in these pathways could help further uncover the neural mechanisms of visual association learning.

Rapid advances in techniques for activation and inhibition of targeted neural populations by optical and molecular methods (Deisseroth et al. 2006; Han and Boyden 2007) will undoubtedly provide investigators with tools for even more selective and precise control of neural activity than we have used. Harnessing these techniques for control of complex behavior, however, will require a detailed understanding of how pattern-specific manipulations of neural activity can be efficiently learned and acted on.

GRANTS

This work was supported by the McDonnell-Pew Program in Cognitive Neuroscience, National Eye Institute Grant R01-EY-014681, and the James S McDonnell Foundation.