Abstract
Experience-dependent changes in the response properties of ventral visual stream neurons are thought to underlie our ability to rapidly and efficiently recognize visual objects. How these neural changes are related to efficient visual processing during natural vision remains unclear. Here, we demonstrate a neurophysiological correlate of efficient visual search through highly familiar object arrays. Humans and monkeys are faster at locating the same target when it is surrounded by familiar compared with unfamiliar distractors. We show that this behavioral enhancement is driven by an increased sensitivity of target-selective neurons in inferior temporal cortex. This results from an increased “signal” for target representations and decreased “noise” from neighboring familiar distractors. These data highlight the dynamic properties of the inferior temporal cortex neurons and add to a growing body of evidence demonstrating how experience shapes neural processing in the ventral visual stream.
Keywords: inferotemporal cortex, primate, object recognition, visual search, familiarity, neurophysiology
Introduction
The primate visual system is grossly divided into two main streams of information processing. The more ventral of these streams is thought to be responsible for the representation and perception of visual objects (Ungerleider and Mishkin, 1982; Goodale and Milner, 1992). The apparent ease with which we recognize objects in the real world suggests that the cells in the ventral stream are wired to efficiently extract information from the complex visual environment. Recent evidence suggests that the response properties of these neurons are more plastic than originally appreciated, and it is now widely believed that experience-dependent plasticity in the ventral visual stream underlies this processing efficiency, especially in cluttered and noisy environments (Kourtzi and DiCarlo, 2006).
In primates, the inferior temporal cortex (IT) plays a critical role in the recognition and perception of complex visual objects (Logothetis and Sheinberg, 1996; Tanaka, 1996). It is likely that experience-dependent changes in the response properties of IT neurons underlie our enhanced recognition of familiar objects. Indeed, there are many reports of such changes after learning (Sakai and Miyashita, 1991; Logothetis et al., 1995; Erickson et al., 2000; Jagadeesh et al., 2001; Baker et al., 2002; Sigala and Logothetis, 2002). In particular, discrimination training leads to sharpened stimulus selectivity (Kobatake et al., 1998; Freedman et al., 2006). These changes have been observed for isolated objects, but what consequences they have for processing during more natural situations, with multiple objects and self-guided viewing, remains unknown. In the current study, we tested the hypothesis that sharpened stimulus selectivity enhances processing efficiency of familiar environments by limiting the interference between the neural representations of adjacent objects. Specifically, we investigated the effects of context familiarity, a property known to affect exploratory visual behavior (Malinowski and Hübner, 2001; Shen and Reingold, 2001; Mruczek and Sheinberg, 2005), on the response profiles of IT neurons.
We have shown previously that humans are more efficient at searching for targets among familiar distractors than among unfamiliar distractors when using complex visual stimuli (Mruczek and Sheinberg, 2005). In a similar task using letters and numbers, Greene and Rayner (2001) compared eye movements across distractor conditions and suggested that this effect results from a wider processing span in the presence of familiar distractors. In other words, a larger area of visual space can be processed during each fixation in a familiar context compared with an unfamiliar one. In the current study, we provide neurophysiological support for this hypothesis. We found that target-selective IT neurons were more sensitive to the same peripheral targets in familiar compared with unfamiliar contexts. These results demonstrate that experience-dependent changes in the ventral visual stream lead to enhanced neural sensitivity to the presence of target objects when surrounding stimuli are well known.
Materials and Methods
Animals, surgery, and recording techniques.
Two male rhesus monkeys (Macaca mulatta; monkey S and monkey M), ages 6 and 10 years and weighing between 9–12 kg, were the subjects in this study. Before the experiment, the monkeys had been familiarized with the behavioral apparatus and had participated in unrelated studies. Both monkeys had a recording chamber implanted over the left hemisphere (Horsley-Clark coordinates: +15 anterior, +20 lateral) and a titanium head post for head restraint. All surgeries were performed using sterile technique while the animals were intubated and anesthetized using isoflurane gas. All procedures conformed to the National Research Council Guide for the Care and Use of Laboratory Animals as well as the Brown University Institutional Animal Care and Use Committee. Both monkeys are currently participating in other experiments.
During each recording session, a single electrode was advanced through a guide tube (25 gauge) inserted to a level just below the dura (monkey S), or through a chronic guide tube (monkey M). Electrodes were composed of a tungsten core with a glass coating (Alpha-Omega, Nazareth, Israel) and were advanced using a micropositioner (David Kopf Instruments, Tujunga, CA). Neural signals were amplified (model A-1; BAK Electronics, Germantown, MD), filtered (monkey S only; model 3364, Kron-Hite, Brockton, MA; 100 Hz to 12 kHz) and digitized at 34 kHz. Cells were isolated on-line using a threshold and two time-amplitude discrimination windows (custom software). However, spike times used for the analyses here were based on a similar off-line analysis run on the stored analog signal.
Eye movements were recorded using an EyeLink II video eye tracking system, running at 500 Hz (SR Research, Mississauga, Ontario, Canada). The analog outputs from the eye tracking hardware were sampled by the control system at 1 kHz (see below) and a moving average was stored to disk every 5 ms (200 Hz). Saccades were automatically extracted from off-line eye records using a velocity-based algorithm written in C, which marked the start and end time, and start and end position for every saccade on each trial. The parameters of this algorithm were set to reliably detect saccades down to ∼0.4° in amplitude. Fixation times were defined as the end of one saccade to the start of the following saccade.
Stimulus set.
The familiar stimulus set contained 100 full-color images of everyday objects (Hemera Photo-Objects, Gatineau, Quebec, Canada). Objects subtended ∼1.5 × 1.5° and appeared on a uniform gray (50%) background. Fifty images were used as targets and the monkeys were rewarded for pressing a button (left for 25 images, right for 25 images) whenever the target appeared in the display. The remaining 50 images did not have a response association and served as familiar distractors. The monkeys had extensive experience with both familiar distractors and targets (during months of preliminary behavioral training). The target-response mappings were consistent across days for each monkey. The images assigned as targets and distractors were different for each monkey.
In addition, a novel stimulus set containing 250 full-color images of everyday objects (Hemera Photo-Objects) was used as unfamiliar distractors. Unfamiliar distractors were defined as images that the monkeys had never seen before an experimental session. After each daily session, any images from the unfamiliar distractors set that had been viewed, even if only during the passive-viewing task (see below), were removed from the novel stimulus set and never used again.
Display and task descriptions.
Stimuli were presented on a dual-processor x86 graphics workstation, running an OpenGL-based stimulation program under Windows XP (Microsoft, Seattle, WA). The screen resolution was 1024 × 768 with a vertical refresh rate of 100 Hz. Behavioral control for the experiments was maintained by a network of interconnected personal computers running the QNX real-time operating system (QNX Software Systems, Ottawa, Ontario, Canada). This system provides deterministic control and acquisition of button responses and eye position and communicates with the dedicated graphics machine using isolated high-speed Ethernet and direct digital input/output. Experimental control and the collection of behavioral measures was conducted using custom-written programs. All behavioral data, such as button responses and eye-position signals, were available for on-line monitoring and stored to disk for off-line analysis.
During the passive viewing task, the monkeys initiated trials by fixating, for 500 ms, a small spot at the center of the screen. After acquisition, the fixation spot was extinguished and three to eight isolated stimuli were presented sequentially at the center of the screen for 200 ms with a 200 ms interstimulus interval. At the end of the stimulus period, a second fixation spot was presented in a randomly selected location 6° above, below, to the right, or to the left of the center of the screen. The monkeys were required to fixate the second spot to receive juice reinforcement.
For the visual search task, monkeys were initially required to fixate a small spot at the center of the screen for 350 ms, after which the fixation spot was removed and the search array appeared. Note that except for the initial fixation, there were no explicit constraints on the monkeys' eye movements during the time that the visual display was present. Rather, the monkeys were allowed to freely view the display. The monkeys' task was to locate any known target and press the button (left or right) that had previously been associated with that target. Correct choices were immediately followed by delivery of a juice reward and the removal of the stimulus array from the screen. Feedback was also provided by auditory cues indicating correct and incorrect trials.
Each individual trial of the search task included one target image and 30 distractor images. The set of 30 distractors included three repetitions of 10 different objects. The distractors on a given trial were either all familiar distractors or all unfamiliar distractors. The specific target and distractor stimuli used for each session were selected based on the stimulus-selectivity of each neuron (see below).
Targets were located at one of five fixed eccentricities from the initial fixation (0, 2, 4, 6, or 8°). Distractor positions were randomly assigned within 9.5° of the initial fixation spot such that no two objects overlapped. For every familiar distractor trial, an unfamiliar distractor trial with the same stimulus configuration (target and distractor positions) was included to ensure equivalent stimulus configurations across distractor conditions. Familiar distractor trials and unfamiliar distractor trials were randomly interleaved in blocks of 40 trials (four targets by two distractor types by five target eccentricities).
Neuron selection.
The inferior temporal cortex was located based on the stereotaxic placement of the recording chamber and by counting white-gray matter transitions. Recordings were made in the ventral surface of the temporal lobe, lateral to the anterior medial temporal sulcus, in the anterior inferior temporal cortex. We tested the response of every well isolated cell encountered in the passive viewing task described above. The selectivity of the cell was analyzed by the experimenter using on-line raster plots. Off-line analysis confirmed the selectivity of our neuron pool.
There were two phases of stimulus-selectivity analysis run on each neuron before behavioral trials. The first phase was to select appropriate target images from the monkey's known set. Any cell that showed sufficient selectivity such that there was one target (with either hand association) that consistently activated the cell to a greater extent than three other targets (one with the same hand association and two with the opposite hand association) was selected for additional analysis. The target that activated the cell consistently is referred to as the “effective” target and the other three targets are referred to as “ineffective” targets. The ineffective targets, like the distractors (see below), were specifically chosen to not evoke a significant response from the recorded cell.
The second phase was used to select appropriate “familiar” and “unfamiliar” distractor images for the search task. Ten distractors, referred to as familiar distractors, were randomly selected from the monkey's known set. In addition, we randomly selected 10 distractors from a set of novel stimuli the monkey had never seen before that day. These stimuli are referred to as unfamiliar distractors. The response of the cell to these 20 distractors was analyzed using the passive viewing task. Any stimulus that elicited a response that was similar in strength to that of the effective target was replaced by distractors randomly selected in the same manner as the initial set. This process continued until we obtained a set of 20 distractors (10 familiar and 10 unfamiliar) that were ineffective at driving the neuron to complement the four selected target stimuli. On average, an equal number of familiar (median of 3; range of 0–10) and unfamiliar distractors (median of 2; range of 0–7) were removed during each of the 30 experimental sessions. All search trials contained one of the four selected targets embedded among either familiar or unfamiliar distractors. Note that because of our stimulus selection process, all single-unit analyses are based on the response of the neuron to the effective target compared across distractor type.
We completed the stimulus-selectivity analysis and a sufficient number of behavioral trials in 30 neurons (11 in monkey S; 19 in monkey M). The results presented below are for the total of 30 experimental sessions both monkeys. Analysis of each monkey's data separately yielded qualitatively similar results (supplemental text, supplemental Fig. 1, available at www.jneurosci.org, as supplemental material).
Data analysis and statistics.
Two separate time windows were used to compare the neural response evoked by familiar and unfamiliar distractors during the passive viewing task. The early period lasted from 50 to 175 ms after stimulus onset and the late period lasted from 175 to 300 ms after stimulus onset. Comparisons were made across all familiar and unfamiliar distractors, as well as the “best” familiar and best unfamiliar distractors (i.e., distractors that caused the largest response for each neuron) and the “worst” familiar and worst unfamiliar distractors (i.e., distractors that caused the smallest response for each neuron). Firing rates were compared across distractor class using a Wilcoxon matched-pairs test.
For the analyses of single-unit activity during visual search, individual spike times were first backward-shifted by the onset latency of each neuron, as measured during the passive viewing task. To calculate onset latency, we first computed spike density functions (SDFs) using an asymmetric filter composed of separate truncated Gaussian filters, one for the prespike time range (SD of 15 ms) and one for the postspike time range (SD of 5 ms). The asymmetric filter provides an estimate of the instantaneous firing rate while minimizing backward biasing of each spike (Thompson et al., 1996; Brincat and Connor, 2004). Onset latency was defined as the time that the SDF exceeded 10% of the baseline-to-peak difference and continued to increase for at least 15 ms.
For the active exploration analysis, we calculated the neuronal firing rate during each fixation, defined as the number of latency-offset spikes occurring during that fixation divided by the fixation duration. We then took the mean firing rate across fixations binned by the distance between the fixation position and the target position. For this analysis, and the corresponding control analyses, we only analyzed fixations within 13° of the target because there were very few fixations outside of this range and the neurons were not sensitive to such eccentric targets. We verified that the distribution of fixations relative to the bin's center did not differ across distractor type (t(29) = −0.40; p = 0.69, paired t test).
To compare activity across neurons with different firing rates, we normalized the response of each neuron according to its activity during the visual search task. For response normalization: Rnorm = (R − Rbaseline)/(Rmax − Rbaseline), where R is the firing rate, Rbaseline is the baseline firing rate, Rmax is the maximum firing rate, and Rnorm is the normalized firing rate of the neuron. Rbaseline was defined as the mean firing rate across all fixations when the monkey was searching for an ineffective target, and Rmax was defined as the mean firing rate across all fixations of the effective target. Thus, for our normalized response scale, the average response during ineffective target searches was 0.0 and the average response to a fixated effective target was 1.0 (for an example cell, see Fig. 4B). For all the population analyses presented below, qualitatively similar results were obtained when using raw firing rates instead of the normalized rates.
Figure 4.
Neural response during active exploration of the search array. A, Example trial with the effective target for one neuron (same cell in Fig. 2A). Horizontal (H) and vertical (V) eye position (top, solid lines) and the target position (dashed lines) as well as the distance from the gaze position to the target (middle) are shown. Shaded regions denote fixations during search. Spike times, backward-shifted by the onset latency of the neuron, are depicted by the black vertical marks. Firing rates during each fixation were binned by the distance between the fixation position and the target position (bottom). Error bars denote SE. B, Normalized population response (±SE) during active exploration. For the effective target, the response magnitude decreased with increasing distance between the fixation position and the target (p < 0.00001) and fell off quicker when targets were embedded among unfamiliar distractors. Across all eccentricities, there was a 25% decrease in neural activity when targets were among unfamiliar distractors (p < 0.001). There was no significant difference across distractor condition when the target of the search was an ineffective target (p = 0.42).
For the neuron population, we analyzed mean normalized responses using a two-way repeated-measures ANOVA (distractor type by distance from fixation to target). In addition, we reran all of the active exploration analyses after excluding the very first fixation of each trial to verify that the observed results were not driven by the initial transient response measured in the array onset analysis described below. All of these analyses revealed qualitatively similar results to those reported.
To verify that the behavioral effects of distractor familiarity did not bias the results of the active exploration analysis, we ran a control analysis on a subset of trials controlling for differences in response time and number of fixations across distractor type. For this analysis, we removed the fastest familiar distractor trials and the slowest unfamiliar distractor trials using an iterative algorithm until the mean response time for familiar distractor trials was greater than the mean response time for unfamiliar distractor trials. More explicitly, we first removed the fastest 1% of familiar distractor trials and slowest 1% of unfamiliar distractor trials and compared the mean response time across distractor type for the remaining trials. If the mean response time for familiar distractor trials was less than the mean response time for unfamiliar distractor trials, we then repeated the procedure, but this time removing 2% of the trials. This process continued until the remaining set of trials was composed of, on average, faster unfamiliar distractor searches.
For the array onset analysis we, calculated the firing rates of each cell in a 155 ms time window aligned to the onset latency of the neuron. This is equivalent to aligning the analysis window to onset of the search array and backward-shifting individual spike times to account for the onset latency of each neuron. The 155 ms window size was chosen to match the median first fixation duration of the monkeys in our task. For the neuron population, we analyzed the mean normalized response using a two-way repeated ANOVA (distractor type by target eccentricity). For response normalization, we used the same formula as for the active exploration analysis above, except Rbaseline was defined as the mean firing rate when the search array contained an ineffective target and Rmax was defined as the mean firing rate across all trials when the effective target was displayed at 0° eccentricity. As with the active exploration analyses, qualitatively similar results were obtained for the array onset population analyses when using raw firing rates instead of the normalized rates.
We quantified the size of the distractor familiarity effect during each experimental session by defining a distractor familiarity effect index (DFI). For behavioral differences in response time (RT), DFIRT = (RTfam − RTunfam)/(RTfam + RTunfam), where RTfam is the mean response time during familiar distractors trials and RTunfam is the mean response time during unfamiliar distractors trials. Negative values for DFIRT indicate faster response times during trials with familiar distractors. For physiological differences in firing rate, DFIFR = (FRfam − FRunfam)/(FRfam + FRunfam), where FRfam is the mean firing rate of the neuron during familiar distractors trials and FRunfam is the mean firing rate of the neuron during unfamiliar distractors trials. Positive values for DFIFR indicate higher firing rates for target-selective neurons in the presence of familiar distractors. We calculated DFIFR independently for the two analysis epochs, array onset and active exploration.
To compare the response of our isolated cells with that of the surrounding population of IT neurons, we measured multiunit activity (MUA) during the passive viewing task and during the visual search task. MUA was extracted off-line from the stored analog signal by setting a threshold to obtain an average of 40 events in a 200 ms time window (200 Hz) preceding stimulus onset across a block of trials. This procedure, has been used in previous neurophysiology studies to minimize the arbitrary nature of the multiunit signal across recording sessions (DeAngelis et al., 1998; DeAngelis and Newsome, 1999). For the passive viewing task, distractor-evoked MUA was compared during an early (50–175 ms after stimulus onset) and late (175–300 ms) response period using a Wilcoxon matched-pairs test. We also compared stimulus-induced MUA during the early and late response periods across familiar and unfamiliar distractors by calculating the selectivity broadness (Kobatake et al., 1998; Freedman et al., 2006). Selectivity broadness was defined as the proportion of stimuli that evoked a significant MUA response across the set of the 10 familiar distractors or 10 unfamiliar distractors used in the search task. A response was deemed significant if the stimulus-evoked MUA exceeded the background response according to a Wilcoxon ranked-sum test (α-level of 0.01). We compared the selectivity broadness of our neuron population across distractor type using a Wilcoxon matched-pairs test. For the visual search task, the MUA analysis was limited to trials with an ineffective target to restrict the influence of the well isolated neuron. We calculated the DFI (see above) for MUA during the array onset and active exploration portions of the search task. DFIMUA was defined using the same formula as DFIFR, but substituting the mean multiunit events/s for the mean single-unit firing rate.
All statistical analyses were performed using STATISTICA (StatSoft, Tulsa, OK) and custom software. Unless otherwise noted, significance was determined at an α-level of 0.05.
Results
Neural selectivity and stimulus selection
We recorded IT neurons from two macaque monkeys performing a visual search task in which the animals reported the presence of one of many known targets embedded in an array of distractors (Fig. 1). Specifically, we report data from 30 neurons with appropriate stimulus-selectivity and sufficient accompanying behavioral data to test the effects of distractor familiarity on neuronal response profiles during visual search. These neurons were initially selected based on their robust selectivity during a passive viewing task for one target, called the effective target, compared with three other targets, called ineffective targets. In addition, we randomly selected 10 familiar distractors (i.e., used as distractors in the search task for months) and 10 unfamiliar distractors (i.e., seen for the first time that day) for each experimental session. Like the ineffective targets, all of the distractors were selected to not evoke a strong response for the recorded cell. Note that, in this case, “familiar” refers to long-term experience rather than stimulus repetition or recency effects and does not imply an explicit response association.
Figure 1.
Visual search task. After fixating a central square, the stimulus array appeared and the monkey was allowed to freely view the display. The monkey's task was to press a button (left or right) that had been previously associated with the target. In the example displays, the target is the image of the silverware. The target appeared at 0° eccentricity in the left panel and at 4° eccentricity in the right panel. Dashed gray lines denote possible target eccentricities (0, 2, 4, 6, or 8°) and were not present in the actual displays. The left and right panels depict trials for monkey S with familiar and unfamiliar distractors, respectively. Note that distractor class is not distinguishable without previous knowledge of the monkey's experience with these specific images. The actual display contained full-color objects.
Figure 2A shows the response of an example neuron to all the targets and distractors used in one experimental session. The population response to the effective target, the ineffective targets, and the best familiar and best unfamiliar distractors (i.e., distractors that caused the largest response for each neuron) is shown in Figure 2B. The effective target caused the largest response and, importantly, there was no difference in the response of the neuron population to familiar or unfamiliar distractors. This was true for both the early (50–175 ms after stimulus onset) and late (175–300 ms after stimulus onset) response periods, and when comparing across all distractors tested (early period, p = 0.66; late period, p = 0.25, Wilcoxon matched-pairs test), as well as the best distractors (early period, p = 1.00; late period, p = 0.15, Wilcoxon matched-pairs test) and worst distractors (early period, p = 0.48; late period, p = 0.53, Wilcoxon matched-pairs test) for each neuron (Fig. 2C). The best familiar and best unfamiliar distractor evoked responses that were, on average, 35 and 34% of the response to the effective target during the early and late period, respectively.
Figure 2.
Stimulus-selectivity for an example neuron and the neuron population. Neurons were chosen based on their selectivity for one target, the effective target, during a passive viewing task. Other targets and distractors were selected such that they did not activate the neuron to the same extent as the effective target. A, Response of an example neuron (same cell in Figs. 4A, 7A) to all 24 stimuli used in the visual search task. The response to the effective target was strongest. B, The same was true across the population of neurons. The normalized response to the effective target, the ineffective targets, and the best distractors (i.e., distractors that caused largest response) are shown. Lightly shaded regions denote 95% confidence intervals. C, Median responses of the neuron population to the familiar and unfamiliar distractors did not differ during the early (50–175 ms) or late (175–300 ms) response period across all distractors tested, or when limiting the comparison to the best distractors or worst distractor (p > 0.15 in all cases). Box denotes the 25th to 75th percentile and line denotes the 10th to 90th percentile.
In the following analyses, we report the activation of these target-selective IT neurons across various search conditions. Although we report comparisons across distractor type (familiar vs unfamiliar), it is important to keep in mind that our stimulus selection process, as well as results reported below, ensure that the observed differences are only indirectly attributable to the distractors themselves. Familiarity with these surrounding objects modulates target-specific processing.
Behavior
Behavioral results from the visual search task are shown in Figure 3. Monkeys performed very well, making errors on <3% of trials, with no significant difference across distractor type (t(29) = 0.24; p = 0.81, paired t test). Consistent with previous reports from human studies (Greene and Rayner, 2001; Mruczek and Sheinberg, 2005), monkeys had shorter response times (606 vs 698 ms; t(29) = −6.46; p < 0.000001, paired t test) and made fewer fixations (3.3 vs 3.8; t(29) = −6.66; p < 0.000001, paired t test) when searching for the same targets embedded among familiar compared with unfamiliar distractors. There was no consistent difference in fixation durations across distractor type (150 vs 149 ms; t(29) = −0.39; p = 0.70, paired t test), which is also consistent with the human data (Greene and Rayner, 2001). Analogous behavioral effects were found when considering each monkey's data individually (supplemental text, supplemental Fig. 1, available at www.jneurosci.org, as supplemental material).
Figure 3.
Behavioral results from the visual search task. A, Visual search for targets embedded among familiar distractors was ∼100 ms faster than search among unfamiliar distractors (p < 0.000001). B, Monkeys also required fewer fixations to locate targets among familiar distractors (p < 0.000001). C, There was no consistent difference in fixation durations across distractor type (p = 0.70). Center line denotes the mean, the box denotes the 25th to 75th percentile, and the line denotes the 10th to 90th percentile. D, Proportion of saccades to the target as a function of distractor class and the distance between fixation and the target position. Monkeys were more likely to saccade to a target embedded among familiar distractors and this difference was limited to nearby targets (p < 0.001). Error bars denote SEM.
There were no explicit constraints on the monkeys' eye position when the search array was on the screen. We analyzed the monkey's eye movement pattern, or “scanpath,” by measuring the proportion of fixations that were followed by a saccade to the target as a function of the distance between each fixation and the target (Fig. 3D). A repeated-measures ANOVA (distractor class by distance between fixation and target) revealed a mean effect of the distance between fixation and target (F(6,174) = 146.38; p < 0.000001); monkeys were more likely to make a saccade toward a physically close target. There was also a main effect of distractor familiarity (F(1,29) = 18.08; p = 0.0002) and an interaction between distractor class and distance between fixation and target (F(6,174) = 4.48; p = 0.0003). As shown in Figure 3D, monkeys were more likely to saccade to a target embedded among familiar distractors, but this difference was limited to nearby targets.
The behavioral results show that monkeys were more efficient at searching through the multistimulus arrays when the distractors were highly familiar. To explore the familiarity-induced changes in neural activity that may account for this effect, we compared the response profile of target-selective IT neurons during visual search with both familiar and unfamiliar distractors.
Neural response during active exploration
We analyzed the response of our target-selective IT neurons during self-guided visual search by calculating the firing rate of each neuron during every fixation, defined as the number of spikes detected during the fixation divided by the fixation duration. Individual spike times were backward-shifted to account for the response latency of each neuron. We then binned fixations by the distance between the fixation position and the target to obtain an average neuronal firing rate in response to a target at that eccentricity during active visual search. Figure 4A shows the results of this analysis for an example neuron. Not surprisingly, during the monkey's self-guided active exploration for the target, this cell maintained its selectivity for the effective target and the response magnitude decreased with increasing distance between the fixation position and the effective target.
To quantify the physiological effects of distractor familiarity during active exploration, we calculated a normalized response profile for each neuron. For response normalization, 1.0 was defined as the mean firing rate for fixations of the effective target and 0.0 was defined as the mean firing rate, across all fixations, when the monkey was searching for an ineffective target (Fig. 4A, bottom) (Materials and Methods). We then compared the average normalized response profiles across the population of neurons as a function of distractor condition (Fig. 4B). We limited our analysis to fixations within a distance of 13° of the target because the monkeys made relatively few fixations outside this range. Furthermore, the target was not apparently discernable at large eccentricities; selectivity for highly eccentric targets was largely abolished in the presence of 30 distractors (Fig. 4). This is in contrast to the behavior of target-selective IT neurons when targets are presented in isolation (supplemental text, supplemental Fig. 2, available at www.jneurosci.org, as supplemental material).
A two-way repeated-measures ANOVA (distractor class by distance from fixation to target) on the normalized population response revealed an expected main effect of distance from fixation to target (F(6,174) = 76.66; p < 0.00001). Neural responses to the effective target decreased with increasing distance between the fixation position and the target. More remarkably, there was an overall trend of stronger responses to the effective target in familiar compared with unfamiliar contexts (F(1,29) = 14.34; p = 0.0007). Overall, there was a 25% decrease in target-related activity for searches through unfamiliar distractors. There was also a significant interaction between distractor class and distance from fixation to target (F(6,174) = 4.02; p = 0.0008). This is likely because the largest differences in firing rate occurred when the monkeys' fixation was relatively close to the target. Analogous neural effects were found when considering each monkey's data individually (supplemental text, supplemental Fig. 1, available at www.jneurosci.org, as supplemental material).
It is important to reiterate that we recorded neurons that were selective for one of the search targets. Indeed, there was no significant difference in neural modulation across distractor condition when the search target was one of the ineffective stimuli (F(1,29) = 1.05; p = 0.31) indicating that the firing rate differences across distractor type do not reflect differences in the selectivity of our neuron population for the distractors themselves. Rather, target-selective IT neurons were more sensitive to peripheral targets during visual search through familiar compared with unfamiliar distractor arrays.
In the next two sections, we verified that the enhanced target sensitivity was not a result of differences in search strategy across distractor type. We confirmed that the observed neural changes were independent of (1) the behavioral effects of distractor familiarity and (2) the deployment of spatial attention associated with saccadic eye movements across distractor condition.
Maintained neural effect in the absence of behavioral effects
As a consequence of the behavioral effects of distractor familiarity the monkeys made more fixations during search through unfamiliar distractors (Fig. 3). Hence, the number of fixations contributing to the active exploration analysis was not balanced across distractor type. Because of this imbalance it is unclear whether the observed differences in neural activity were the result of (1) a modest decrease in neural response for all fixations during unfamiliar distractor trials or (2) a large decrease in neural response for some fixations during unfamiliar distractor trials. This second scenario might occur if, for example, the response of the neuron population was similar across distractor type at the time the monkey “located” the target, but there were more fixations preceding this event for unfamiliar distractor trials.
To verify that the observed neurophysiological differences were not a result of an unbalanced analysis, we performed an analysis on a subset of trials controlling for response time and number of fixations across distractor type. For each neuron, we removed the fastest familiar distractor trials and the slowest unfamiliar distractor trials using an iterative algorithm (see Materials and Methods) until the mean response time for familiar distractor trials was greater than the mean response time for unfamiliar distractor trials. The result of the trial-reduction procedure is shown for an example neuron in Figure 5A. For this neuron, 14% of the total trials were removed (light gray bars). On average, we removed 6.2% of the total number of trials from each data set for this control analysis.
Figure 5.
Control analysis for the unbalanced number of fixations across distractor type contributing to the active exploration analysis. A, Histogram of response times for trials with familiar (upward bars) and unfamiliar distractors (downward bars) for an example neuron. For this analysis, the fastest familiar distractor trials and slowest unfamiliar distractor trials were removed (light gray bars). For the remaining subset of trials, search was, on average, faster for unfamiliar distractor trials. B, Mean response times and number of fixations across the population of control trial subsets. In contrast to the results for the full trial set (see Fig. 3), response times were longer (p < 0.001) and more fixations were required (p = 0.001) for search among familiar distractors. The box denotes the 25th to 75th percentile and the line denotes the 10th to 90th percentile. C, Normalized population response during active exploration of the search array for the subset of control trials. Results were comparable with the same analysis on the full data set (see Fig. 7). For the effective target, the response magnitude decreased with increasing distance between the fixation position and the target (p < 0.00001) and fell off quicker when targets were embedded among unfamiliar distractors. Across all eccentricities, there was a 24% decrease in neural activity when targets were among unfamiliar distractors (p = 0.005). There was a significant interaction between target eccentricity and distractor type (p = 0.0002). There was no significant difference across distractor condition when the target of the search was an ineffective target (p = 0.17).
We confirmed that for the subset of control data monkeys were slower (622 vs 612 ms; t(29) = 5.78; p = 0.000003, paired t test) and required more fixations (3.4 vs 3.3; t(29) = 3.51; p = 0.0015, paired t test) when searching for targets among familiar compared with unfamiliar distractors (Fig. 5B). Note that these differences are in the opposite direction of the behavioral differences reported for the full set of trials (Fig. 3). Performance (3 vs 2.5% errors; t(29) = −0.84; p = 0.41, paired t test) and fixation duration (148 vs 149 ms; t(29) = −0.35; p = 0.73, paired t test) did not differ across distractor type for the control subset of trials. However, the monkeys were still more likely to saccade toward nearby targets during familiar distractor trials (F(6,174) = 4.78; p = 0.0002, ANOVA, distractor class by distance between fixation and target interaction). We address potential effects of saccade pattern in the next section.
We repeated the active exploration analysis on the set of control trials. Figure 5C shows the normalized population response across distractor condition during active exploration for the subset of control trials. A two-way repeated-measures ANOVA (distractor class by distance from fixation to target) revealed near-identical results as the analysis using the full data set (Fig. 4). Neural responses to the effective target decreased with increasing distance between the fixation position and the target (F(6,174) = 72.48; p < 0.00001) and were significantly stronger when the surrounding distractors were familiar compared with unfamiliar (F(1,29) = 9.14; p = 0.005). There was also a significant interaction between target eccentricity and distractor type (F(6,174) = 4.59; p = 0.0002). Overall, there was a 23% decrease in target-related activity for searches through unfamiliar distractors. There was no significant difference in neural modulation across distractor condition when the search target was an ineffective stimulus (F(1,29) = 1.97; p = 0.17).
This analysis indicates that the enhanced processing efficiency for target-selective IT neurons during free viewing of familiar contexts is not an artifact of an analysis using a different number of fixations for familiar and unfamiliar distractor trials. Furthermore, these data indicate that when the behavioral effect of distractor familiarity are eliminated, and in fact reversed, the neural effects remain robust. Thus, the neural effect of distractor familiarity is not dependent on the behavioral effect. Rather, the increase in target sensitivity is a product of context familiarity and this is reflected in the monkeys' search behavior.
Neural effects are independent of saccade pattern
The monkeys' saccade pattern, in terms of the likelihood of target fixation, differed for search among familiar compared with unfamiliar distractors (Fig. 3D). Before the initiation of an eye movement, there is an obligatory shift of visual attention to the endpoint of the impending saccade (Hoffman and Subramaniam, 1995; Kowler et al., 1995; Deubel and Schneider, 1996). This attentional shift can modulate the response of visual neurons, for example, in area V4 (Fischer and Boch, 1981; Moore et al., 1998) and IT (Sheinberg and Logothetis, 2001). Thus, the results presented above may be related to differences in the deployment of spatial attention on familiar and unfamiliar distractor trials.
To verify that this is not the case, we reran the active exploration analysis on a subset of data that only included fixations that were not followed by a targeting saccade. This allowed us to measure the activity of target-selective neurons in the absence of directed spatial attention to the target. On average, we removed 22% of the total number of fixations collected for every data set. After this fixation-reduction procedure, two data sets were left with insufficient data to be included in the control analysis. Figure 6 shows the normalized population response across distractor condition during active exploration for fixations that were not followed by a targeting saccade. A two-way repeated-measures ANOVA (distractor class by distance from fixation to target) revealed similar results to the analysis using the full data set (Fig. 4). Neural responses to the effective target decreased with increasing distance between the fixation position and the target (F(6,156) = 83.47; p < 0.00001) and were significantly stronger when the surrounding distractors were familiar compared with unfamiliar (F(1,26) = 8.53; p = 0.007). Overall, there was a 25% decrease in target-related activity for searches through unfamiliar distractors. There was also a significant interaction between target eccentricity and distractor type (F(6,156) = 3.91; p = 0.001) for this control analysis. There was no significant difference in neural modulation across distractor condition when the search target was an ineffective stimulus (F(1,27) = 0.78; p = 0.38).
Figure 6.
Control analysis for possible differences in eye movement patterns across distractor type. The normalized population response during active exploration of the search array is plotted as a function of distractor type and distance between fixation and target position. For this subset of trials, all fixations that were immediately followed by a saccade to the target were removed. The results were comparable with the same analysis on the full data set (see Fig. 7). For the effective target, the response magnitude decreased with increasing distance between the fixation position and the target (p < 0.00001) and fell off quicker when targets were embedded among unfamiliar distractors. Across all eccentricities, there was a 25% decrease in target-related activity for searches through unfamiliar distractors (p = 0.007). There was a significant interaction between target eccentricity and distractor type (p = 0.001) for this control analysis. There was no significant difference in neural modulation across distractor condition when the search target was an ineffective stimulus (p = 0.38). D, Distractor; T, target.
This analysis indicates that the enhanced processing efficiency for target-selective IT neurons in familiar contexts during free-viewing was not the result of potential differences in saccade patterns across distractor type. Furthermore, because of the close relationship between eye movements and spatial attention during free-viewing behavior, the observed differences cannot be attributed to differences in the spatial distribution of attention that follow changes in search behavior across distractor type. Rather, the enhanced target sensitivity in familiar contexts is likely the result of experience-dependent changes in ventral visual stream neurons, which underlie the monkeys' search behavior.
Neural response to search array onset
To further explore the effects of distractor familiarity on the response properties of target-selective neurons we focused on the neural response during the initial fixation after the onset of the search array. The neural response during the initial fixation is primarily driven by bottom-up activity that determines stimulus saliency rather than top-down activity reflecting internally driven shifts of attention. Figure 7A shows the activity of an example neuron aligned to the onset of the search array. For trials including the effective target, this cell responded with a transient burst of activity and the magnitude of this burst decreased with increasing target eccentricity. Furthermore, the transient response was generally stronger for familiar distractor trials.
Figure 7.
Neural response to the onset of the search array. A, Raster and spike density functions for an example neuron (same cell as in Fig. 2A) aligned to stimulus onset as a function of target eccentricity and distractor type. For effective target trials, the transient response magnitude decreased with increasing target eccentricity and was generally stronger when distractors were familiar. Rasters are only shown for effective target trials. B, Normalized population response histograms aligned to the onset latency of each neuron. C, Normalized population response (±SE) to the array onset in a 155 ms time window (shaded region in A and B) starting at the response latency of each neuron, as measured in a separate passive-viewing task. For the effective target, the response magnitude decreased with increasing target eccentricity (p < 0.00001) and fell off quicker when targets were embedded among unfamiliar distractors. Across all eccentricities, there was a 28% decrease in response magnitude when targets were among unfamiliar distractors (p < 0.001). There was no significant difference across distractor condition when the target of the search was an ineffective target (p = 0.37). D, Similar effects of distractor (D) familiarity were apparent when limiting this analysis to only those trials in which the monkey's first saccade was not to the target (T).
To quantify the physiological effects of distractor familiarity, we calculated the firing rates of each cell in a 155 ms time window aligned to the onset latency of each neuron as measured during the passive-viewing task (Fig. 7A–B, shaded region). This window size was chosen to match the median first fixation duration. We aligned the analysis window to the neuronal onset latency to better compare neurons with different onset latencies (Kobatake et al., 1998; Rolls et al., 2003), which ranged from 64–151 ms (median of 97 ms) for our population. Figure 7B shows the population response histograms aligned to the neural onset latency of each cell. Similar to the example neuron, the population response decreased with increasing target eccentricity and was generally stronger when the distractors were familiar.
The average normalized neural response across the population of neurons is shown in Figure 7C. A two-way repeated-measures ANOVA (distractor class by target eccentricity) on the normalized population response revealed comparable effects of distractor familiarity for the array onset analysis as shown in the active exploration analysis above. Neural responses to the effective target decreased with increasing target eccentricity (F(4,116) = 34.11; p < 0.00001) and were significantly stronger when the surrounding distractors were familiar compared with unfamiliar (F(1,29) = 13.30; p = 0.001). There was also a significant interaction between distractor class and target eccentricity (F(4,116) = 4.30; p = 0.003), indicating that the largest differences in firing rate across distractor type occurred for midrange target eccentricities. Overall, there was a 28% decrease in target-related activity during unfamiliar distractor searches. Analogous behavioral effects were found when considering each monkey's data individually (supplemental text, supplemental Fig. 1, available at www.jneurosci.org, as supplemental material).
Also analogous to the active exploration analysis, there was no significant difference in neural modulation across distractor conditions when the search target was an ineffective stimulus (F(1,29) = 0.82; p = 0.37), indicating that our results are only indirectly attributable to the distractors themselves.
Finally, we reran the array onset analysis on a subset of control trials in which the monkeys' first saccade was not toward the target to verify that the observed differences were maintained in the absence of directed spatial attention to the target (Fig. 7D). For this analysis, we removed an average of 26% of the total number of trials. Three cells were excluded from this analysis because they did not maintain a sufficient number of trials after the trial-reduction procedure. A two-way repeated-measures ANOVA (distractor class by distance from fixation to target) revealed similar results to the analysis using the full data set. Neural responses to the effective target decreased with increasing distance between the fixation position and the target (F(4,104) = 29.60; p < 0.00001) and were significantly stronger when the surrounding distractors were familiar compared with unfamiliar (F(1,26) = 5.45; p = 0.03). Overall, there was a 22% decrease in target-related activity for searches through unfamiliar distractors. There was no significant interaction between target eccentricity and distractor type (F(4,104) = 1.76; p = 0.14) for this control analysis. There was no significant difference in neural modulation across distractor condition when the search target was an ineffective stimulus (F(1,26) = 0.34; p = 0.56).
Correlation between behavior and physiology
Because of the random nature of distractor selection, it is likely that some experimental sessions would yield strong behavioral and neural effects and other days would yield more modest effects. We quantified the effect of familiarity on the monkeys' behavior and the response profile of single neurons by defining a DFI (Materials and Methods) for both response time (DFIRT) and firing rate (DFIFR). Figure 8 shows the distributions of DFIRT (far right) and DFIFR for both the array onset (top left) and active exploration (top right) epochs. For this analysis, we excluded the first fixation on each trial from the active exploration epoch to avoid replicating the results of the array onset epoch.
Figure 8.
Comparison of behavioral and neural effects across experimental session. Histograms of DFIs are shown for response time (far right) and firing rate during the array onset (top left) and active exploration (top right) analysis epochs. Positive values for DFIFR indicate higher firing rates in response to the effective target when distractors were familiar. Negative values for DFIRT indicate lower response times for familiar distractor searches. Arrowheads denote distribution means, which were all significantly different from zero (p < 0.00001 in all cases). The magnitudes of the behavioral and neural effects were significantly correlated (Pearson coefficient, rp) across experimental session for both the array onset analysis (p = 0.02) and the active exploration analysis (p = 0.02). The open circle denotes the example neuron used in Figures 2A, 4A, and 7A.
Negative values for DFIRT indicate faster response times during trials with familiar distractors. Confirming the behavioral results presented above, the vast majority of DFIRT were negative (27 of 30) and all individually significant values were less than zero (n = 16; p < 0.05, permutation test). The population value was also significantly less than zero (mean of −0.07; p < 0.00001, permutation test). Positive values for DFIFR indicate higher firing rates for target-selective neurons in the presence of familiar distractors. Confirming the neurophysiological results presented above, the vast majority of DFIFR for both the array onset and active exploration epochs were positive (array onset, 23 of 30; active exploration, 27 of 30) and all individually significant values were greater than zero (array onset, n = 3, p < 0.05, permutation test; active exploration, n = 5, p < 0.05, permutation test). The population values were also significantly greater than zero (array onset, mean of 0.10, p < 0.00001, permutation test; active exploration, mean of 0.09, p < 0.00001, permutation test).
To better understand the relationship between response-time and firing-rate differences across distractor familiarity we, measured the correlation between behavioral and neural changes across experimental session. Figure 8 shows the comparison of DFI values across all 30 neurons for both the array onset analysis (bottom left) and the active exploration analysis (bottom middle). For both analyses, there was a significant negative correlation between firing rate and response time indices (array onset, Pearson's r = −0.44, p = 0.02, permutation test; active exploration, Pearson's r = −0.40, p = 0.02, permutation test). Thus, the enhanced processing efficiency of target-selective IT neurons and the enhanced search efficiency when distractors are familiar were closely related.
Multiunit activity
The results presented above have focused on the response of target-selective IT neurons during visual search. We have shown that these neurons respond more efficiently to peripheral targets in familiar compared with unfamiliar contexts. This increase in target sensitivity indicates that peripheral targets are more salient to the monkeys during visual search through familiar distractors. When discussing saliency, however, it is important to understand how the activity of single IT cells compares with that of the general population, especially with regard to local circuitry. Thus, we simultaneously recorded MUA from our microelectrodes.
Figure 9A shows the MUA evoked by familiar and unfamiliar distractors during the passive-viewing task. We observed a stronger MUA response to unfamiliar distractors over a broad time range, although the observed differences were generally larger during the later phase of the response. This was true across all presented distractors (early period, p = 0.00002; late period, p = 0.000002, Wilcoxon matched-pairs test), or when limiting the comparison to the best distractors (early period, p = 0.002; late period, p = 0.00006, Wilcoxon matched-pairs test) or worst distractors (early period, p = 0.0001; late period, p = 0.000002, Wilcoxon matched-pairs test) of each class (Fig. 9B). Additionally, in the passive viewing task a higher proportion of unfamiliar distractors produced a significant MUA response (selectivity broadness) compared with familiar distractors (Fig. 9C) (early period, p = 0.004; late period, p = 0.000006, Wilcoxon matched-pairs test). This indicates sparser representations, or higher stimulus selectivity, for familiar distractors. These data are consistent with previous reports of experience-related effects on neural selectivity for unbiased IT neuron populations (Kobatake et al., 1998; Freedman et al., 2006), as well as reports of decreased activity with stimulus repetition and familiarity (Miller et al., 1991; Li et al., 1993; Xiang and Brown, 1998).
Figure 9.
Multiunit activity during passive-viewing and visual search. A, Population MUA during passive viewing. Shaded region denotes 95% confidence intervals. B, Unfamiliar distractors induced stronger MUA than familiar distractors during both the early (50–175 ms) and late (175–300 ms) response periods. This difference was significant across all distractors tested, as well as when limiting the comparison to the “best” distractors or “worst” distractor of each class (p < 0.002 in all cases). The center line denotes median, the box denotes the 25th to 75th percentile, and the line denotes the 10th to 90th percentile. C, Selectivity broadness (proportion of stimuli causing a significant MUA response) was higher across unfamiliar distractors compared with familiar distractors during passive viewing for both the early (p = 0.004) and late (p = 0.000006) response phases. D, Histograms of distractor familiarity effect indices for MUA (DFIMUA) are shown for the array onset (top) and active exploration (bottom) analysis epochs. These data are limited to ineffective target trials. Arrowheads denote distribution means, which were both significantly less than zero (p < 0.05 in both cases).
We also calculated the distractor familiarity index (see above) for MUA (DFIMUA) during the array onset and active exploration epochs of the visual search task (Fig. 9D). For this analysis, we only included trials with ineffective targets to limit the influence of the well isolated single-unit response, which was selective for the effective target. A large majority of DFIMUA for both the array onset and active exploration epochs were negative (array onset, 22 of 30; active exploration, 22 of 30) and most individually significant values were less than zero (array onset, 12 of 13, p < 0.05, permutation test; active exploration, 7 of 9, p < 0.05, permutation test). The population values were also significantly less than zero (array onset, mean of −0.014, p = 0.001, permutation test; active exploration, mean of −0.010, p = 0.046, permutation test). Negative values for DFIMUA indicate higher MUA for unfamiliar distractor searches. The results for the MUA analysis are in direct contrast to the single-unit analysis presented above (Fig. 8). Whereas target-selective IT neurons showed relatively higher sensitivity to their preferred stimulus in familiar contexts, the MUA from surrounding cells was relatively higher for unfamiliar contexts.
The population DFIMUA values were approximately an order of magnitude smaller than the population DFIFR values, indicating relatively small differences in evoked MUA across distractor type (mean of ∼9 events/s). However, this is not surprising for two reasons. First, our electrodes were designed to isolate single neurons and were not ideally structured for multiunit recordings. Second, there is evidence that neighboring IT neurons may share some response properties (Fujita et al., 1992; Erickson et al., 2000) and we specifically chose electrode positioning based on the response pattern of one neuron. Given this, it is likely that we are underestimating the neural response difference across distractor conditions for the entire IT population.
Discussion
We recorded the activity of target-selective IT neurons and observed stronger response modulation to parafoveal targets when those targets were embedded among familiar distractors. This was true during both the initial transient response to the onset of the search array, as well as during self-guided, active exploration for the target. This neural difference was not dependent on the behavioral effects of distractor familiarity or the distribution of spatial attention related to eye movements. Additionally, there was no difference across distractor type when the target was an ineffective stimulus, indicating that the firing rate differences for effective targets were not directly induced by the distractors. Rather, target-selective IT neurons responded effectively to specific peripheral targets in the presence of familiar distractors. This enhanced processing efficiency was also reflected in behavioral measures; response times were shorter and fewer fixations were made when distractors were familiar. Furthermore, variability in the size of the neural and behavioral effects was correlated across experimental session.
One might wonder if the observed effect of distractor familiarity counteracts the effects of clutter from the 30 distractors. In other words, is a highly familiar but cluttered context equivalent to an isolated target condition? Although we did not include such a condition in the present study, a comparison of our results with a previous study (Sheinberg and Logothetis, 2001) suggests that this is not the case (supplemental text, supplemental Fig. 2, available at www.jneurosci.org, as supplemental material).
How does long-term experience with the distractor images lead to the neural effects we observed during visual search? We propose that experience-dependent changes in the ventral visual stream lead to a higher signal-to-noise ratio for target representations when the distractors are familiar (Fig. 10). In other words, the target is more salient in familiar contexts than in unfamiliar contexts.
Figure 10.
Basic diagram of the experience-dependent changes in the response properties of IT neurons accounting for the distractor familiarity effect. A, B, The activity in a hypothetical population of IT neurons is shown during visual search through unfamiliar (A) and familiar distractors (B). Active neurons are denoted in black. Local inhibitory neurons are drawn as circles; output neurons are drawn as hexagons. T denotes a target-selective output neuron and D denotes distractor-selective output neurons. The wider the line connecting a neuron and its target, the stronger the connection. Long-term experience with a distractor set leads to (1) sparser activity in IT for both local inhibitory neurons and distractor-selective output neurons, (2) less inhibitory influence on the output neurons of IT, and (3) a stronger output for target-selective neurons. The end result is a higher signal-to-noise ratio for target representations compared with familiar distractor representations. Note that changes in other areas, such as the input to IT from “lower” visual areas, are also likely to occur, but are not depicted here.
Consistent with previous reports (Miller et al., 1991; Li et al., 1993; Xiang and Brown, 1998; Freedman et al., 2006) (but see, Kobatake et al., 1998), our MUA analyses indicate that, in general, unfamiliar distractors elicit more neural activity across IT. This is in contrast, but not incompatible with our results for target-selective neurons, which were more robustly activated in the presence of familiar distractors. It is likely that the strong MUA during unfamiliar distractor trials reflects neural activity that interfered with target processing. Although it is difficult to assess the properties of neurons contributing to the MUA signal, both excitatory and inhibitory neurons could cause such interference. Inhibitory neurons make up ∼25% of monkey temporal cortex (Hendry et al., 1987), contribute mainly to local-circuitry (Tanigawa et al., 1998), and are known to affect neural response properties of stimulus-selective IT neurons (Wang et al., 2000). It is likely that the target-selective neurons we recorded were subject to more inhibitory influence in the presence of unfamiliar distractors, when MUA activity was stronger. This is consistent with the decreased response in target-selective neurons to peripheral targets during search through unfamiliar distractor arrays. An increase in the activity of local excitatory neurons could also affect target processing by decreasing the signal-to-noise ratio for target representations for downstream neurons.
Previous studies suggest that familiar distractors evoke less MUA activity for multiple reasons. For example, IT neurons display sharper stimulus selectivity for familiar items (Kobatake et al., 1998; Freedman et al., 2006). Sharper selectivity suggests that fewer neurons participate in the representation of familiar distractors compared with unfamiliar distractors. In addition, IT neurons represent stimulus saliency; learning to ignore a stimulus leads to a decrease in the neural response to that stimulus (Jagadeesh et al., 2001). In our experiment, monkeys may have learned to ignore the familiar distractors but did not have that opportunity for unfamiliar distractors. Both sharpened selectivity and salience-related response decreases could have contributed to the decrease in distractor-related activity during search through familiar arrays. To reiterate, this decrease in activity leads to less interference with target-selective neural responses, making search more efficient.
In addition to changes in local circuitry within IT, it is probable that experience-dependent plasticity altered bottom-up inputs to IT, such as those from area V4. The output of V4 neurons is more informative after learning to discriminate pictures under noisy conditions (Rainer et al., 2004). Indeed, we observed stronger responses to the target in the context of familiar distractors even during the first fixation after the onset of the search array, which is presumably driven primarily by feedforward inputs.
The neural and behavioral effects of distractor familiarity are consistent with the role of IT in the perception of visual objects. Activity in stimulus-selective IT neurons has been linked to the conscious perception of visual objects (Sheinberg and Logothetis, 1997; Mruczek and Sheinberg, 2007). Particularly relevant to the present study, peripheral target identification is tightly linked to the modulation of target-selective IT neurons (Mruczek and Sheinberg, 2007). We found the largest firing-rate differences across distractor type for eccentric targets. Thus, in familiar contexts, peripheral targets can more effectively modulate the firing rates of target-selective neurons, leading to faster detection of those targets. This can be thought of as an increase in the functional receptive field size of IT neurons when the surrounding context is highly familiar.
To ensure that the response of our IT neurons could be attributed to the effective target, and not directly to the distractors themselves, it was necessary to use a strict image selection process. Of course, the real world is not tailored to our specific neuron population and it is likely that a single IT neuron that is selective for a particular target (e.g., the silverware) will also respond to a small percentage of the distractors, both familiar and unfamiliar. However, it is highly unlikely that all of the silverware-selective IT neurons will respond to the same distractors. Distractor-driven responses will add noise to, but not completely disrupt target-selective responses across the entire IT neuron population. Thus, we believe that our results generalize to more natural conditions.
Our results support the hypothesis (Greene and Rayner, 2001) that the visual system has a wider processing span when confronted with familiar items, accounting for the behavioral effects of distractor familiarity. Familiarity with a context leads to an increased signal-to-noise ratio for object representations, which makes computations more efficient in the ventral visual stream. Furthermore, our results emphasize that the response profile of a neuron is affected by not only the physical attributes of a stimulus, but also an individual's experience with that stimulus, as well as the context in which that stimulus appears.
Footnotes
This work was supported by the James S. McDonnell Foundation, National Institutes of Health Grant RO1-EY014681, and National Science Foundation Grant SBE-0542013. We thank B. Anderson, D. Berson, J. DiCarlo, M. Paradiso, and anonymous reviewers for helpful comments on this manuscript.
References
- Baker et al., 2002.Baker CI, Behrmann M, Olson CR. Impact of learning on representation of parts and wholes in monkey inferotemporal cortex. Nat Neurosci. 2002;5:1210–1216. doi: 10.1038/nn960. [DOI] [PubMed] [Google Scholar]
- Brincat and Connor, 2004.Brincat SL, Connor CE. Underlying principles of visual shape selectivity in posterior inferotemporal cortex. Nat Neurosci. 2004;7:880–886. doi: 10.1038/nn1278. [DOI] [PubMed] [Google Scholar]
- DeAngelis and Newsome, 1999.DeAngelis GC, Newsome WT. Organization of disparity-selective neurons in macaque area MT. J Neurosci. 1999;19:1398–1415. doi: 10.1523/JNEUROSCI.19-04-01398.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
- DeAngelis et al., 1998.DeAngelis GC, Cumming BG, Newsome WT. Cortical area MT and the perception of stereoscopic depth. Nature. 1998;394:677–680. doi: 10.1038/29299. [DOI] [PubMed] [Google Scholar]
- Deubel and Schneider, 1996.Deubel H, Schneider WX. Saccade target selection and object recognition: evidence for a common attentional mechanism. Vision Res. 1996;36:1827–1837. doi: 10.1016/0042-6989(95)00294-4. [DOI] [PubMed] [Google Scholar]
- Erickson et al., 2000.Erickson CA, Jagadeesh B, Desimone R. Clustering of perirhinal neurons with similar properties following visual experience in adult monkeys. Nat Neurosci. 2000;3:1143–1148. doi: 10.1038/80664. [DOI] [PubMed] [Google Scholar]
- Fischer and Boch, 1981.Fischer B, Boch R. Selection of visual targets activates prelunate cortical cells in trained rhesus monkey. Exp Brain Res. 1981;41:431–433. doi: 10.1007/BF00238903. [DOI] [PubMed] [Google Scholar]
- Freedman et al., 2006.Freedman DJ, Riesenhuber M, Poggio T, Miller EK. Experience-dependent sharpening of visual shape selectivity in inferior temporal cortex. Cereb Cortex. 2006;16:1631–1644. doi: 10.1093/cercor/bhj100. [DOI] [PubMed] [Google Scholar]
- Fujita et al., 1992.Fujita I, Tanaka K, Ito M, Cheng K. Columns for visual features of objects in monkey inferotemporal cortex. Nature. 1992;360:343–346. doi: 10.1038/360343a0. [DOI] [PubMed] [Google Scholar]
- Goodale and Milner, 1992.Goodale MA, Milner AD. Separate visual pathways for perception and action. Trends Neurosci. 1992;15:20–25. doi: 10.1016/0166-2236(92)90344-8. [DOI] [PubMed] [Google Scholar]
- Greene and Rayner, 2001.Greene HH, Rayner K. Eye movements and familiarity effects in visual search. Vision Res. 2001;41:3763–3773. doi: 10.1016/s0042-6989(01)00154-7. [DOI] [PubMed] [Google Scholar]
- Hendry et al., 1987.Hendry S, Schwark H, Jones E, Yan J. Numbers and proportions of GABA-immunoreactive neurons in different areas of monkey cerebral cortex. J Neurosci. 1987;7:1503–1519. doi: 10.1523/JNEUROSCI.07-05-01503.1987. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hoffman and Subramaniam, 1995.Hoffman JE, Subramaniam B. The role of visual attention in saccadic eye movements. Percept Psychophys. 1995;57:787–795. doi: 10.3758/bf03206794. [DOI] [PubMed] [Google Scholar]
- Jagadeesh et al., 2001.Jagadeesh B, Chelazzi L, Mishkin M, Desimone R. Learning increases stimulus salience in anterior inferior temporal cortex of the macaque. J Neurophysiol. 2001;86:290–303. doi: 10.1152/jn.2001.86.1.290. [DOI] [PubMed] [Google Scholar]
- Kobatake et al., 1998.Kobatake E, Wang G, Tanaka K. Effects of shape-discrimination training on the selectivity of inferotemporal cells in adult monkeys. J Neurophysiol. 1998;80:324–330. doi: 10.1152/jn.1998.80.1.324. [DOI] [PubMed] [Google Scholar]
- Kourtzi and DiCarlo, 2006.Kourtzi Z, DiCarlo JJ. Learning and neural plasticity in visual object recognition. Curr Opin Neurobiol. 2006;16:152–158. doi: 10.1016/j.conb.2006.03.012. [DOI] [PubMed] [Google Scholar]
- Kowler et al., 1995.Kowler E, Anderson E, Dosher B, Blaser E. The role of attention in the programming of saccades. Vision Res. 1995;35:1897–1916. doi: 10.1016/0042-6989(94)00279-u. [DOI] [PubMed] [Google Scholar]
- Li et al., 1993.Li L, Miller EK, Desimone R. The representation of stimulus familiarity in anterior inferior temporal cortex. J Neurophysiol. 1993;69:1918–1929. doi: 10.1152/jn.1993.69.6.1918. [DOI] [PubMed] [Google Scholar]
- Logothetis and Sheinberg, 1996.Logothetis NK, Sheinberg DL. Visual object recognition. Annu Rev Neurosci. 1996;19:577–621. doi: 10.1146/annurev.ne.19.030196.003045. [DOI] [PubMed] [Google Scholar]
- Logothetis et al., 1995.Logothetis NK, Pauls J, Poggio T. Shape representation in the inferior temporal cortex of monkeys. Curr Biol. 1995;5:552–563. doi: 10.1016/s0960-9822(95)00108-4. [DOI] [PubMed] [Google Scholar]
- Malinowski and Hübner, 2001.Malinowski P, Hübner R. The effect of familiarity on visual-search performance: evidence for learned basic features. Percept Psychophys. 2001;63:458–463. doi: 10.3758/bf03194412. [DOI] [PubMed] [Google Scholar]
- Miller et al., 1991.Miller EK, Li L, Desimone R. A neural mechanism for working and recognition memory in inferior temporal cortex. Science. 1991;254:1377–1379. doi: 10.1126/science.1962197. [DOI] [PubMed] [Google Scholar]
- Moore et al., 1998.Moore T, Tolias AS, Schiller PH. Visual representations during saccadic eye movements. Proc Natl Acad Sci USA. 1998;95:8981–8984. doi: 10.1073/pnas.95.15.8981. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mruczek and Sheinberg, 2005.Mruczek RE, Sheinberg DL. Distractor familiarity leads to more efficient visual search for complex stimuli. Percept Psychophys. 2005;67:1016–1031. doi: 10.3758/bf03193628. [DOI] [PubMed] [Google Scholar]
- Mruczek and Sheinberg, 2007.Mruczek RE, Sheinberg DL. Activity of inferior temporal cortical neurons predicts recognition choice behavior and recognition time during visual search. J Neurosci. 2007;27:2825–2836. doi: 10.1523/JNEUROSCI.4102-06.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rainer et al., 2004.Rainer G, Lee H, Logothetis NK. The effect of learning on the function of monkey extrastriate visual cortex. PLoS Biol. 2004;2:E44. doi: 10.1371/journal.pbio.0020044. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rolls et al., 2003.Rolls ET, Aggelopoulos NC, Zheng F. The receptive fields of inferior temporal cortex neurons in natural scenes. J Neurosci. 2003;23:339–348. doi: 10.1523/JNEUROSCI.23-01-00339.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sakai and Miyashita, 1991.Sakai K, Miyashita Y. Neural organization for the long-term memory of paired associates. Nature. 1991;354:152–155. doi: 10.1038/354152a0. [DOI] [PubMed] [Google Scholar]
- Sheinberg and Logothetis, 1997.Sheinberg DL, Logothetis NK. The role of temporal cortical areas in perceptual organization. Proc Natl Acad Sci USA. 1997;94:3408–3413. doi: 10.1073/pnas.94.7.3408. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sheinberg and Logothetis, 2001.Sheinberg DL, Logothetis NK. Noticing familiar objects in real world scenes: the role of temporal cortical neurons in natural vision. J Neurosci. 2001;21:1340–1350. doi: 10.1523/JNEUROSCI.21-04-01340.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shen and Reingold, 2001.Shen J, Reingold EM. Visual search asymmetry: the influence of stimulus familiarity and low-level features. Percept Psychophys. 2001;63:464–475. doi: 10.3758/bf03194413. [DOI] [PubMed] [Google Scholar]
- Sigala and Logothetis, 2002.Sigala N, Logothetis NK. Visual categorization shapes feature selectivity in the primate temporal cortex. Nature. 2002;415:318–320. doi: 10.1038/415318a. [DOI] [PubMed] [Google Scholar]
- Tanaka, 1996.Tanaka K. Inferotemporal cortex and object vision. Annu Rev Neurosci. 1996;19:109–139. doi: 10.1146/annurev.ne.19.030196.000545. [DOI] [PubMed] [Google Scholar]
- Tanigawa et al., 1998.Tanigawa H, Fujita I, Kato M, Ojima H. Distribution, morphology, and-aminobutyric acid immunoreactivity of horizontally projecting neurons in the macaque inferior temporal cortex. J Comp Neurol. 1998;401:129–143. doi: 10.1002/(sici)1096-9861(19981109)401:1<129::aid-cne8>3.0.co;2-d. [DOI] [PubMed] [Google Scholar]
- Thompson et al., 1996.Thompson KG, Hanes DP, Bichot NP, Schall JD. Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. J Neurophysiol. 1996;76:4040–4055. doi: 10.1152/jn.1996.76.6.4040. [DOI] [PubMed] [Google Scholar]
- Ungerleider and Mishkin, 1982.Ungerleider LG, Mishkin M. Two cortical visual systems. In: Ingle DJ, Goodale MA, Mansfield RJW, editors. Analysis of visual behavior. Cambridge: MA: MIT; 1982. pp. 549–586. [Google Scholar]
- Wang et al., 2000.Wang Y, Fujita I, Murayama Y. Neuronal mechanisms of selectivity for object features revealed by blocking inhibition in inferotemporal cortex. Nat Neurosci. 2000;3:807–813. doi: 10.1038/77712. [DOI] [PubMed] [Google Scholar]
- Xiang and Brown, 1998.Xiang JZ, Brown MW. Differential neuronal encoding of novelty, familiarity and recency in regions of the anterior temporal lobe. Neuropharmacology. 1998;37:657–676. doi: 10.1016/s0028-3908(98)00030-6. [DOI] [PubMed] [Google Scholar]