Orientation saliency without visual cortex and target selection in archer fish

Abstract

Our visual attention is attracted by salient stimuli in our environment and affected by primitive features such as orientation, color, and motion. Perceptual saliency due to orientation contrast has been extensively demonstrated in behavioral experiments with humans and other primates and is believed to be facilitated by the functional organization of the primary visual cortex. In behavioral experiments with the archer fish, a proficient hunter with remarkable visual abilities, we found an orientation saliency effect similar to that observed in human subjects. Given the enormous evolutionary distance between humans and archer fish, our findings suggest that orientation-based saliency constitutes a fundamental building block for efficient visual information processing.

The vast amount of information present in early vision necessitates the operation of selection processes whose bottom-up components are commonly referred to as saliency-based attention (1–4). Some neural correlates of saliency-based processing have been observed in animals as low as Drosophila (5). In the mammalian cortex, however, and in particular primates, these processes interact with several other cortical features, such as orientation selectivity of neurons (6), their topographical organization (6–9), and their connectivity patterns (8, 10–13), to result in a conspicuous form of visual saliency that is based on orientation (14, 15). Here, in a much simpler animal (the archer fish), we report on orientation-based visual saliency that is qualitatively similar to that observed in humans. These observations demonstrate how evolutionary pressures for efficient visual processing bring distant evolutionary paths to express similar functional solutions and suggest that orientation-based saliency constitutes a universal building block for efficient visual information processing.

In their natural environment, archer fish exhibit remarkable preying behavior (Fig. 1A). They are capable of detecting insects resting on overhanging vegetation, knocking them down with accurate streams of water sprouted from their mouth, and efficiently inferring the ballistics of the falling prey to intercept it at the predicted impact point with the water (16, 17). Recent studies have demonstrated that these fish can accurately shoot flying insects as well (18), an ability possibly facilitated by their human-like pattern of eye movements (19). Here we ask whether on top of these outstanding capacities, archer fish also exhibit such visual behaviors that are typically associated with the presence of a visual cortex, which these fish lack.

Fig. 1.

Archer fish and our experimental setup. (A) The archer fish shoots down its prey by accurate jets of water (photograph courtesy of Alan Parker). In the laboratory, this remarkable capacity can be adapted for hitting targets on a computer screen. (B) One shooting frame from our experiment. (See Movies S1 and S2 for sample experimental trials.)

Numerous behavioral studies in humans and primates have convincingly demonstrated saliency effects related to visual features such as color, motion, and orientation, through experimental configurations designed to test the so-called “pop out” effect (1, 3, 20). When subjects are presented with an array of lines, all but one having the same orientation, the singleton that differs in its orientation is readily detected, and the time required for detection remains constant irrespective of the number of distracters (3, 21). Directly related to this orientation-based saliency in visual search tasks is the phenomenon of orientation-based texture segregation, whereby a salient figure texture is quickly and effortlessly segregated from the textured background because of an orientation contrast along its boundaries (22–25). Because all of these interrelated behavioral phenomena are repeatedly linked to functional and anatomical properties of the mammalian visual cortex (26–29), it is not immediately clear whether low-level animals like archer fish could also exhibit them. However, while hunting for prey, archer fish are subjected to heavy competition from other predators, as well as from other school members (16, 17). To benefit from fast targeting of prey, archer fish should therefore exploit every possible visual cue available. If the orientation of the prey and that of the texture behind it are selected independently, they would be generically transverse, giving rise to an orientation contrast that could be used as a robust target detection cue. For animals that crucially rely on efficient visual processing for their survival, this could introduce an evolutionary pressure toward orientation-based visual saliency processing.

Results

To test orientation-based saliency in archer fish, we adapted the figure–ground segregation paradigm to accommodate for the fish behavioral constraints. Their task was to shoot one of two oriented target bars displayed on orientation-defined textured backgrounds (Fig. 2). Our hypothesis was that if the fish possess orientation-based saliency mechanisms, targets whose orientation is incongruent (i.e., orthogonal) to the background's orientation would appear more salient to them, resulting in higher probability of selection compared with bars whose orientation is congruent (i.e., parallel) to the background.

Fig. 2.

Stimuli and experimental design. (A) Stimuli consisted of a circular textured background with fading margins on which semitransparent target bars were superimposed at two adjacent corners of a rotated virtual square (see Materials and Methods). This virtual square is depicted here (as a dotted yellow diamond), but it was not part of the presented stimuli. For clarity, in this figure the contrast between the target bars and the background is twice the one used in the archer fish stimuli and four times larger than the one used in the rapid display experiments with human subjects. (B) Luminance contrast of congruent and incongruent target bars. To obtain a luminance contrast measurement of the stimuli used in our experiments, we first sampled multiple bar targets and target-size regions adjacent them. We then computed the average target–background contrast from the average luminance in these regions via , where T is the average intensity inside the target, and B is the average intensity in the adjacent region. Shown here is an example of one sampled region adjacent to an incongruent target (red) and one sampled region adjacent to a congruent target. Sixteen such samples were used to test for statistical differences in the mean contrast of congruent and incongruent targets and confirmed that no such statistically significant differences exist (P > 0.73, n = 8, t test). (C) The three dimensions that determine a particular stimulus. First column represents the three types of texture backgrounds, namely horizontally oriented, vertically oriented, or isotropic. On top of each textured background the target bars could be positioned in one of four position pairs as depicted in the second column. At each position pair, the bars could be oriented in four different combinations as shown in the third column. Together, these summed up to 48 possible stimuli. (D) Two sample stimuli. The stimulus in D1 is obtained by choosing from each dimension the element marked in blue, whereas the one in D2 is obtained from the elements marked in red. (E) Temporal procedure in the fish experiment. Each trial began with a presentation a circular textured area (for simplicity, the fading margins are not depicted in these icons). Immediately after the fish started orienting itself toward the screen, the partially transparent bar targets (shown here as opaque bars for presentation clarity) were displayed. The targets were removed after 2 s, and after an additional 2 s the texture was removed as well.

The experimental setup consisted of a liquid crystal display (LCD) screen that was positioned 35 cm above and parallel to water level (Fig. 1B). On each experimental trial, the fish was presented first with one of three possible background types: vertically oriented, horizontally oriented, or isotropic (Fig. 2). Immediately after the fish changed its body position toward the screen, two low-contrast bars were presented on top of the textured area for 2 s, during which the fish was expected to detect the targets, select one and localize it in space, optimize its body position, and shoot. For each successful shot (i.e., when the fish accurately shot one of the two targets rather than at arbitrary nontarget screen position or between targets) the fish was rewarded for its accuracy, irrespective of which target it chose.

Each of the two targets could appear at one of four possible locations in the textured area, as defined by the corners of a virtual square rotated 45° relative to the screen boundaries (Fig. 2 A and C). To keep the distance between targets fixed across trials, they were always set at two adjacent corners of the virtual square (≈12.8° apart). The orientation of the two bars was set to horizontal or vertical, independently of each other and of the background type. Hence, the experimental conditions were three dimensional, whereby the four bar pair positions, three backgrounds, and four combinations of bar orientations totaled 48 possible types of stimuli.

For the purpose of statistical analysis, we have divided the 48 types of trials into three categories. The first category, the test trials, was made of trials in which the stimuli had an oriented background (either vertically or horizontally) and the two target bars had different orientations (i.e., one vertical and one horizontal) (e.g., Fig. 2D1). This group consisted of the critical condition in our experiment.

The second category, the prior trials, was made of trials in which the background texture was isotropic and the two targets had different orientations. Having nonoriented background, these control trials facilitate the estimation of any internal bias of the fish to shoot a horizontal or vertical bar regardless of any orientation saliency cues. As discussed below, these trials were used for estimating the baseline hypothetical probabilities in the binomial analysis of the test trials.

A third category of trials, the control trials, consisted of all trials in which both targets had the same orientation regardless of the type of background texture (e.g., Fig. 2D2). These were used to control for possible confounding effects due to interactions between the different types of backgrounds and bar positions. As described in Materials and Methods, our analysis has excluded such sources of interactions.

A total of five archer fish (Toxotes jaculatrix) participated in the experiment, each performing 4 ≤ k ≤ 8 repeats of all 48 trial types, for a total 192–384 trials per subject. Trials were ordered randomly and divided into daily sessions of 6–40 trials in accordance with the feeding constraints of each animal. Trials in which the fish did not respond within the 2-s window and trials with response whose target could not be resolved unequivocally (25.34% of the trials) were excluded from the analysis. The results for the two main categories of trials, the test trials and the prior trials, are summarized in Fig. 3. The first evident finding is that all fish (individually and on average) exhibited clear preference to hit the target that contrasted in orientation with the background, as would be predicted if the vision of the archer fish was affected by orientation saliency.

Fig. 3.

Responses of archer fish to the critical and control conditions (test and prior trials). Shown here are results of individual fish and the pooled results for all five fish. Blue bars depict the probability of the fish to hit a vertical target, whereas red bars depict the probability to hit a horizontal target, both broken down by background texture type. Error bars represent SEM. Note the tendency of all fish to prefer the target whose orientation is incongruent with the orientation of the background. Gray asterisks denote the conditions for which a significant effect is observed (P < 0.05) when assuming an equal a priori probability of the fish to hit the vertical or horizontal bar (i.e., assuming q = 0.5). The effect is highly significant (P < 10⁻¹²) for the pooled results. The results for isotropic background represent an internal bias of the fish toward certain target orientation. Black asterisks denote the conditions for which a significant effect is observed (P < 0.05) even after this bias is discounted. When results from all fish are pooled together the effect is again highly significant (P < 10⁻⁴) for both background orientations.

For statistical significance of the results we used the binomial test to estimate the probability P to observe our orientation saliency results by chance, while assuming that the fish has hypothetical probabilities of success q of hitting any of the particular bar orientations (i.e., horizontal vs. vertical targets, regardless of background orientation). First we analyzed the statistical significance of our results assuming a hypothetical probability of q = 0.5, that is, assuming the fish have no internal bias for either the horizontal or vertical targets. The resulting P values of this analysis are statistically significant for individual fish and are highly significant when they are pooled together (P < 10⁻¹², binomial test; Fig. 3). We then repeated this analysis while using the prior trials to estimate the baseline hypothetical probability of success q of hitting a horizontal or vertical target when no orientation saliency cues are present. Although on average no significant preference has been observed for either target orientation (P > 0.29), at least some of our fish subjects (e.g., fish 4; see Fig. 3) exhibited some tendency to prefer one target orientation when presented on isotropic background. As shown in Fig. 3, even under this stricter analysis, each fish exhibited a significant saliency effect for at least one of the two critical conditions, and the effect (70.35% vs. 29.65%) is highly significant for both saliency conditions (P < 10⁻⁸ and P < 10⁻⁴ for horizontally and vertically oriented backgrounds, respectively, binomial test) when all trials are pooled together.

Our experiments of orientation-based saliency in the archer fish were motivated by known findings in human observers. However, the stimuli used in psychophysical experiments with humans are typically different from those used in our experiment, which raises the possibility that the results in the two species are stimuli dependent. To exclude this possibility we therefore tested orientation-based saliency in human observers using the exact stimuli from our fish experiments. To adjust it for human subjects, however, we replaced the selection task of the fish with a rapid detection task. At each trial subjects were presented with one of the three background textures, with either two, one, or no target bar already superimposed. Five subjects participated in this experiment and were simply asked to report how many targets were observed in each of these rapid displays. Response accuracy was significantly higher for targets incongruent with the background compared with congruent ones, in both single-bar trials (0.74 vs. 0.39, P < 10⁻¹³, binomial test) and presentations of two bars of similar orientation (0.94 vs. 0.44, P < 10⁻¹⁵).

To probe this general effect further, we have also applied isolated analyses to correct trials only (i.e., trials in which subjects correctly detected the number of presented bars). For each background type and target orientation, the number of corresponding single-target correct trials were divided by the total number of correct trials having the same background type (i.e., regardless of target orientation). The resultant ratios, termed here the normalized response accuracy, could readily indicate any bias toward incongruent vs. congruent targets and are directly comparable across conditions. Indeed, the results plotted in Fig. 4A show significant advantage for targets incongruent with the background compared with congruent targets (0.69 vs. 0.31, P < 10⁻⁵, binomial test). Similar analysis and results are obtained for correct trials with two targets of similar orientation (0.66 vs. 0.34, P < 10⁻⁴, binomial test; Fig. 4B).

Fig. 4.

Results with human subjects. (A) Normalized accuracy of detecting a single target, shown for each type of background. Results with isotropic background represent observers' intrinsic bias for certain target orientation (in this case, horizontal). Even after accounting for this bias (applying a binomial test with hypothetical probabilities of success being estimated from trials with isotropic background), target detection on oriented background exhibits strong orientation saliency effect. *P < 10⁻⁵, binomial test. (B) Results of trials with two targets having the same orientation. Again, a strong saliency effect is observed, in full correspondence to the archer fish results. *P < 10⁻⁴, binomial test. (C) Detection error rates in trials with a single target presentation, broken down by background type. By depicting the probability of subjects to wrongfully report that no bar was presented, this plot shows that the accuracy drops drastically when the background is oriented compared with the baseline with isotropic background. *P < 10⁻¹⁰, binomial test. (D) Detection error rates in trials with two targets of different orientations. Results denote subjects’ probability to incorrectly report a single bar. Note how the accuracy is significantly reduced when the background is oriented compared with the baseline with isotropic background. *P < 10⁻¹⁰, binomial test. Considered together, the results in all these panels suggest a strong saliency effect for the incongruent target compared with the congruent one.

Finally, to get additional insights, we also analyzed incorrect trials (i.e., trials in which subjects reported incorrect number of targets). As shown in Fig. 4C, in single-target incorrect trials, subjects’ probability to report no bar was significantly higher when the target was congruent to the background (P < 10⁻¹⁰, binomial test). A similar effect was observed for two targets of similar orientations (0.56 vs. 0.056 error probabilities, P < 10⁻¹⁰, binomial test). Correspondingly, in incorrect trials with two targets of different orientations, the probability to report only one target was significantly higher for oriented backgrounds compared with isotropic ones (0.55/0.49 for vertically/horizontally oriented backgrounds vs. 0.09 for isotropic background, P < 10⁻¹⁰, binomial test; Fig. 4D). All these results strongly support the same orientation saliency effect and suggest that incongruent targets are significantly more salient to human observers than congruent ones.

Discussion

By using identical stimuli both for archer fish and human subjects, all of the above results draw a direct behavioral analogy between our unique orientation-based saliency findings in archer fish and those well known in humans. As mentioned above, however, in humans this capacity is attributed to mechanisms found in the mammalian visual cortex, and in particular to orientation selectivity and intracortical horizontal connectivity. Hence, given the substantial differences in brain structure of these two distant species, the underlying neural mechanism that facilitates orientation-based saliency in archer fish becomes an intriguing open question. Indeed, a handful of studies have found that orientation selectivity of single neurons is not confined to mammals but is also present in birds, at least in some limited form (30, 31). Some sparse evidence implies that goldfish may also possess orientation-selective units (32), and behavioral experiments suggest that it may be trained to discriminate between oriented patterns (33). Still, even if neural mechanisms in archer fish are similar to those in goldfish, the evident gap to mammalian-like cortical mechanisms excludes an easy answer. The scarcity of reported orientation-selective cells in goldfish (3 of 113 recorded units) and their huge (≈30°) receptive fields (32) are unlikely to facilitate a computation of orientation contrasts as those defined by the microtextures (of high spatial frequency) that were used in our experiments. Furthermore, whereas previous discrimination results (33) may be explained by plastic changes that could be introduced to the underlying neural circuits via extensive training to particular target patterns, no such changes can explain our results because we did not train the fish for specific targets. Because both bar orientations could serve as the salient targets in our experiments, the results are more likely to emerge from preexisting neural architecture.

Another intriguing question raised by our findings is the implications of the evolutionary relationship between orientation saliency in the two species investigated. If orientation saliency mechanisms in archer fish and humans are homologous (i.e., derived from common ancestry), then the fact that this functionality has been preserved for so long (34) implies that these mechanisms are of high functional importance and that no better alternatives have been found during the course of evolution. This further suggests orientation saliency as a fundamental building block for visual representations and efficient visual processing. Similarly, if orientation saliency mechanisms in archer fish are analogous to those of humans (i.e., reflect independently convergent evolutionary processes), it would strongly support the notion that orientation saliency has computational optimality in a wide variety of contexts. Hence, both evolutionary alternatives suggest that orientation-based saliency constitutes a universal building block for efficient visual information processing.

In humans, the perceptual pop out effect of salient objects is associated (in fact, by definition) with parallel processing of the visual content. Evidently, our orientation saliency findings do not necessarily entail the existence of such parallel processes in the archer fish, nor do they eliminate this possibility. Although these more specific questions await future research, at the very least we hypothesize that, as in primates and other mammals, orientation-selective units are expected to exist in the archer fish as well. However, our findings also suggest that visual processes like orientation-based saliency may not necessarily require the elaborate cortical structures that are typically associated with it. Hence, unveiling the neural mechanisms that facilitate these processes in low-levels animals like archer fish may bring important insights about saliency processing in higher organisms as well. In addition, the fact that such saliency mechanism is accessible to behavioral experimentation in a relatively simple organism suggests the archer fish as a promising animal model for the study of visual attention in general.

Materials and Methods

Fish Training.

Fish were trained gradually to respond to low-contrast bars on textured background presented on an LCD screen. Each of the five fish subjects was housed in separate water tank 30 × 50 × 40 cm in size. Water level was set at 22 cm. Fish were first trained to shoot insect images (instead of real insects) and were rewarded with a food pellet for each successful shot (i.e., when the fish accurately shot a target rather than an arbitrary nontarget screen position). After the fish was trained to hit an image of insect, it was replaced with an image of an elongated (5.3° × 0.53°) opaque dark bar presented at arbitrary locations and orientations on brighter background. Later, the dark target bar was superimposed on a textured background as soon as the fish would orient its body toward the screen. Next, we gradually reduced the contrast of the bar by increasing its transparency. At the last training stage the target was presented for shorter durations. Training was completed when the fish could successfully hit the low-contrast bar within 2 s on at least one third of the training trials. At no stage during this training procedure were the fish exposed and trained to multitarget configurations (and in particular, to the critical condition that was tested in the experiment).

Stimuli and Procedures.

In all experiments, both with human and archerfish subjects, we used the same type of stimuli. Isotropic (i.e., nonoriented) background textures were based on white-noise images. The oriented background textures were constructed by application of the line integral convolution method (35) on a white-noise seed image, with either vertical or horizontal vector field of constant magnitude. To prevent any margin artifacts, however, all textures were faded gradually to the black background.

Experiments with fish subjects were carried out in a fluorescent-illuminated room, and the stimuli were presented on an LCD screen placed on top of a transparent glass plate 35 cm above water surface. Experiments with human subjects were carried out in a dark room, and stimuli were presented on a cathode ray tube monitor 100 cm from the subjects. Average local luminance was constant and equal for all textures (16.8 cd/m² in fish experiments, 31.9 cd/m² in human experiments). The circularly fading texture stimuli were ≈48° in diameter in the fish experiments and ≈17° in the human experiments. In both cases, the entire stimulus could easily fit in the observer's field of view (36, 37).

Target bars were superimposed on the textures as if they were semitransparent black rectangles. Being transparent, the target bars were textured similarly to their surroundings but had a lower average luminance defined by the degree of transparency. The transparency for both the fish and human experiments needed to be large enough to avoid saturation in detection performance, and on the basis of a pilot study these contrasts were set to 0.24 and 0.12, respectively (contrast was computed by , where B and T represent the average local luminance of the background and targets, respectively). Each target bar was ≈5.3° × 0.53° for the fish and 1.8° × 0.18° for human subjects, and target pairs could appear at any two adjacent corners of a virtual square rotated 45° relative to the screen boundaries (see yellow diamond in Fig. 2A). The distance between target centers was therefore defined by the length of the virtual square's side, being ≈12.8° in the fish experiments and ≈4.3° in the human experiments. By design, no statistical differences should exist in the local contrasts of congruent and incongruent target bars, although this was also verified experimentally as described in Fig. 2B. In addition, we have excluded the possibility that congruent and incongruent targets could be discriminated by the response of archer fish retinal ganglion cells. Receptive fields of such cells were modeled according to their reported functional properties (19), and their response to our stimuli was simulated computationally. Indeed, no statistical differences between responses to congruent and incongruent target bars were found at that level.

Fish Data Analysis.

The control trials, in which both target bars had the same orientation, were analyzed for possible interactions between the type of textured background, bar position, and bar orientation. Given the identical target orientation, any statistical effect observed here should be attributed solely to the background type or bar position. If found, such effect must be taken into account in the analysis of target selection in the test trials. However, the results excluded any confounds from an interaction between texture type, bar orientation, and bar position (P > 0.22, P > 0.56, P > 0.35, P > 0.87, df = 2, ANOVA).

Additional Methods for Human Experiments.

All five subjects were graduate students, all having normal or corrected-to-normal vision, and all naïve to the purpose of the experiment. Each trial began with the presentation of a red fixation dot in the center of a gray circular mask (≈17° in diameter) with average luminance equal to those of the textured backgrounds. One second after the fixation cue the display was replaced with the stimulus, which comprised the circular background texture (≈17° in diameter) with two, one, or no target bar already superimposed on it. Hence, in addition to the original 16 target pair conditions (4 pair positions × 4 combinations of bar orientations), an additional 8 single-bar and 1 no-bar trial types were added for each background condition. Presentation durations were much shorter (150 ms), bar contrast was half of that used in the fish experiments, and stimulus was followed by a circular mask similar to the one that preceded it (although this time without the fixation mark). At this point the subject was prompted to specify how many targets were present (by pressing the appropriate numeral key), and auditory feedback about the correctness of the response was provided immediately after. The number of targets presented were counterbalanced (each condition occurring on one third of the trials, to prevent any bias in subjects response), and the different trials were ordered randomly through the 576-trials experiment. Stimuli incorporating two targets were identical to those used in the fish experiments (except for the target–background contrast, which was lower).

Human Experiment Analysis.

We observed an advantage for horizontal targets compared with vertical ones, when these are presented on isotropic textures. To account for this bias, we applied a binomial test with the bias probabilities estimated from trials of isotropic texture. In addition, general detection accuracy was smaller for oriented backgrounds (63%) compared with isotropic ones (82%). Therefore, we normalized the results for each condition separately, by dividing the number of correct vertical/horizontal bar detections by the total number (vertical + horizontal) correct detections.