Efficient Allocation of Attentional Sensitivity Gain in Visual Cortex Reduces Foveal Sensitivity in Visual Search

SUMMARY

The human visual system has a high-resolution fovea and a low-resolution periphery. When actively searching for a target, humans perform covert search during each fixation, and then shift fixation (the fovea) to probable target locations. Previous studies of covert search under carefully-controlled conditions provide strong evidence that, for simple and small search displays, humans process all potential target locations with the same efficiency they process those locations when individually cued on each trial. Here we extend these studies to the case of large displays where the target can appear anywhere within the display. These more natural conditions reveal an attentional effect where sensitivity in the fovea and parafovea is greatly diminished. We show that this “foveal neglect,” is the expected consequence of efficiently allocating a fixed total attentional sensitivity gain across the retinotopic map in visual cortex. We present a formal theory that explains our findings and the previous findings.

Walshe and Geisler find that when humans perform covert search over a large display region, there is a striking and unexpected loss of visual sensitivity in the fovea and parafovea – foveal neglect. The authors explain how this phenomenon can be explained by the efficient allocation of attentional resources in the retinotopic map of visual cortex.

INTRODUCTION

Humans, and other primates, have a high-resolution fovea, a low-resolution periphery, and a sophisticated eye-movement system to control the fixation locations encoded by the high-resolution fovea. One of the fundamental tasks that drove the evolution of our foveated visual system is the task of visually searching for specific objects in the environment^1–4. The rigorous study of visual search is complicated because the human visual system is strongly foveated. This complexity is revealed in part by considering the major components of the visual search task represented by the flow diagram in Figure 1. The inputs in the search task (pink trapezoids) are (i) the image falling on the retina during a specific fixation, (ii) some prior knowledge about what the search target is and where it might be located, and (iii) some knowledge about one’s own foveated visual system; that is, some knowledge of how detectable the search target would be at different retinal locations—a “d’map.” During a given fixation, the image is encoded by the foveated visual system, and thus the encoding will depend on the specific fixation location. Given the encoded image, prior, and d’map, posterior beliefs are computed about whether or not the target is present in the scene, and if so, where the target might be located. If the posterior beliefs are strong enough, then the search ends with a decision that the search target is either absent or present at a specific location. If the beliefs are not yet strong enough, additional information is collected by selecting another fixation location.

Figure 1. Schematic flow diagram of the generic components of processing in a typical overt search task.

Inputs: pink trapezoids. Processing: blue rectangles. Decision: gray diamond. Responses: green ovals. In covert search, fixation selection (gray arrow path) is avoided by presenting stimuli for the duration of a typical single fixation.

For simple search displays, with proper control for the effects of the foveated visual system, the general finding is that covert search accuracy is consistent with near-optimal, unlimited-capacity, parallel processing^1,2. Specifically, as the number of potential target locations increase, the accuracy of detecting and locating the target decreases, but the magnitude of the decrease is predicted quantitatively by an “ideal searcher” with independent noise at the different potential target locations⁵. The ideal searcher processes all target locations in parallel, with the fidelity set by the foveation, independent of the number of potential target locations. Evidence for parallel, unlimited-capacity processing in covert search has also been obtained from simultaneous behavioral and neurophysiological measurements in humans⁶ and non-human primates⁷. However, for simplicity and maximizing experimental control, these studies focused on searches with a relatively small number of well-separated potential target locations. Under natural conditions, there is generally a continuous prior over potential target locations, and the backgrounds in which the target appears are often large and complex.

Here, we extend covert-search studies to large noise backgrounds with continuous priors on target location. We also extend the ideal-searcher theory to properly include the effects of foveation, intrinsic position uncertainty, and the continuous prior on target location. We find that overall performance in our experiments is largely consistent with this foveated ideal searcher, and hence with parallel, unlimited-capacity processing of the outputs from the foveated retina. However, paradoxically, we find that human error rates do not decline in the foveal region. This result is not predicted by the foveated ideal searcher, which predicts that the errors should decrease, reaching zero near the fovea.

The simplest hypothesis for this “foveal neglect” is that processing remains parallel and unlimited in capacity, but there is a cognitive bias: a tendency to misestimate the prior probability of targets appearing in and near the fovea. This bias is equivalent to applying an inappropriate distribution of selective attention. This hypothesis is not implausible given the evidence that humans are often poor at estimating probabilities⁸. However, we find that this hypothesis is inconsistent with the spatial distribution of false alarms and false hits (incorrectly localized correct responses when the target is present). Thus, the most viable hypothesis is that humans have limited, attentionally-controlled, sensitivity resources not revealed by earlier studies with only a few potential target locations.

We represent sensitivity attention as a gain on target detectability (d’), where the maximum gain occurs when attention is restricted to a single potential target location. We find that achieving near ideal search performance, while simultaneously displaying foveal neglect, is the expected consequence of efficiently allocating a fixed total attentional sensitivity gain across the retinotopic map in primary visual cortex (V1). In other words, foveal neglect is the expected consequence of optimally allocating attention across a limited number of neurons in visual cortex, rather than across a limited number of locations in visual space (the traditional assumption^2,3). Further, we find that the foveated ideal searcher, combined with this limitation in attentional sensitivity gain, predicts the detailed pattern of human search performance in two experiments with different prior probability distributions over target location.

RESULTS

We begin by describing the two covert search experiments and the overall performance measured in those experiments. In the covert search experiments, four practiced observers viewed a calibrated display while the head was kept in position with a chinrest and headrest. Each trial began with 750 ms of fixation between two small marks in the center of the circular search region (Figure 2A). The fixation marks were then extinguished. After 250 ms, a 16° diameter search field appeared for 250 ms (the approximate duration of a single fixation during overt search). On half the trials, a 6 cycle per degree (cpd) vertical sinewave target was added at a randomly selected location in the search field, which was filled with Gaussian white noise having a root-mean-squared (RMS) contrast of 20%. Following presentation of the search field, observers indicated with a button press whether the target was present or absent. If the response was “present,” they indicated the estimated target location with a mouse click. Feedback was given on each trial. Based on preliminary measurements, the contrast of the target was set so the overall accuracy of the human observers was in the neighborhood of 70% correct (target contrast = 35% Michelson).

Figure 2. Stimuli and cued detection performance.

A. Stimuli and trial sequence (for cued detection and search). Screen pixels = 60 pixels/deg. Noise and target pixels = 2×2 screen pixels (i.e., 30 pixels/deg). B. Average psychometric functions in the cued-location detection experiment. Upper panels: detectability, d’, as a function of eccentricity and target contrast in the fovea. Lower panels: same sensitivity data expressed in units of bias-corrected proportion correct. Similar results were obtained for all four observers (see Figure S1). Notice the poorer sensitivity in the superior visual field. The black circle is the average measured proportion correct in the center of the fovea (based on 600 trials in two subjects), for the target contrast in the search experiment: rcw (0.98), az (0.98). Shading in the lower panels represents 67% confidence intervals. C. Measured average d’ map for the four observers in the cued-detection experiment, after correction for the effects of intrinsic position uncertainty (see Results and Methods). Black curves are iso-d’ contours.

To measure the effect of the prior probability over target location, two experiments were run. In one, the target could appear at any location with a uniform probability (uniform sampling). In the other, the distance of the target from fixation was first sampled from a uniform probability distribution, then the direction was sampled from a uniform probability distribution (polar sampling). With polar sampling, pixel locations further from fixation have a lower prior probability of being the location of the target. All observers completed 2400 trials in each of the two experiments.

The observers’ responses were initially sorted into five categories: hits, misses, correct rejections, false alarms, and false hits. Except for false hits, these are the standard response categories for single-interval forced-choice (Yes-No) tasks⁹. False hits are trials in which the observer correctly reports that the target was present, but indicates a location not near the actual location of the target (see Methods). False hits were counted as errors. The green bars in Figure 3A show the overall percent correct of the four human observers (chance performance = 0.5); the green bars in Figure 3B show the proportions in the five response categories. All subjects showed a similar pattern of results (see Supplementary Information).

Figure 3. Overall search performance.

A. Proportion correct for the average human searcher and three model searchers, for uniform-prior and polar-prior probability distributions over target location. Gray bars represent the average proportion correct across all trials assuming no extrinsic and no intrinsic uncertainty. Error bars are the standard error across subjects. B. Proportions of the five different categories of response for the average human observer and three model searchers. Error bars represent the 67% confidence interval. Individual subject data are given in the Figure S2A.

Foveated ideal (FI) searcher

Next, we describe the maximum performance that can theoretically be achieved in the covert search experiments, given the human observers’ measured detection performance across the search region (i.e., their measured d’ maps) when the target’s location is cued/known on each trial. Detectability (d’) was measured as a function of target contrast in the fovea, and then as a function of retinal eccentricity along the inferior, superior, and temporal axes, for the target contrast used in the search experiment (Figure 2B). Note that the target is far above threshold in the fovea (see black circle and caption for Figure 2B).

Even when target location is known, there remains intrinsic (internal) position uncertainty, which can be estimated from separate psychophysical measurements (see Methods). Figure 2C shows the measured d’ map (over the search region) for the average observer, after correcting for the effect of the intrinsic position uncertainty and interpolating from the measurements made along the cardinal axes (Figure 2B). Correcting for the effect of intrinsic position uncertainty is essential when the target can appear anywhere within the search region, because it is subsumed by the extrinsic uncertainty. However, if the potential target locations are well separated, then the extrinsic uncertainty does not subsume the intrinsic uncertainty, and hence it is unnecessary (inappropriate) to correct for intrinsic uncertainty (see Methods).

In the Methods, we derive the ideal searcher assuming white-noise backgrounds and a known or measured d’ map. We will call this parameter-free model searcher the foveated ideal (FI) searcher. The FI searcher processes every location of the search region in parallel with the same fidelity it would process each location if it were the only potential target location (i.e., the processing after sensory encoding is in parallel with unlimited capacity). The FI searcher optimally combines knowledge of the d’ map, the prior of target absent, and the prior over potential target locations, to compute the posterior probability that the target is absent and the posterior probability that it is present at each of the possible target locations (see Figure 1).

The burnt-orange bars in Figure 3 show the performance of the FI searcher. Human performance falls somewhat below ideal for both the uniform-sampling experiment and for the polar-sampling experiment. Nonetheless, the human observers do not fall far below ideal, and the proportions for the five response categories closely track that of the ideal. We also include in Figure 3A (gray bars) the predicted accuracy of the ideal if there were no intrinsic or extrinsic position uncertainty. Comparison of the gray and orange bars illustrates the effect of position uncertainty on search performance, even with parallel unlimited processing capacity.

Figure 3 shows that the human observers are performing only somewhat below optimal, given their d’ maps. This finding is largely consistent with previous findings of unlimited parallel processing in search tasks having a smaller number of potential target locations^1,2. However, a more detailed look at the data reveals a rather different story. In Figure 4A, we binned the proportion of hits and misses as a function of the retinal eccentricity of the target—the distance of the target from the fixation location. The orange curves show the predictions of the FI searcher. When the target is near fixation, the d’ map reaches its maximum (see Figure 2C) and hence the FI searcher’s hit rate is high (100% correct) and its miss rate is low (0% errors). Its performance only drops down beyond about 3° eccentricity. The results for the average human observer (green curves) are very different. Human performance is far below optimal near the fovea. For the average human observer, performance remains relatively flat out to approximately 4° eccentricity, and even declines slightly at the center of the fovea. This pattern is similar for all four observers (although there are individual differences): performance was far below expected in the foveal region and closer to optimal in the periphery. In other words, when human observers are searching for a target over a large region, there is substantial “foveal neglect”. It may seem contradictory that overall performance should approach optimal, when foveal performance is far below optimal. However, this is explained because the fovea covers only a small fraction of the search area. When engaged in covert search over a large area, foveal performance can be sacrificed with little effect on overall performance.

Figure 4. Search performance as a function of target and mouse-click eccentricity.

A. Radially binned hit and miss plots for the average human searcher and two model searchers, ideal (FI) and attention gain (ELF). Note that proportions of hits and misses do not sum to 1.0 because of false hits, and thus the hit and miss plots are not redundant. Error bars represent 67% confidence intervals. B. Radially binned click locations for false hits and false alarms for the average human observer and two model searchers. (Data and model predictions for the individual observers are in the Figures S3, S6.) It is important to note that the foveal neglect effect is underestimated in these plots because of the ceiling effect in the FI observer at smaller eccentricities. A better measure is the attention-gain maps described later (see Figure 6D).

There are three kinds of errors that observers can make: misses, false alarms, and false hits. Figure 4B and Figure 5 show how the errors are distributed across the search region. The green and orange points in Figure 4B show the proportion of mouse clicks binned as a function of eccentricity. The green and orange points in Figures 5A and 5C show the raw click locations. The distribution of ideal and human click locations is qualitatively similar. On the other hand, the distribution of target locations for misses and false hits are qualitatively different between human and ideal (Figures 5B and 5D): humans have many misses and false hits in the fovea and the ideal observer has none.

Figure 5. Scatter plots of click locations and target locations, for the four human searchers (9600 trials combined) and two model searchers, ideal (FI) and attention gain (ELF).

A. Click locations of false alarms. B. Target locations of false hits. C. Click location of false hits. D. Target locations of misses. The data and model predictions for the individual observers are in Figures S4, S5.

The simplest hypothesis for the measured foveal neglect is that the observers misestimate or misapply the prior probability of the target appearing in and around the fovea. In other words, the observers have parallel unlimited-capacity processing, but they apply the wrong prior when estimating the likelihood of the target being present at each image location. In the Bayesian modeling framework, selective attention (without sensitivity loss) is implemented by applying a prior probability distribution^10–12. Thus, another way to describe this hypothesis, in the context of our human results, is that the observers are applying an inappropriate distribution of selective attention. We implemented this suboptimal searcher, which we call the biased foveated ideal (BFI) searcher, but it is rejected because it qualitatively fails to predict false-alarm and false-hit click locations in the foveal region (Figures 5A and 5B), and because the overall quantitative fit is relatively poor (see later section on the BFI searcher). Thus, we conclude that the measured foveal neglect is most likely due to limited attentional sensitivity resources that are distributed so as to favor the periphery over the fovea. We first describe the predictions of a model searcher with limited sensitivity resources (the ELF searcher) and then describe the BFI searcher.

Efficiency-limited foveated (ELF) searcher

We represent the limited, attentionally-controlled, sensitivity resources with a gain map g (x) that scales the d’ map

$$d_g^{\prime }(x)=g(x)d^{\prime }(x)$$ (1)

where x = (x, y) is a retinal image location, and the maximum gain is 1.0, g (x) ≤ 1.0. We assume that there is a baseline gain g₀ everywhere, plus some incremental attentional gain Δg (x), and thus g (x) = Δg (x) + g₀. Because the gain values scale the d’ values, they can be regarded as efficiency scalars⁹; thus, we refer to this model searcher as the efficiency-limited foveated (ELF) searcher. We note that a gain factor on sensitivity is not equivalent to a gain factor on neural response, and that there are multiple possible neural mechanisms that can reduce efficiency: increased response variance, reduced response gain (increased normalization), increased pairwise noise correlations between neurons¹³, and related factors described in the psychophysical literature^1,14.

The ELF searcher is based on two key assumptions. The first is that there is a limited total available attentional gain Δg_tot, and hence in search tasks with a sufficiently large search region there must be locations where the attentional gain is less than 1.0. The second is that the total attentional gain is distributed over neurons in visual cortex (e.g., there is a maximum total number of cortical neurons that can receive a gain of 1.0). This latter assumption is important because cortical neurons that encode peripheral locations are fewer in number and have larger receptive fields. Thus, peripheral neurons effectively apply their efficiency gain over a larger area of the retinal image. In other words, attentional sensitivity resources applied in the periphery necessarily cover a larger area of the retinal image than what is covered by applying the same resources nearer the fovea. This important fact could explain why human observers neglect the fovea and parafovea relative to the periphery when the search region is large.

To formalize this hypothesis, we created retinotopic maps based on the spacing of ganglion-cell receptive fields in the human eye estimated from anatomical measurements in 6 human eyes¹⁵. We based the maps on ganglion-cell anatomy, because the cortical magnification factor is similar to the retinal ganglion-cell magnification factor¹⁶, and because there exist simple formulas for ganglion cell receptive-field spacing. Specifically, we used the midget ganglion-cell spacing functions estimated from the data of Curcio & Allen¹⁵ by Drasdo et al.¹⁷ and Watson¹⁸. We created retinotopic maps under the assumptions that in the maps, the horizontal meridian falls on a horizontal line and the iso-eccentricity contours fall on vertical lines. These constraints produce maps (Figure 6A lower) that are similar to those directly measured in human visual cortex¹⁹ (Figure 6A upper). Other topological constraints could be used to get a closer match to human V1, but as long as the maps accurately represent the receptive-field spacing function, the specific constraints do not affect the current predictions. We note that although our focus here is on retinotopic maps in primary visual cortex, the retinotopic maps in other visual cortical areas are consistent with an increase the receptive field size and spacing with retinal eccentricity. Thus, the same logic would hold for limited attentional-gain resources in other visual areas.

Figure 6. Retinotopic attentional-gain maps.

A. Upper: full retinotopic map of human primary visual cortex (adapted from Adams et al.¹⁹) Lower: full retinotopic map based on a human midget-ganglion cell spacing function derived in Drasdo et al.¹⁷ from anatomical measurements in 6 human eyes¹⁵. B. Upper: estimated map of attentional gain on d’ over the 16° diameter search area, plotted in retinotopic space, for the uniform-prior experiment. Lower: effective d’ map after applying the attentional gain map to the d’ map in Figure 2C. C. Upper: estimated attention-gain map for the polar-prior experiment. Lower: effective d’ map. D. Cross sections of the attention gain maps along the horizonal meridian in B and C. E. Upper left: attention-gain map that maximizes accuracy in the uniform-prior experiment. Upper right: attention-gain map that maximizes accuracy in the polar-prior experiment. Lower: attention-gain map that covers the largest possible central region (radius = 5.3°) with a gain of 1.0. All the gain maps have the same total attentional gain Δg_tot.

A retinotopic map is a vector-valued function x(i) = [x(i), y(i)] that gives the retinal location corresponding to each cortical-map location i = (i, j). Our assumption is that the total attentional gain distributed across the retinotopic map cannot exceed some maximum value Δg_tot:

$$\sum _i\Delta g(i)\le \Delta g_{tot}$$ (2)

The gain map in retinal-image coordinates g (x) is obtained by using the inverse retinotopic map i(x) = [i(x), j (x)] to interpolate the gain map in cortical coordinates: g (x) = Δg [i(x)] + g₀. Substituting into Equation 1 gives the attentionally-modified d’ map, $d_g^{\prime }(x)$. For more details see Methods.

We have considered a number of different kinds of attentional-gain map. Our first goal was to determine whether, in agreement with intuition, a map that increases monotonically with retinal eccentricity could explain the human data for the uniform-prior experiment. Specifically, we fit all the uniform-prior data in Figures 3, 4, and 5 assuming a monotonically-increasing Weibull function having four parameters: the minimum gain in the fovea Δg_min, the maximum gain in the periphery Δg_max, a steepness parameter, and a shape parameter. We used maximum likelihood estimation to find the values of these parameters, together with the value of the total gain parameter Δg_tot, that best fit all the data (see Methods). In our experiment there is no way to distinguish the effect of the baseline gain g₀ and Δg_min, thus without loss of generality we simply set g₀ = 0. The fits were obtained separately for the average observer and the four individual observers (fits for individual observers are given in the Supplementary Information). Figure 6B shows the estimated gain map (in retinotopic coordinates) and the resulting d’ map, for the average human observer. The lighter curve in Figure 6D plots attention gain as a function of retinal eccentricity (it corresponds to horizontal slice at 0° in Figure 6B). The predictions, which are quite accurate, are shown by the blue bars in Figure 3, and the blue symbols in Figures 4 and 5. The predictions are also quite accurate for the individual observers (see Supplemental Information). There are individual differences in the estimated attention gain functions (Figure S7); the gain increases by a factors ranging from about 1.4 to 2 from the fovea to the periphery. These differences, and the differences in the observers’ d’ maps, account for the individual differences in the patterns of hits, misses, false alarms and false hits (Figures S2–6).

For comparison, we also generated predictions under the assumption of a constant gain across the whole search area, for the same total gain Δg_tot as above. This results in a d’ map that is a scaled down version of that in Figure 2C. The overall accuracy is lower (purple bar in Figure 3A), confirming the advantage of foveal neglect over spreading the same attentional resources uniformly over the search area. Furthermore, the pattern of response proportions is more poorly predicted (purple bars in Figure 3B), as is the pattern of results in Figures 4 and 5 (not shown).

The gain map in Figure 6B explains human search performance, but is it the gain map that achieves the highest possible accuracy, for the same total attentional gain? The upper left map in Figure 6E is the map that would maximize accuracy. The overall accuracy is 84% for the upper-left map compared to 71% for the map in Figure 6B. Thus, greater overall accuracy can be obtained by almost completely suppressing the foveal region (the central 1.8°). The overall accuracy of the foveated ideal observer is 87% correct (Figure 3). In other words, almost completely suppressing the foveal region would have allowed the ELF searcher to reach an accuracy just 3% below that of the foveated ideal (FI) searcher, which has no attentional gain limitations. Note, however, that setting the gain to near zero in the fovea would make the fovea completely blind. This is the right thing to do in our specific task, but is a bad strategy in the natural world, where less drastic foveal suppression has advantages (see Discussion).

A uniform prior on target location over a search area is the least informative prior; thus, the polar-prior conditions should allow better performance. Also, because pixel locations further from fixation are less likely to be target locations, the attentional-gain map may not increase monotonically with eccentricity.

The ELF searcher has a fixed total attentional gain available, independent of the prior over target location; thus, in estimating the gain maps for the polar-prior conditions, we required that the total gain parameter remain the same as for the uniform-prior conditions. The attention-gain map that best explains the polar-prior data is shown in Figure 6C and by the darker curve in Figure 6D. As can be seen, it is almost identical to the gain map for the uniform-prior conditions. This is true for all the human observers (see Figure S7). The predictions for the polar-prior conditions are also quantitatively accurate (Figures 3, 4, and 5). The ELF searcher correctly predicts for the polar conditions, relative to the uniform conditions, (i) the better overall human performance, (ii) the increased false hit locations in the fovea, and (iii) the increased concentration of misses in the fovea. We conclude that the ELF searcher with a fixed total attentional gain can explain human performance in both experiments.

Interestingly, the maximum-accuracy map for the polar prior conditions is non-monotonic: gain is reduced in the foveal region, reaches a maximum at intermediate eccentricities and drops off strongly at the furthest eccentricities of the search region (upper right map in Figure 6E). Also, there is an equally good maximum-accuracy map that monotonically decreases with eccentricity (not shown). (To allow for the possibility of non-monotonic or decreasing gain maps, a second Weibull function was included; see Methods.) It appears that our moderately practiced human observers did not adjust their attentional-gain maps in a way that qualitatively matches the gain maps that are optimal for the polar prior on target location. In the Discussion, we return to the issue of how flexible are human attention gain maps.

Biased foveated ideal (BFI) searcher

As mentioned earlier, the simplest hypothesis for the measured foveal neglect is that the observers have parallel unlimited-capacity processing, but they apply the wrong prior when estimating the likelihood of the target being present at each image location. If we represent the prior over target location by p(x), and a bias map by b(x), then the biased prior is

$$p_b(x)=p(x)b(x)$$ (3)

To obtain predictions we assumed that the bias varied monotonically with eccentricity according to a Weibull function with four parameters: a starting value in the fovea, an ending value in the periphery, a slope parameter and a steepness parameter (see Methods). Again, the parameter values were estimated by finding the four parameter values that maximized the likelihood of all the data.

The dark blue points and bars in Figure 7 show the maximum likelihood fit to the uniform prior conditions. As before, the green and orange points and bars show the average human data and the predictions of the foveated ideal (FI) searcher, respectively. The BFI observer makes substantially worse predictions than the ELF searcher for the foveal neglect effect (Figure 7A), the proportions of different response categories (Figure 7B), and the scatter plots of false-alarm clicks (Figure 7C), false-hit clicks (Figure 7E), false-hit target locations (Figure 7D), and miss target locations (Figure 7F). A salient qualitative failure of the BFI searcher is its inability to predict false-alarm and false-hit clicks in the foveal region (Figures 7C and 7E).

Figure 7. Predictions of the biased foveated ideal (BFI) searcher for uniform prior conditions.

Dark blue points and bars are predictions of the BFI searcher for the maximum likelihood fit to all data (BFI2). The light blue points and bars are predictions for the maximum likelihood fit that concentrates on hits and misses as function of eccentricity (BFI1). The green points and bars are the human data. The orange points and bars are the predictions of the foveated ideal (FI) searcher.

To obtain some insight into the failures, we also estimated the parameters by concentrating the maximum-likelihood estimation on the hits and misses as a function of target eccentricity. The lighter blue points and bars show the resulting fits. The fits are a little better for the hits and misses (the foveal neglect effect), but the fits are worse for other aspects of the data. We also tried other shape families for the bias map (e.g., we applied the Weibull bias function to the log of the priors), but none of these performed better. We conclude that relative to the ELF searcher the BFI searcher is not a viable model.

Alternative hypotheses for foveal neglect

We also considered some other hypotheses for the measured foveal neglect. Might foveal neglect be due to fixation biases or anticipatory eye movements? This hypothesis is unlikely because any deviation from central fixation should reduce accuracy for both the uniform and the polar prior conditions, because the average target location would be further in the periphery. In other words, there should be no incentive for the human observers (or an ideal observer) to deviate from central fixation. Nonetheless, we ran a control replication of the uniform and polar prior conditions with eye tracking, on two of the observers. We found no systematic fixation bias, no evidence for anticipatory eye movements, and no difference in detection accuracy as a function of the measured fixation distance from the center of the display (see Figure S7). Another potential concern might be that the initial fixation target produced a masking effect in the foveal region. This hypothesis can be ruled out because the fixation target was extinguished 250 ms before search display onset, and because the same fixation targets were used in measuring the d’ maps, which are the basis for quantifying foveal neglect.

DISCUSSION

Covert search for a small sinewave target in large noise backgrounds was measured for uniform and polar prior probability distributions over target location. We found that overall search performance was largely consistent with a Bayesian ideal searcher that is limited only by the foveated detectability maps (d’ maps) directly measured in the human observers. This finding for overall performance is consistent with many previous studies, which measured covert search for smaller displays and small numbers of discrete potential target locations^1,2. However, surprisingly, the spatial distribution of errors revealed a new phenomenon: when humans are searching over a large area, their performance is severely depressed near the fovea, but is nearer optimal in the periphery. We considered two general hypotheses for this foveal-neglect effect in covert search.

One hypothesis is that there is a limitation of selective attention, where the observer misestimates and/or misapplies the prior probability distribution of the target’s location^10–12. Suboptimal selective attention involves applying an inaccurate prior; for example, not ignoring responses from irrelevant locations. The overall pattern of results and particularly the distribution of false-alarm and false-hit clicks rejects the selective-attention hypothesis. Of course, selective attention plays a major role in the search task, accounting for most of the differences in performance between the cued task used for estimating the d’ maps and the search task where the target can appear anywhere in the search region. The selective-attention hypothesis fails only in explaining the observed foveal neglect.

The second hypothesis is that observers have a large, but still limited, total amount of attentional sensitivity gain that they can flexibly distribute over the retinotopic representation in visual cortex. This hypothesis predicts that efficient allocation of the sensitivity gain should result in foveal neglect. Furthermore, this hypothesis correctly predicts (with just a few parameters) the detailed pattern of results measured in the covert search experiments.

Attentional sensitivity gain

There is considerable psychophysical evidence for endogenous (and exogenous) attentional effects on visual sensitivity^2,3. For example, attention can cause modest increases in apparent contrast^20,21, brightness²², and contrast sensitivity²³. There is also considerable neurophysiological evidence for effects of attention on sensitivity^2,3,14. Single-unit studies in various cortical and subcortical visual areas in behaving primates, show that attention directed toward or away from the spatial region containing the receptive field can cause moderate (5% to 30%) modulations in responsiveness¹³. These are likely to represent even smaller percentage changes in sensitivity (d’), given that the variance of single neuron responses typically increases in proportion to the mean response. Given the small size of these effects, they are unlikely to underlie the foveal-neglect effect measured here. Indeed, it is more likely that these small sensitivity changes are part of the neural implementation of selective attention; for example, a small attentional modulation in responses may be used to drive the selection of responses in winner-take-all networks.

Foveal neglect is also different from inattentional blindness, where observers engaged in one task fail to detect salient, task-irrelevant objects^24–26. Inattentional blindness is the result of highly-effective selective attention, which suppresses task-irrelevant signals. In the current experiments the targets at all search locations are task relevant. Said another way, inattentional blindness is not seeing something right in front of you when you are not looking for it; foveal neglect is not seeing something right in front of you when you are looking for it.

We are thus drawn to the conclusion that the sensitivity attention revealed in our covert search task may be a new phenomenon not revealed in previous tasks designed to study attention in humans and non-human primates. This conclusion is strengthened by the fact that the total attentional gain resources estimated in our search task is quite substantial. Specifically, the estimated total gain is sufficient to fully cover a search region having a diameter of 10.6° (Figure 6E lower). If humans are able to flexibly distribute the total gain, then for search diameters less that 11° they would show no foveal neglect and would behave like the FI (ideal) searcher. With this much attentional gain available, most previous studies would not have taxed the attentional resources sufficiently to reveal the foveal-neglect effect. Although foveal neglect may not have been a factor in many previous studies of covert search, it is likely to be an important factor in many real-world covert and overt search tasks where search regions (i.e., the image locations with a non-zero prior probability of containing the target) tend to be larger than 11° × 11°.

Flexibility of attentional gain maps

Our results show that when a single target location is cued, the gain at the target location is 1.0, and when the potential target locations cover a region having a diameter of 16°, the gain is adjusted to be about 0.5 in the fovea and 0.85 in the periphery. However, the results for the polar-prior experiment suggest that there may be limitations to the flexibility of the gain maps. The optimal map is inverted-U shaped or monotonically decreasing (Figure 6E upper right), whereas the human observers have maps that are monotonically increasing, like the map for the uniform prior (Figure 6C and 6D).

Although the flexibility of the gain maps needs to be tested further in experiments, the observed limited flexibility is plausible. It is quite possible that stereotypic attention gain maps, either concentrated in the fovea or concentrated in the periphery, evolved to support visual performance under natural conditions. It is also possible that the stereotypic gain maps for large image regions are biased toward the assumption of a uniform prior on target location. The uniform prior is the least constraining assumption and, arguably, might be the best default assumption.

We also note that the optimal gain map for the uniform prior shown in Figure 6E is not realistic. There must be a baseline gain g₀ that is not under attentional influence and that exists over the whole visual field. If not, there would be totally blind regions of visual space, which would have negative consequences for survival in natural environments. Also, a zero baseline is not consistent with the fact that all locations in the visual field can drive cortical neurons, even if an animal is anesthetized or attending to a particular location. Thus, it is likely that the maps we estimated from human performance are closer to the real-world optimal for uniform priors.

One reason that the attention gain maps may not be very flexible is that the total gain resource is so large that there is little need to be more flexible. When the visual system is aiming to process information in the central visual field, the attentional gain slides toward the fovea (toward the back of the brain in V1) to cover the central 11 deg (map in Figure 6E lower). When the visual system is aiming to process information over the whole visual field, the attention gain slides toward the periphery (toward the front of the brain in V1; Figures 6B and 6C). Even if the attention gain is set to 1.0 at some irrelevant locations, the selective attention mechanisms will remove the influence of those responses on task performance.

In addition to testing the flexibility of attention gain in covert search, it will be important to determine the role of attention gain in overt search. A plausible hypothesis is that when human searchers are choosing the next fixation location (when the search area is large) the gain map shifts to emphasize the periphery (foveal neglect), but when a saccade is executed toward a location having features consistent with the target, the map shifts to emphasize the foveal region. On the other hand, sometimes new fixations are selected to maximize the acquisition of new information; for example, fixating into the middle of an unexplored area of the search display^27,28. In such cases, the map emphasizing the periphery may be retained across the saccade. This could help explain the anecdotal reports that in overt search, humans sometimes fixate near the target without seeing it²⁹. Previous overt search studies in noise backgrounds (with no constraints on search time), have obtained results generally consistent with the predictions of uncertainty over target location^30,31 and hence provide no evidence of foveal neglect. However, this is to be expected given the size of search displays (which are under 11° diameter) and the absence of time constraints.

Other kinds of attentional sensitivity gain in covert search

There are other kinds of sensitivity attention in covert search. These arise when a more cognitive subtask is required as part of the covert search task. To take an extreme example, consider the search task where one must respond whether one of the many two-digit numbers in a brief display is divisible by three. Arbitrary computations like division of numbers do not occur in parallel, so limits on short-term memory result in large reductions in detectability compared with what would be measured for a single target location. Less extreme, but similar, examples arise because of perceptual grouping. For example, texture regions containing elements that are mirror symmetric [e.g., or ] do not perceptually segregate and hence relatively high-level cognitive computations are required to identify the regions³². Covert search with such elements also shows evidence of capacity limitations³³. The capacity limitation, in terms of number of locations, is generally much more severe than those underlying the foveal neglect effect described here.

STAR METHODS

RESOURCE AVAILABILITY

Lead contact

Further information about the research and requests for resources should be directed to and will be fulfilled by the Lead Contact, R. Calen Walshe (calen.walshe@utexas.edu).

Materials availability

The study did not generate any unique reagents.

Data and code availability

The R (version 4.0.3) and MATLAB (R2020b) programming environments were used for data analysis and visualization of the reported results. Experiment data and original code can be accessed at the Open Science Framework (OSF) repository and are publicly available as of the date of publication. DOIs are listed in the key resources table.

KEY RESOURCES TABLE

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Deposited Data
Open Science Framework	https://osf.io	DOI 10.17605/OSF.IO/YBGJ6
Software and algorithms
MATLAB 2020b, MATLAB 2019a	https://www.mathworks.com/
R (version 4.0.3)	https://www.r-project.org/about.html
Psychophysics Toolbox 3	http://psychtoolbox.org/

EXPERIMENT MODEL AND SUBJECT DETAILS

The experimental procedures were carried out in accordance with the review board of The University of Texas at Austin and all subjects provided informed consent prior to running in the study. The study was conducted in compliance with the Declaration of Helsinki.

Three males and one female participated in the study, including one of the authors. All subjects had normal or corrected-to-normal vision with ages 28, 27, 24, and 39.

METHOD DETAILS

The experimental data was collected on a Sony GDM-FW900 monitor, 85Hz refresh rate. The gamma value of the monitor was measured with a Spectrascan PR655 and this value was used to gamma-compress the images prior to display on the screen. The images were quantized to an 8-bit precision and presented at a display resolution of 60 pixels per degree (full display size was 1920 × 1200 pixels). The minimum and maximum gray levels were 0.67 cd/m² and 90.41 cd/m², respectively.

Covert Search Experiments

All stimuli were generated and presented using MATLAB 2019a with the Psychophysical Toolbox. A schematic and example stimulus for the search experiments is shown in Figure 2A. Each participant ran 2400 trials in both a uniform sampling experiment, where each pixel had an equal probability of being selected as the search target location, and a polar sampling (uniform sampling in polar coordinates) experiment. There was an equal probability that a target would be present or absent on a specific trial. Each subject performed at least 1000 practice trials before running the experiment to become familiar with the task. A trial sequence proceeded as follows: first the participant fixated for 750 ms on a uniform gray display image at the center of the search region (center of the monitor). The ensure that the participant maintained fixation, two horizontally oriented gray flanker bars were presented on either side of the fixation location at a separation of 0.33°. Next, the gray bars were extinguished, and following a duration of 250 ms, the search stimulus was presented. The search stimulus was a 16° diameter circular field of Gaussian white noise having a root-mean-squared (RMS) contrast of 20% presented with or without a target. On target present trials a 6 cycle per degree, vertically oriented Gabor target with a Michelson contrast of 35% was added at a randomly selected location. The target contrast was determined via preliminary experiments to obtain search performance of approximately 70% correct. The stimulus was presented at a resolution of 30 pixels/degree with each noise and target pixel corresponding to a 2×2 pixel region on the monitor. The stimulus was presented for 250 ms. Immediately following stimulus presentation, the flanker bars were presented again on a mean gray background. The participant then used a mouse to indicate their response. The participant responded “present” by moving the mouse cursor to a location in the search display and clicking the left mouse button. To respond “absent” the participant clicked on right mouse button. Auditory feedback was then provided about whether the response was correct, and the trial was then terminated.

Control Covert Search Experiment with Eye Tracking

To evaluate the potential effects of fixation bias, we replicated the uniform-prior and polar-prior search experiments on two of the observers while tracking eye position with an EyeLink 1000 (SR Research).

Detection Experiments

The detection task was similar in most aspects to the search task (see Figure 2A) except for the following details: we utilized horizontally oriented flanker bars to cue the location where the target was located and the luminance of the flanker bars was set to 70% of the maximum gray level. We measured detectability at 7 eccentricities: 0° (the center of the fovea), 3°, 3.3°, 4.3°, 5.8°, 8°, 8.5°. These eccentricities span the full range of potential eccentricities in the search tasks. Subject performance was measured along the temporal, inferior, and superior cardinal axes, with 240 trials at each eccentricity. The distance of the flanker bars from the target increased with eccentricity according to the rule: separation = 0.33° + 0.1° × eccentricity. The target shape and background contrast were the same as in the search task. The foveal task was completed separately because target contrast rather than eccentricity was varied. We selected five target contrasts for the foveal task, 10%, 12%, 15%, 17%, 20% Michelson contrast, with 240 trials per contrast level. Flanker bars were also included in the foveal task: separation = 0.33°. Before each trial a cue was presented at the center of the monitor to reorient a subject’s gaze to the center of the display.

QUANTIFICATION AND STATISTICAL ANALYSIS

Here we first describe the specification of the d’ map from the measurements in the cued detection experiment. We then define and derive several covert searchers that are optimal, given specific constraints. The foveated ideal (FI) searcher has the same detectability, d’, across the visual field as the human searchers. The efficiency-limited foveated (ELF) searcher is identical to the FI searcher but has a fixed total efficiency gain that it can flexibly distribute over the search area. The biased foveated ideal (BFI) searcher is also identical to the FI searcher except that it misestimates the prior probability of targets being presented near the fovea.

Specification of the d’ maps

To specify the human detectability map we start with the well-known fact that the detectability of the ideal observer in Gaussian white noise is given by d’ = a/σ, where a is the amplitude (square root of the energy) of the target and σ is the standard deviation of the noise, which is the product of the mean luminance and the RMS contrast of the background σ = LC. Ignoring for the moment the effect of intrinsic position uncertainty, human d’ in the fovea is approximately proportional to that of the ideal observer³⁴, with a proportionality constant (efficiency scalar) η_T that depends on the target T. There is also evidence that detectability varies with retinal location approximately inversely with the spacing of midget ganglion cell RFs, d′(x) ∝ [1 + ϕ_Tsp(0)]/[1 + ϕ_Tsp(x)], where sp(x) is the spacing function, and ϕ_T is another target-dependent constant (e.g., d′ falls off more rapidly for high-spatial-frequency targets). The spacing function we use was obtained directly from human anatomical measurements^17,18,35. Combining these factors, we obtain a principled descriptive model of human detectability, in the absence of position uncertainty, for arbitrary targets in white-noise backgrounds:

$$d^{\prime }(x)={\eta }_T\frac{a}{LC}\frac{\left[1+{\varphi }_{T}sp(0)\right]}{\left[1+{\varphi }_{T}sp(x)\right]}$$ (4)

This model has two free parameters, η_T and ϕ_T, that can be estimated from measurements of human detectability across the visual field, once the effect of intrinsic position uncertainty is taken into account.

Even when the target location is cued on each trial, there remains intrinsic position uncertainty; i.e., human observers will accept features that match those of the target, if the features are within some small retinal-image region around the cued location. Thus, cued detection can be regarded as visual search over a small region. Correcting the measured d’ values for the effect of intrinsic position uncertainty is essential for generating sensible model predictions for continuous priors on target location, because the intrinsic uncertainty is subsumed by the extrinsic uncertainty. For example, using the uncorrected d’ map in the ideal (FI) searcher predicts search accuracy far below human performance (51% for the uniform prior conditions and 58% for the polar prior conditions). On the other hand, correction for intrinsic uncertainty is not appropriate when the potential target locations are well separated, because the uncorrected d’ map already represents the correct level of intrinsic position uncertainty that exists at each of the M separated locations.

To take account of intrinsic position uncertainty, we exploited a closed-form expression³⁶ that closely approximates the effects of intrinsic position uncertainty on detectability

$$d_U^{\prime }(x)=\text{ln}\left[\frac{\text{exp}\left(k_d{d}^{\prime }(x)\right)+k_u{\sigma }_U(x)}{1+k_u{\sigma }_U(x)}\right]$$ (5)

where σ_U(x) is the standard deviation of the intrinsic position uncertainty. Michel & Geisler³⁷ measured the standard deviation of human pointing errors (errors in the perceived location of the target) in cued detection tasks, and found that the standard deviation increased in proportion to midget ganglion cell spacing: σ_U (x) = k_σ sp(x). For essentially the same target used in the current study, they found that the standard deviation was approximately 0.45 deg at 5 deg along the superior vertical meridian. Because spacing in the center of the fovea is 0.0083 (the spacing between photoreceptors), k_σ ≅ 10. To estimate k_u we computed detectability (values of $d_U^{\prime }$) at different target contrasts and eccentricities from simulated template responses to our 6 cpd target in noise, with intrinsic-uncertainty standard deviations given by σ_U (x) = 10sp(x). The estimated value of k_u (43.2) was the value that gave the best fit to the simulated detectabilities using Equation 5. (For the current stimuli k_d in Equation 5 could be set to 1.0 without loss of accuracy).

Equations 4 and 5 cannot be used to directly generate search predictions, because they only apply in a small neighborhood of a given retinal location. However, they can be used to estimate the two free parameters of the detectability equation (Equation 4) by fitting human detectability at various retinal locations measured in a simple detection task, where the location of the target is known/cued. Once η_T and ϕ_T are estimated, it is possible to generate predictions for the three covert searchers considered here, by simulating search trials.

Figure 2B shows the maximum likelihood fit of Equations 4 and 5 to the average psychometric data from the four human observers in the cued-location detection experiment. (We used the spacing functions of Drasdo¹⁷ and Watson¹⁸, but the fit using Drasdo was slightly better, so that is what shown here.) The fits to the individual observers are given in the Supplementary Information. The equations fit the data reasonably well and capture the major trends. For example, the ganglion-cell spacing function derived from average human anatomy correctly predicts that detectability along the inferior and temporal cardinal axes is higher than along the superior cardinal axis (although, as expected, there are some individual differences).

Ideal searcher: statistically independent locations

Before describing the three searchers that we compared with the human search data, we describe the generic ideal covert searcher, where the responses at different locations are statistically independent. On each search trial there is a response from each of the n possible target locations: R = [R(x₁), ⋯, R(x_n)], where x_i = (x_i − x_f, y_i − y_f) is the i^th possible target location relative to the fixation location (x_f, y_f). Let the prior probability over target location be p(x_i), where i ranges from 0 to n. For mathematical convenience we define x₀ to be an imaginary location representing “target absent.” The decision rule that maximizes accuracy is to pick the location with the maximum posterior probability: $\widehat{x}=\underset{x_i}{\text{arg max }}p\left(x_i\mid R\right)$. Using Bayes rule, the posterior probability is given by

$$p\left(x_i\mid R\right)=\frac{p\left(R\mid x_i\right)p\left(x_i\right)}{\sum _{j=0}^{n}p\left(R\mid x_j\right)p\left(x_j\right)}$$ (6)

Dividing top and bottom by the numerator we have

$$p\left(x_i\mid R\right)=\frac{1}{1+\sum _{j\ne i}\frac{p\left(R\mid x_j\right)p\left(x_j\right)}{p\left(R\mid x_i\right)p\left(x_i\right)}}$$ (7)

Assuming statistical independence and cancelling terms gives

$$p\left(x_i\mid R\right)=\frac{1}{1+\sum _{j\ne i}\frac{p\left(x_j\right)\prod _{k=1}^{n}p\left[R\left(x_k\right)\mid x_j\right]}{p\left(x_i\right)\prod _{k=1}^{n}p\left[R\left(x_k\right)\mid x_i\right]}}$$ (8)

Finally, because the bottom product does not depend on j, we obtain

$$p\left(x_i\mid R\right)=\frac{p\left(x_i\right)\prod _{k=1}^{n}p\left[R\left(x_k\right)\mid x_i\right]}{\sum _{j=0}^{n}p\left(x_j\right)\prod _{k=1}^{n}p\left[R\left(x_k\right)\mid x_j\right]}$$ (9)

Foveated ideal (FI) searcher

The FI searcher assumes that (i) the responses at every possible target location are Gaussian distributed, which is appropriate given the Gaussian white-noise backgrounds, and that (ii) the variance of the responses is the same for target present and target absent trials, which is appropriate given the additive targets. Without loss of generality, we can normalize the responses so that the response standard deviations are always 1.0, and so that the mean response when the target is present at x_i is 0.5d’(x_i) and when absent is −0.5d’(x_i) (this puts the cross point between the distributions at 0). It follows that when the location of the target is x_i, where i ≠ 0, then

$$\prod _{k=1}^{n}p\left(R_k\mid x_i\right)={(2\pi )}^{-n/2}\text{exp}\left(-0.5{\left[R\left(x_i\right)-0.5d^{\prime }\left(x_i\right)\right]}^2-0.5\sum _{k\ne i}{\left[R\left(x_i\right)+0.5d^{\prime }\left(x_k\right)\right]}^2\right)$$ (10)

whereas, when the target is absent (i = 0)

$$\prod _{k=1}^{n}p\left(R_k\mid x_i\right)={(2\pi )}^{-n/2}\text{exp}\left(-0.5\sum _{k=1}^n{\left[R\left(x_i\right)+0.5d^{\prime }\left(x_k\right)\right]}^2\right)$$ (11)

(In the current study n = 190,000.)

Substituting into Equation 9 and simplifying gives

$$p\left(x_i\mid R\right)=\frac{p\left(x_i\right)\text{exp}\left[R\left(x_i\right)d^{\prime }\left(x_i\right)\right]}{\sum _{j}p\left(x_j\right)\text{exp}\left[R\left(x_j\right)d^{\prime }\left(x_j\right)\right]}$$ (12)

where d’(x₀):= 0. Because the denominator in Equation 12 is the same for all i, the optimal decision rule is simply:

$$\widehat{x}=\underset{x_i}{\text{arg max}}\left[\text{ln}p\left(x_i\right)+R\left(x_i\right)d^{\prime }\left(x_i\right)\right]$$ (13)

This result differs from the classic ideal observer for covert search with independent noise at each target location, which sums likelihoods across all locations³⁸. This difference occurs because, in the classic task, false hits are counted as correct responses. If they are instead counted as errors, then Equation 13 is the ideal decision rule. Arguably, counting false hits as errors is a more appropriate cost function in most real-world tasks (e.g., identifying the wrong location of a food item in a scene does not lead to reward). In simulating search performance for all the model searchers (including the FI searcher), we used the same definition of a false hit we used to score the human observers: the identified target location must be within 1.25° of the true location to be counted as a hit.

To simulate the responses of the FI observer, we randomly select its location x_T from the prior probability density p(x_i). Next, we generate the normalized responses for each location x_i in the search region

$$R\left(x_i\right)~N\left[d^{\prime }\left(x_T\right)(T\otimes T)\left(x_i-x_T\right)-0.5d^{\prime }\left(x_i\right),1\right]$$ (14)

where (T ⊗ T)(x) represents the autocorrelation function of the target shape, which has been normalized so that the dot product of the target shape with itself is 1.0 (i.e., (T ⊗ T)(0) = 1.0). For locations not near the target, the normalized responses are random samples from the target-absent distribution. For locations near the target, the matched-template used by the ideal searcher overlaps the target, and hence the mean responses differ from the target-absent distribution (i.e., there is some target information). At the exact location of the target, the normalized response has the maximum mean of 0.5d’(x_T). Finally, the normalized responses are substituted into Equation 13 to obtain the FI observer’s response of whether the target is present or absent, and if present the estimated location. The FI searcher’s responses are analyzed in the same way as the human searchers’ responses. Because of the slight correlations introduced by Equation 14 (a violation of strict statistical independence), it is necessary to vary the target-absent prior, p(x₀), assumed by the FI observer in order to maximize its search accuracy.

Efficiency-limited foveated (ELF) searcher

The ELF searcher is identical to the FI search except that the detectability map d’(x) is scaled by an efficiency gain g (x) that may vary depending on the task

$$d_g^{\prime }(x)=d^{\prime }(x)g(x)$$ (15)

where the efficiency gain equals a baseline gain plus an attentionally-controlled increment in gain: g (x) = Δg (x) + g₀. The maximum gain at any location is 1.0: g (x) ≤ 1.0. For example, in a detection task where the location of the target x_T is known (fixed) the gain is defined to be 1.0, and hence $d_g^{\prime }\left(x_T\right)=d^{\prime }\left(x_T\right)$. However, we assume there is a limited total available incremental gain Δg_tot, and hence in search tasks with a sufficiently large search area there will be locations were the efficiency gain is less than 1.0. We further assume (plausibly) that the total efficiency gain (a “limited resource”) is distributed over neurons in visual cortex (i.e., there is a maximum total number of cortical neurons that can receive a gain of 1.0), rather than across space as assumed in previous models^2,3.

To formalize this hypothesis, we created retinotopic maps based on the spacing of ganglion-cell receptive fields in the human eye estimated from anatomical measurements in 6 human eyes¹⁵. The maps are based on ganglion-cell anatomy, because the cortical magnification factor is similar to the retinal magnification factor¹⁶, and because there exist simple formulas for ganglion cell receptive-field spacing^17,18.

We created retinotopic maps under the assumptions that in the maps, the horizontal meridian falls on a horizontal line and the iso-eccentricity contours fall on vertical lines. These constraints produce maps that are similar to those directly measured in human visual cortex (see Figure 5A and 5B). Other topological constraints could be used to give a closer match to human V1, but as long as the maps accurately represent the receptive-field spacing function, the specific constraints do not affect the current predictions. The specific steps for generating the maps are described in the next subsection.

A retinotopic map is a vector-valued function x(i) = [x(i), y(i)] that gives the retinal location corresponding to each cortical-map location i = (i, j). Our key assumption is that the total efficiency gain distributed across the cortical map cannot exceed some maximum value g_tot:

$$\sum _i\Delta g(i)\le \Delta g_{tot}$$ (16)

Importantly, the gain assigned to neurons with larger receptive fields and greater spacing (neurons further in the periphery) will automatically apply the gain to a larger region in the retinal image. In other words, attentional resources applied in the periphery will cover a larger area of the retinal image than what is covered by applying the same resources nearer the fovea. The gain map in retinal-image coordinates g (x) is obtained by using the inverse retinotopic map i(x) = [i(x), j(x)] to interpolate the gain map in cortical coordinates:

$$g(x)=\Delta g[i(x),j(x)]+g_0$$ (17)

Given a total incremental efficiency gain of Δg_tot, we ask what distribution of gain best explains human performance in the search tasks, and how this compares to the optimal distribution of gain. In the uniform-prior experiment, the search region is circular, and the prior probability of the target location is uniform in cartesian coordinates. Under these conditions we expect the cortical gain map to be constant with a value of 1.0, if the search region is below some critical size. If the search region is above the critical size, then we expect the gain to increase monotonically with retinal eccentricity. To explore this space of maps we assumed that the gain is constant along each vertical line (retinal eccentricity) in the cortical map, and that the gain starts at some minimum level Δg_min in the center of fovea and increases monotonically toward a maximum level Δg_max according to a Weibull function with steepness parameter a and shape parameter b:

$$\Delta g(i)=\Delta g_{\text{min}}+\left(\Delta g_{\text{max}}-\Delta g_{\text{min}}\right)\left[1-\text{exp}\left(-{\left(\frac{i}{a}\right)}^b\right)\right]$$ (18)

For present purposes we set g₀ = 0 and hence Δg_min ≥ 0 and Δg_max ≤ 1.0. We determined, by maximum-likelihood fitting, the values of these three parameters, and the parameter Δg_tot, that best explains human search performance. For this value of Δg_tot, we also generated predictions assuming constant gain, and for the three parameter values that gave the highest accuracy in the search task.

In the polar-prior experiment, the prior probability is uniform in polar coordinates, and thus falls inversely with retinal eccentricity, p(x) ∝ 1/∥x∥. Under these conditions, there is a competition between the prior probability and the reduced receptive-field sampling in the periphery. The reduced sampling favors increasing gain in the periphery; whereas, the reduced prior favors reducing gain in the periphery. To represent the effect of the reduced prior we included a second Weibull function, with steepness and shape parameters c and d, that monotonically increases from the search boundary toward the fovea. The gain map was taken to be the minimum of the two Weibull functions:

$$\Delta g(i)=\Delta g_{\text{min}}+\left(\Delta g_{\text{max}}-\Delta g_{\text{min}}\right)\text{min}\left[1-\text{exp}\left(-{\left(\frac{i}{a}\right)}^b\right),1-\text{exp}\left(-{\left(\frac{i_{\text{max}}-i}{c}\right)}^d\right)\right]$$ (19)

In all cases the total efficiency gain was constrained to have the same value for the polar and uniform conditions.

Retinotopic map

The retinotopic map was generated separately for each quadrant of visual space, and then the quadrants were combined. To generate the retinotopic map for, say, the upper right quadrant, we start with the assumption that the horizontal meridian lies along the horizontal row at the bottom of the map for that quadrant. The center of the fovea is at i = (0,0) and thus x(0,0) = (0,0). Starting from the fovea, retinal coordinates along the horizontal meridian are obtained sequentially using spacing function sp(x, y):

$$x(i+1,0)=x(i,0)+sp[x(i,0),0]$$ (20)

and

$$\text{y(}i\text{+1,0)=0}$$

We next use the constraint that the iso-eccentricity contours lie along vertical lines. Consider the vertical line touching the horizontal meridian at map location (i,0). Again, starting at this location, the retinal coordinates along the vertical line are obtained sequentially using the spacing function. Specifically, the retinal coordinate at map location (i, j + 1) must satisfy the equation

$${[x(i,j)-x(i,j+1)]}^2+{[y(i,j)-y(i,j+1)]}^2=s{p}^2[x(i,j),y(i,j)]$$ (21)

and thus,

$$y(i,j+1)=y(i,j)+\sqrt{s{p}^2[x(i,j),y(i,j)]-{[x(i,j)-x(i,j+1)]}^2}$$ (22)

Because the iso-eccentricity contours are assumed to lie on vertical lines

$$x^2(i,j+1)+y^2(i,j+1)=x^2(i,0)$$ (23)

Thus, x(i, j + 1) must satisfy the following equation obtained by substituting Equation 22 into Equation 23:

$$x^2(i,j+1)+{\left[y(i,j)+\sqrt{s{p}^2[x(i,j),y(i,j)]-{[x(i,j)-x(i,j+1)]}^2}\right]}^2=x^2(i,0)$$ (24)

After solving for x(i, j + 1) with numerical software, the value of y(i, j + 1) is obtained by substituting the solution for x(i, j + 1) into Equation 23. Maps were generated for spacing functions based the Drasdo¹⁷ and Watson¹⁸. The maps are very similar. Shown here are maps based on the Drasdo spacing function.

Biased foveated ideal (BFI) searcher

The BFI searcher is identical to the FI searcher except that it does not accurately represent the prior probability density p(x). We assumed that the biased prior probability p_b (x) is the true prior probability scaled by a non-negative function b(x) that depends only on the retinal eccentricity of image location x:

$$p_b(x)=p(x)b(x)$$ (25)

As with the ELF searcher we assumed that the bias function is described by a Weibull function of eccentricity with a minimum value of b_min, a maximum of b_max, a steepness parameter, and a shape parameter:

$$b(x)=b_{\text{min}}+\left(b_{\text{max}}-b_{\text{min}}\right)\text{min}\left[1-\text{exp}\left(-{\left(\frac{\Vert x\Vert }{a}\right)}^b\right)\right]$$ (26)

where b_min ≥ 0 and b_max ≤ 1.0. Maximum likelihood fits were obtained in two different ways. One was the same as for the ELF searcher: fitting all the data simultaneously (BF2). The other way was to emphasize the fit to the hits and misses (BF1). Finally, to expand the range of possible bias-function shapes we also obtained fits where the Weibull function multiplied the log of the prior probability (the fits were slightly better, so here we only show the fit for weighting the log prior).

• Human visual search accuracy measured for targets in large fields of noise texture

• Humans display surprisingly poor accuracy in and around the fovea

• This foveal neglect predicted by efficient allocation of attention in visual cortex