Sensory ecology and perceptual allocation: new prospects for neural networks

Abstract

Sensory ecology provides a conceptual framework for considering how animals ought to design sensory systems to capture meaningful information from their environments. The framework has been particularly successful at describing how one should allocate sensory receptors to maximize performance on a given task. Neural networks, in contrast, have made unique contributions to understanding how ‘hidden preferences’ can emerge as a by-product of sensory design. The two frameworks comprise complementary techniques for understanding the design and the evolution of sensation. This article reviews empirical literature from multiple modalities and levels of sensory processing, considering vision, audition and touch from the viewpoints of sensory ecology and neuroethology. In the process, it presents modifications of extant neural network algorithms that would allow a more effective integration of these diverse approaches. Together, the reviewed literature suggests important advances that can be made by explicitly formulating neural network models in terms of sensory ecology, by incorporating neural costs into models of perceptual evolution and by exploring how such demands interact with historical forces.

1. Introduction

All of animal behaviour can be considered a series of choices—at any given moment an animal must decide whether to mate, eat, sleep, fight or simply rest. Such decisions require estimates of the immediate environment, and despite the diversity of those estimates, they are all carried out by sensory systems and the neural functions contingent on them. This functional diversity is central to the concept of sensory drive (Endler 1992; figure 1), which notes that animal mating, foraging and other activities are evolutionarily coupled through their shared dependence on sensory systems and local environments. In light of the many demands made of a sensory system, what does it mean to design one well?

Figure 1.

Sensory drive depicts the coupling of multiple ecological factors with the design of sensory systems and its influence on interaction between organisms. Interactions corresponding to ‘sensory exploitation’ are depicted in grey. Based on Endler (1992).

It is often useful to consider how an ideal receiver would perform on a given task. Aside from the potentially conflicting demands posed by different aspects of one's environment, there are additional reasons to think that such an approach may not be complete. The climb to a global optimum can be a tortuous one, complicated by genetic drift, allelic diversity and phylogenetic history. Analytic models often focus on defining the best possible performance and neglect the existence of alternative local optima, or the ability to arrive at such optima through evolutionary processes. In sexual selection, researchers have suggested pleiotropy in sensory systems may be a key feature that shapes the direction of evolution (Kirkpatrick & Ryan 1991). Pleiotropy could emerge when building a complex structure from a limited number of genes or from the multiple functions fulfilled by a common structure. Female guppies, for example, prefer orange males, and they also prefer orange food items (Rodd et al. 2002). We need means of understanding how such coincident preferences emerge, and what their consequences are for behavioural evolution. In this review, I advocate a dual approach that combines first principles of sensory ecology with neural network models to gain a more balanced and nuanced view of sensory design.

Neural network models have been used to investigate the origins of hidden preferences—attributes of nervous systems that inadvertently bias their interactions with the outside world. Early studies used model visual systems, simple ‘feed-forward’ networks, to investigate how biases towards signals that were symmetric or exaggerated could emerge as a by-product of selection on simple recognition tasks (Enquist & Arak 1992, 1993; Johnstone 1994). Additional advances came from linking such studies to empirical data from particular species (Phelps & Ryan 1998; Phelps et al. 2001), permitting an assessment of the external validity of the models. These studies provided a broader view of sexual selection, a field that has historically focused on the strategic design of signals (e.g. Zahavi 1975). One insight from the neural network models was that preferences could emerge for stimuli as a by-product of selection in other contexts. Potential indicators of male condition were not necessarily favoured for their ability to reveal male status, but rather for their conformity to the pre-existing perceptual biases of the receiver (Basolo 1990), a phenomenon known as ‘sensory exploitation’ (Ryan 1990; Ryan et al. 1990b). Such complexity is not easily explained. Since neural network models simulate complex decision making, and arrive at those decisions through evolutionary processes, they provide a logical complement to more abstract models that describe performance at a global optimum.

This manuscript aims to briefly survey work on each of several modalities (vision, audition and touch) and levels of sensory processing (peripheral and central). Firstly, within each domain, I describe work in sensory ecology which takes broad models for how receivers ought to be designed and compares them with physiological and behavioural data. Secondly, I also examine work from neuroethology, which often has the same general assumptions but is not always focused on testing whether sensory performance is optimal for a given task. Lastly, in each section, I suggest novel neural network models that could further inform our understanding of perceptual allocation. By exploring what it would mean to design a sensory system optimally, one can test whether a given system conforms to those expectations. No less importantly, one can also detect deviations from predictions that might direct us to the roles of other evolutionary forces. By making these questions explicit, and by outlining how extant neural network models could be modified to ask such questions, this review aims to stimulate thought and experiment on how evolution shapes nervous systems to accommodate a multitude of tasks.

Before beginning our survey, it may be useful to review a few basic concepts in sensory design for readers unfamiliar with neuroscience. The first is to remind the reader that each sensory modality corresponds to a type of stimulus energy able to change the voltage of a sensory receptor. Within a modality, there are subtle variations of energy that include wavelengths of light, frequencies of sound, and the depth and the duration of touch. The investment in one modality or submodality over another is assumed to reflect its value. How does value interact with the costs of receiving and processing such information? Such questions are central to receiver design, and by extension to questions in foraging, mating and the various other choices which comprise an animal's behavioural repertoire. I begin our exploration with a discussion of empirical work on visual allocation at the periphery and how such allocation influences behavioural decisions.

2. Allocation at the periphery

(a) Vision

To what extent can a visual system be said to be operating ‘optimally,’ and in what ways may we estimate its efficacy? Some of the most sophisticated ecological analyses of animal sensation have focused on how the tuning and abundance of colour cones reflect evolutionarily important properties of the environment (see Osorio & Vorobyev 2005 for recent review). A central formalism in this work describes the ‘quantum catch’, the number of photons (Q) absorbed by a sensory system (Wyszecki & Stiles 1982)

$$Q=\int Q_0(\lambda )·R(\lambda )·T(\lambda ,d)·S(\lambda )\text{d}\lambda .$$ (2.1)

In this equation, Q₀(λ) is the distribution of coloured light in an environment, a value called the quantum flux and expressed as a function of wavelength (λ). This has been shown to vary as a function of the openness of terrestrial habitats, or depth of aquatic ones (Lythgoe & Partridge 1991; Endler 1992, 1993). It is shaped by both the distribution of wavelengths in sunlight, and by the reflectance and absorbance properties of organisms within the local environment. R(λ) is the reflectance pattern for an object to be detected—whether that object is a patch of coral or the epaulet of a red-winged blackbird. The product of these two represents the stimulus energy emerging from the target. T(λ, d) represents the transmittance of light in the environment at a distance d. Lastly, S(λ) is the spectral sensitivity of the photopigment of interest. The quantum catch of multiple kinds of cones can be calculated independently. These can be summed across cone classes to generate a measure of luminance, or differences between cones can be used to estimate chroma. Based on such calculations, one can visually sample local environments and calculate how well the visual system is equipped for detecting objects varying in brightness and hue. As a perceptual allocation problem, the aim is to understand how the organism should tune its eye to extract the most useful information.

Extracting chromatic information requires multiple cones tuned to different wavelengths. The simplest such system is dichromatic vision, found in most mammals, in which animals possess a pair of cones each tuned to a different wavelength. In contrast, stomatopods may possess eight or more narrowly tuned receptor types, providing the potential for fine discrimination of individual differences in body spot coloration (Cheroske & Cronin 2005). The eyes of Lycaena butterflies express four cone types; wing colour and receptor tuning vary by species, and sexes vary in the relative abundance of cone types across the eye. Females use long-wavelength cones to detect host plants and, in accord, this cone can be found in the dorsal portion of the female eye (Bernard & Remington 1991). Assessments of colour perception by bees have utilized knowledge of cone absorbance and colour opponency (combinations of cones used to perceive colour) to predict performance in foraging tasks. These studies demonstrate that chromatic contrast predicts rates of flower detection (Lunau et al. 1996; Spaethe et al. 2001).

Among fishes, the diversification of visual signalling is particularly interesting in the cichlid fishes of Lake Victoria, a group well known for its recent and explosive radiation. Closely related sympatric species differ substantially in sexually dimorphic coloration; environmental degradation has caused increased turbidity and a breakdown of visual barriers to hybridization (Seehausen et al. 1997). Cones differ substantially in their relative abundance across species (Carleton & Kocher 2001) and there is evidence of selection on opsin loci associated with variation in colour and habitat (Terai et al. 2002; Spady et al. 2005). Interestingly, there is also substantial phenotypic plasticity in receptor allocation. Halstenberg et al. (2005) demonstrate a diurnal rhythm in opsin expression that is entrained by light. Similarly, in killifish, Fuller et al. (2003, 2004, 2005) find both heritable and environmental variations in opsin distribution across killifish lineages occupying clear and tannin-stained waters.

Most mammals are dichromats, and so the evolution of trichromatic vision within primates is remarkable. The emergence of trichromacy has evolved in two distinct ways. First, in old world apes, the medium/long-wavelength opsin locus has duplicated and diverged, generating red- and green-responsive cones (Surridge et al. 2003). Visual samples of primate habitats combined with quantum flux calculations reveal that trichromacy substantially improves the detectability of both new leaves and ripe fruits (Sumner & Mollon 2000; Surridge et al. 2003). New leaves tend to reflect more red wavelengths; both leaves and fruits are more nutritious and less tough (Dominy & Lucas 2004). In both old and new world primates, trichromats are more likely to eat red-shifted leaves (Lucas et al. 2003). In new world primates, there is a surprising condition in which trichromacy has emerged not by duplication of a locus, but by allelic divergence within a locus. Some New World monkeys have as many as five different medium/long-wavelength alleles at a single locus (Jacobs & Deegan 2005). Using visual models and images of natural tamarin foods, Osorio et al. (2004) suggest both frequency-dependent selection and heterozygote (trichromatic) advantage in the evolution of such polymorphism.

(b) Modifying a visual neural network

The feed-forward network seems a promising tool for modelling complex scenarios of visual allocation. Enquist & Arak (1993) used a simple feed-forward network to discriminate between stimuli representing long-tailed conspecific and short-tailed heterospecifics, and observed emergent preferences for still longer tails. (A network similar to theirs is presented in figure 2a.) As one might predict, coevolutionary simulations result in the evolution of still longer tails. What variations on this architecture might be used to investigate sensory drive more broadly?

Figure 2.

Use of a feed-forward network to explicitly investigate sensory drive. The proposed architecture at top (a) is similar to that used by Enquist & Arak (1993). (b) The task is to require recognition of either (i) a conspecific or (ii) a food item using distinct output neurons, but not to respond to a stimulus consisting of only (iii) background illumination. (c) A hypothetical distribution of reflected light from targets and background, posing the neural network task in terms of sensory ecology. Note that the coding of colour is not specified, and represents a non-trivial attribute of this model.

The traditional feed-forward network used in these studies has an input layer corresponding to a retina that detects illumination, a hidden layer that extracts features from the pattern on the retina and a single output neuron that conveys whether a target pattern has been detected. Genetic algorithms code network architectures as lists of weights, and select those best at performing the desired task (Mitchell 1996). Instead of this original Enquist & Arak (1993) formulation, one might allow each pixel to have a distribution of reflectance intensities defined by the function, R(λ). The image falling on the retina is derived from the product of the reflectance intensities, the environmental transmittance T(λ) and the ambient light Q₀(λ) (figure 2b). Figure 2c shows distributions of reflected wavelengths (Q₀(λ)·R(λ)) for each of several potential views. The investigator could allow the sensitivity of the receiver S(λ) to wavelengths to vary genetically, perhaps assigning each spot in the retina one or more of several available cone classes. Such a model could also incorporate some formulation of colour theory, defining the networks as dichromats or trichromats and using existing models to extract hue and luminance (Wyszecki & Stiles 1982).

A second modification might allow networks to make multiple decisions. For example, a network could use separate output neurons to represent mating and feeding decisions (figure 2a). Would the processing of food-related cues compel signallers to evolve exaggerated versions of these stimuli? Such a finding would be consistent with sensory exploitation and sensory drive. It would also be strikingly reminiscent of the ethological notions of ‘ritualization’ and ‘emancipation’ whereby cues from one domain become exaggerated and freed from their original source, as suggested for the elaboration of pheasant trains (Bradbury & Vehrencamp 1998). These considerations begin to take us beyond peripheral allocation, into how sensory representations impinge on the decisions that comprise an animal's behavioural repertoire.

In short, such neural network models can readily be cast in terms of conventional sensory ecology. They provide valuable supplements to more general models because they facilitate the investigation of complex interactions between phenotypes and settings. Since parameters of interaction can be controlled precisely, they can be used to uncouple the components of sensory drive and to observe how these components interact to produce the higher level patterns that characterize behavioural evolution.

(c) Audition

Neuroethologists recording from the periphery of the auditory system find that its frequency sensitivity roughly matches the distribution of frequencies in species-specific acoustic signals (figure 3). Tuning that favours mating signals has been found in insects (Meyer & Elsner 1996), fish (Sisneros et al. 2004), frogs (Frishkopf et al. 1968) and songbirds (Konishi 1969; Dooling et al. 1971). Similarly, both bats and barn owls contain ‘acoustic fovea’ matching the echolocation calls used for navigation and hunting (Bruns & Schmieszek 1980; Vater 1982; Koppl et al. 1993). Capranica & Moffat (1983) point out that such tuning can be considered a ‘matched filter’, a receiver design that in many conditions serves as an optimal signal detector (Dusenberry 1992).

Figure 3.

Matched filters in auditory coding. (a) Conceptual example showing the relationship between the dominant frequency (DF) for (i) a noise-free call, (ii) a noisy call and the corresponding (iii) best excitatory frequency (BEF) in an auditory organ acting as a matched filter for the species-specific call; after Capranica & Moffat (1983). (b) Plot of DF and BEF for 36 species of anurans, from Gerhardt & Schwartz (2001). (c) Similar plot of nine species of grasshoppers, from Meyer & Elsner (1996).

The tuning curves from different species are obtained in a diverse number of ways. For example, in amphibians, these may include recording directly from the eighth nerve or from more central midbrain neurons; sounds may be broadcast as ‘free field’ stimuli or coupled to the auditory apparatus of the organism. Moreover, thresholds are estimates of the minimum stimulation required to evoke some fixed response from a population of neurons. They are an incomplete representation of the abundance and nature of neurons coding information about a given frequency. Owing to these ambiguities, researchers in the neuroethology of audition have focused on the relationship between the frequency to which the receiver exhibits the lowest threshold (the ‘best excitatory frequency’; BEF) and the dominant frequency (DF) of an advertisement call. The match between signal and receiver is surprisingly strong (figure 3). In a definitive review of matched filters in anurans, Gerhardt & Schwartz (2001) demonstrate that there is a very strong correlation between the DF of a call and the BEF estimated from neurophysiologic data; similar findings have been reported in orthoptera (Meyer & Elsner 1996; see Gerhardt & Huber 2002 for discussion). These data demonstrate a remarkable congruence between receiver perception and signal design, even in the absence of data concerning sources of background noise and signal degradation imposed by a given environment.

Following the ‘quantum flux’ model for visual observers, one can imagine a similar formalism for the sound energy (E) captured by a receiver.

$$E=\int C(f)·T(f,d)·S(f)\text{d}f.$$ (2.2)

In this equation, C(f) represents the spectral content of the signal as function of frequency (f), T(f, d) corresponds to the transmission of energy at a particular frequency (f) and distance (d) and S(f) corresponds to the sensitivity of the receiver to various frequencies. One can describe the information capacity of the frequency domain by measuring the difference between the total sound energy perceived by a receiver when a signal is present and when it is not. Defining the energy in the target plus background as C_T+B(f) and the background as C_B(f), one can describe the information content (H_T) of a signal received at a given distance as follows:

$$H_{\text{T}}={\text{log}}_2\left[\int C_{\text{T}+\text{B}}(f)·T(f,d)·S(f)\text{d}f\right]-{\text{log}}_2\left[\int C_{\text{B}}(f)·T(f,d)·S(f)\text{d}f\right].$$ (2.3)

As in vision, the efficacy of communication is contingent on receiver, signal and environment. In contrast to the studies of vision, there is little work that combines all three measures into a common assessment. Most studies have focused on the interaction of signal and environment, the two simplest parameters to measure. The acoustic adaptation hypothesis (Morton 1975; Wiley & Richards 1978, 1982), for example, predicts that signals evolve to minimize the degradation due to environmental differences between habitats. In equation (2.3), this is analogous to designing signals to maximize C_T+B(f)·T(f, d). A related idea, that of an ‘acoustic window’ (Waser & Waser 1977; Waser & Brown 1984) posits that signals should use frequencies least evident in the background, roughly equivalent to minimizing C_B(f). These ideas have certainly met with considerable success. Even after considering such factors, however, there is a great deal of variation in signal structure that is not readily explained by habitat. Ryan & Brenowitz (1985), for example, report that although natural variation in vocalization frequency is significantly predicted by habitat, body size is a far better predictor.

To consider how one might integrate perceptual allocation into considerations of transmission, environmental noise and call structure, it is worth considering the communication system of the cricket frog, Acris crepitans. Cricket frogs are small temperate frogs inhabiting the south and eastern United States. Studies by Ryan et al. have focused on two populations in Central Texas, Acris crepitans crepitans, which lives in wet pine forest, and Acris crepitans blanchardi, which lives in open habitats. Perhaps surprisingly, it does not seem that each call is specialized to minimize degradation within its own habitat. Instead, calls from the more acoustically challenging pine forest transmit with greater fidelity in both pine and open habitats (Ryan et al. 1990a). To examine population differences in receivers, Wilczynski et al. (1992) measured the auditory tuning of frogs and found that the average tuning of the basilar papilla is similar to the dominant frequency in each call (see also Capranica et al. 1973; Ryan et al. 1992). To examine the interaction of receiver tuning, call structure and transmission fidelity, Sun et al. (2000) recorded calls at varying distances from a speaker. They removed background noise from recordings, passed the calls through an acoustic filter approximating the auditory tuning curve for each population and estimated how well the receivers would be able to detect a call at short and long distances. Their results describe how the transmission environment can alter the ideal receiver tuning depending on the distance at which a signal is perceived. What is striking, however, is that receivers from the more challenging pine forest are much better in both habitats. Similarly, Witte et al. (2005) find that tuning curves from the pine forest animals are better at filtering equivalent intensities of background noise from either habitat. Combined with earlier data, it seems both signal and receiver from the pine habitat are more effective.

Although the cricket frog work uniquely assesses the interaction of auditory allocation, transmission environment and signal structure, Witte et al. (2005) note the importance of one missing variable, potential habitat differences in absolute background noise level. As background gets more intense, the relative difference between a sound sample containing the signal and one that does not get smaller (equation 2.3). An appropriate follow-up might measure calls without filtering out background noise, and use equation (2.3) to compare the sound energy received with and without the call present. If the information is no more valuable in the pine forest than in the open forest, and the pine forest proves to be louder, the total amount of information transmitted in the two systems may be equivalent. Indeed, if the costs of communication are higher in the pine forest, one may find that the net information transmitted has gone down despite the efficiencies of signal and receiver. This hints at the need to assess the costs of communication to both receiver and signaller, features rarely treated empirically (see Bradbury & Vehrencamp 1998, 2000).

I have emphasized the optimal design of receivers, and how they might interact with signallers. However, as mentioned in §1, matching of signal and receiver do not explain the entirety of receiver design. In auditory communication, there are a number of examples of poor matches between signallers and receivers (e.g. Schul & Patterson 2003). In the case of cricket frogs, though the pine forest animals have a peak frequency tuned to within 80 Hz of the dominant call frequency, the mismatch in open habitat animals is 740 Hz. Does this reflect other uses of the auditory system? Certainly it is possible. Auditory systems are likely means of detecting nearby predators, and the presence of frequency responses outside the range of mating signals is often interpreted as serving this function (e.g. frogs, Frishkopf et al. 1968; birds, Konishi 1969; insects, Schul & Patterson 2003). Receiver compromises between detecting mates and predators are in principle amenable to analytic treatments. Are there other, less readily explained patterns of peripheral tuning? In another anuran example, at first it seems that the basilar papilla of túngara frogs is tuned to roughly match the dominant frequency of a call component called a chuck. Congeners possess similar tuning, however, despite lacking the capacity to chuck altogether—a finding suggesting the tuning substantially predates the evolution of its ‘matched’ target (Ryan et al. 1990b; Wilczynski et al. 2001). In this example, the matching of signal and receiver can more easily be attributed to the evolution of the signal than to the design of the sensory system. Any complete understanding of sensory function and perceptual allocation must include the possibility that receivers depart from optimal performance due to the contingencies of history and genetic architecture. I now describe a neural network model of call recognition that enabled the exploration of such forces. As with the visual networks discussed above, these auditory networks model higher level decision processes as well as peripheral mechanisms of stimulus filtering.

(d) A simple auditory model

We began with the intent of understanding how historical and contemporary selection on species recognition mechanisms could produce patterns of mate-choice preference. This approach was motivated both by prior work emphasizing how the selection for simple recognition mechanisms could generate hidden preferences (Enquist & Arak 1993), and by data suggesting evolutionary history was a significant contributor to the responses of túngara frogs (Ryan & Rand 1995). Unlike prior neural network studies, however, we hoped to anchor the evolutionary simulations to a specific model system so that we could compare our results to data from behavioural studies.

We chose a simple recurrent network similar to an Elman net (Elman 1990), consisting of a frequency-specific set of input neurons, two reciprocally connected hidden layers (called feature detector and context layers, by analogy with an Elman net), and a single output layer (Phelps & Ryan 1998; figure 4). The reciprocal connections between hidden layers enabled neurons to make responses to current frequency inputs contingent upon prior patterns of stimulation. This struck us as the simplest possible abstraction of neural processing that could recognize a túngara frog call in a biologically realistic manner. We settled on this general model because the precise mechanisms underlying túngara frog call recognition are not known.

Figure 4.

Recurrent neural network used to model call recognition in túngara frogs. The network has a recurrent loop between the feature detector and context layers that allows the detection of time-varying stimuli. The stimulus is a matrix of Fourier transform coefficients from a túngara call presented to the input layer one time interval per step. From Phelps & Ryan (1998).

We next performed fast Fourier transforms to generate a time-by-frequency breakdown of the túngara call suitable for presenting to the networks. We trained networks using a genetic algorithm in which networks were represented as binary strings corresponding to a concatenated list of neuron weights and biases. A more complete discussion of the network architecture and the training algorithm is provided in Phelps & Ryan (1998) and Phelps (2001). During training, the calls were randomly placed within a time window large enough to house any of the potential test stimuli. We made a matching noise signal by randomly assigning the energies in a given time window to a new frequency. Additional noise was added to both stimuli by adding a small, fixed probability of being assigned a value at random. Networks were selected to distinguish between calls and noise according to a fitness function defined as

$$W={\left[\sum \frac{{(C_i-N_i)}^2}{n}\right]}^{1/2}+0.01.$$ (2.4)

In this equation, W is fitness, C_i is the network response to a given call i, N_i its response to a noise and n is the number of calls presented. The small constant 0.01 provides an external component of fitness which minimizes the chance that networks will get caught at early local optima. We found that not only could neural networks evolve to discriminate this call from noise, but also the generalizations they made to novel call stimuli were excellent predictors of the responses made by females in phonotaxis tests (34 stimuli, R²=0.88, p<0.001; Phelps & Ryan 1998). A surprisingly large amount of variation in female responses could be explained solely on the basis of selection for conspecific recognition.

Owing to data suggesting that historical forces had shaped responses of female túngara frogs, we next set out to manipulate histories of neural network models and observe the consequences on the emergent patterns of preference. To do so, we selected networks to recognize a call corresponding to the reconstructed ancestor of the túngara frog clade (figure 5). Once networks were reliably able to do so, responses to this call were no longer explicitly selected, and networks were selected to recognize the next node along the trajectory leading to the call of the túngara frog. The influence of past recognition mechanisms was immediately evident in the increasing ease with which networks evolved to recognize each new target call (Phelps 2001). To control the cumulative variety of target calls, we also generated a control history in which the trajectory was rotated in a multidimensional call space defined by a principal components analysis (PCA) of variation within the clade. This ‘mirrored’ history possessed an equal number of steps as the ‘mimetic’ history. Also, like the mimetic history, the mirrored history converged on the call of the túngara frog. We found that the two history types were equally able to recognize the call of the túngara frog, but differed substantially in how they generalized to other novel calls (Phelps & Ryan 2000). Assessing the pattern of responses across such novel calls, we found the mimetic history was significantly better than the mirrored history at predicting female responses, a finding consistent with the hypothesis that females harboured biases which were vestiges of their evolutionary histories (Phelps & Ryan 2000).

Figure 5.

Vestigial preferences in neural networks and túngara frogs. (a) Networks were given one of the two histories matched for the diversity of the calls in training set. The mimetic history consisted of a series of representing reconstructed ancestral nodes in the path leading to túngara frogs; the mirrored history is a control made by rotating this trajectory in a call space defined by PCA and synthesizing the resulting calls. Sonograms on right depict frequency on the y-axis and time on the x-axis for corresponding histories. (b) Mimetic history networks retain an ability to recognize calls with short time to half frequencies, resembling ancestors. Mirrored histories exhibit biases in the opposite direction (not shown). (c) Number of females (of 20) approaching calls synthesized to resemble a reconstructed ancestor (left of túngara), or control calls of comparable similarity to túngara frog calls (right of túngara). Real females respond significantly more often to ancestor-like calls than to controls that do not resemble an ancestor. Figure modified from Phelps et al. (2001).

To further assess the vestigial preferences, we constructed a series of stimuli that varied in only a single call parameter—the time to half frequency, a measure that varied significantly between mimetic and mirrored histories. The resulting stimuli were different from training stimuli, yet networks exhibited clear asymmetries in their patterns of preference, favouring calls resembling those of ancestors to those that did not (Phelps et al. 2001; figure 5b). To examine females for comparable markers of vestigial preferences, we constructed a series of stimuli that varied along a dimension which ranged from the extant túngara call to the reconstructed call of an ancestor in one direction, to equidistant calls that did not resemble an ancestor in another direction. We tested such calls in both one- and two-choice phonotaxis experiments. We found that females, like the neural networks, exhibited strong preferences for calls resembling those of ancestors (figure 5c).

Despite their simplicity, the neural network models were able to demonstrate how history and species recognition could interact to produce a complex pattern of responses in an extant species. One can imagine supplementing the extant studies with the sort of modifications described for feed-forward networks—assessments and manipulations of the ambient noise or call transmission, or the performance of multiple tasks. I suggest, however, a novel set of parameters often omitted from sensory ecology, that of perceptual cost.

(e) Incorporating costs

Since perceptual resources are limited, organisms must decide how much they will invest in information processing, and how to allocate those resources across alternative tasks. Consider two extreme examples: the energy budget attributed to the human brain is 20% of basal metabolic rate (Clarke & Sokoloff 1999); and for a mormyrid electric fish, it is 60% of basal metabolic rate (Nilsson 1996). Within mammalian nervous systems, Ames (2000) estimates approximately 5–15% of this budget is attributable to maintaining neuron function, approximately 30–50% to maintaining and recycling the contents of synapses and approximately 50–60% to maintaining ion gradients needed for initiation and propagation of voltage changes.

Fortunately, each of these major categories has an obvious and quantifiable analogue in neural network models. Neuron maintenance costs can be assigned by multiplying the maintenances cost per neuron by the number of neurons in the network (c_mn) or, if neuron number is not allowed to vary, the number of neurons active in the task. The costs of maintaining synapses can be assigned by summing the values of weights across the network ($\sum \sum w_{ij}{c}_{\text{w}}$, where w_ij is the weight between a pair of neurons i and j and c_w is the cost per unit of synapse). If one keeps track of the output of each neuron in each time-step, one could assign costs based on the activity of each neuron as well ($\sum \sum c_{\text{s}}(a)$, where c_s(a) is the function describing how signalling costs accrue as a function of activity, summed across neurons and time-steps). In fact, it seems as if costs of perceptual allocation could be modelled using extant neural networks simply by altering fitness functions. Using the example given for the túngara frog network, the modified fitness function would simply be

$$W={\left[\sum \frac{{(C_i-N_i)}^2}{n}\right]}^{1/2}-\text{costs}+0.01={\left[\sum \frac{{(C_i-N_i)}^2}{n}\right]}^{1/2}-\left(c_{\text{m}}n+\sum \sum w_{ij}{c}_{\text{w}}+\sum \sum c_{\text{s}}(a)\right)+0.01.$$ (2.5)

The function c_s(a) describing how costs increase with firing rate merits special attention, because it lies at the heart of work on the design of efficient coding schemes. Laughlin et al. (1998) point out that the relationship between energy use and information coding (bit rate) is not linear. Instead, energy use is an accelerating function of bit rate—at high bit rates, doubling the amount of information encoded more than doubles its cost. As a result, it is often more efficient to parse the coding of information between two neurons rather than sustain the high costs of coding the same information in a single neuron. Indeed, this is a general feature of intensity coding (Gardener & Martin 2000). In the auditory system of anurans, for example, many of the frequency thresholds shown in figure 3b correspond to the minimum intensity needed to elicit some basal level of population-level activity. An examination of the firing thresholds for individual neurons reveals that the intensity of a given frequency is coded by a distributed set of neurons that vary in their individual thresholds (e.g. Konishi 1969; Capranica & Moffat 1983). If efficient coding is a major concern of sensory design, neural networks could prove invaluable tools for its exploration.

3. Central allocation

Sensory information does not, of course, end at the periphery. Sensory stimuli must ultimately reach brain regions that combine sensory and affective information to influence behaviour. A comprehensive review of higher levels of sensory processing or its intersection with downstream decision mechanisms is well beyond our current aims. Even a broad survey of classic literature, however, reveals a remarkably congruent picture of perceptual allocation.

Sensory receptors coming from a peripheral organ converge on relay sites, each of which transforms the information into more useful forms. These relays and the higher representations they feed into exhibit a topographical representation of sensation. In other words, adjacent neurons represent similar domains of sensory space. Notably, the configuration of a map does not faithfully reproduce the structure of the sensed world, but is distorted in favour of certain highly sensitive and highly important areas. The primary visual cortex, for example, consists of a series of columns of neurons that arrange visual information by eye, position and orientation. The representation of the fovea, however, is far in excess of the size of the corresponding visual field. Similarly, in moustache bats, the primary auditory cortex exhibits an inflated representation of 60–62 kHz sounds, a range corresponding to the echolocation calls used to navigate and hunt (reviewed in Covey 2005). I now discuss a well-studied representation, that of somatosensation, or touch.

(a) Touch

The mammalian somatosensory cortex has been a long-standing subject for neurophysiologists. Marshall et al. (1941) recorded the activity of neurons in somatosensory cortex in response to touching different regions of an animal's body surface. Penfield and colleagues demonstrated that such maps were causally related to perception by asking epileptic patients, whose cortices were being probed to uncover the foci of seizures, what they felt in response to electrical stimulation at different sites (Rasmussen & Penfield 1947). The resulting maps depict a homunculus distorted by the density of sensory innervations in distinct body parts (indeed, we now know there are multiple such maps; Kaas et al. 1979).

It is now accepted that topographic maps are not static representations of the periphery, but are gradually modified by experience. Jenkins et al. (1990) trained monkeys to perform a task that involved touching a disk with their fingertips; over the ensuing months the representation of fingertips grew at the expense of the adjacent phalanges. String players display an enlarged representation of left but not right fingers, and such changes are correlated with the ages at which musicians began playing (Elbert et al. 1995). Even the nature of the topography can be modified by experience. Adjacent fingers are next to one another, for example, but somatosensory maps contain sharp divisions between neurons of one finger and those of another. A notable exception occurs when patients suffer from congenital syndactyly, in which fingers are fused. Surgically freeing the digits causes the individuation of the representations of the formerly joined fingers (Mogilner et al. 1993).

Topographical representations and their distortions reflect the underlying ability of an organism to resolve physical differences between stimuli. This relationship is born out by studies of species diversity in the organization of somatosensory cortex. Welker and colleagues, for example, demonstrated that coatimundis, which have relatively sparse representation of the front paw in the somatosensory cortex, have less digit dexterity than the related and more elaborately represented raccoon (Welker & Seidenstein 1959; Welker & Campos 1963). More dramatic, perhaps, is a series of studies on the star-nosed mole, Condylura cristata, an insectivore which possesses an elaborate star-shaped mechanosensory organ on the tip of its nose. The organ appears to be an elaboration of a basal mole pattern in which the rostrum is specialized for touch (Catania 2005). The high surface area of the star is combined with a fovea-like sensitivity of the central appendages (Catania & Kaas 1997). This peculiar arrangement facilitates capture of small prey on the order of 230 ms, a handling time that makes even tiny items surprisingly profitable (Catania & Remple 2005). This high value for mechanosensory input is accompanied by an enlarged representation of somatosensory cortex, and a disproportionate amount of this region allocated to the central appendages of the star (Catania & Kaas 1997; Catania & Remple 2005).

(b) Neural network models of topographic maps

It may seem at first that such complexity is beyond the scope of immediate modelling efforts in animal behaviour. Although the rules by which the nervous system assembles maps of sensory space are an active area of investigation (for recent review of somatosensory cortex, see Feldman & Brecht 2005), there are a number of surprisingly simple algorithms appropriate for modelling the emergence of such maps. Such rules tend to focus on changing weights based on patterns of correlation between neurons (‘Hebbian’ rules, after Hebb 1949), or on competitive interactions between neurons (e.g. Kohonen 1982; Kohonen & Hari 1999). Hebbian mechanisms have long been the focus of physiological work on map assembly (e.g. Bear et al. 1987). Competitive learning methods, in contrast, are the focus of more computational work and less physiological work; their mechanisms are biologically feasible, however, and seem likely to be a component of natural map formation. Since one type of competitive learning map, Kohonen's self-organizing map (Kohonen 1982; Kohonen & Hari 1999) is readily available in a number of software packages (e.g. MatLab), I will briefly describe how it could be used to investigate perceptual allocation. The same questions could be asked of Hebbian maps as well.

Kohonen's self-organizing map (SOM) commonly begins with an array of neurons which randomly weight each dimension of an input vector. The dimensions of this vector can be considered the activities of a set of input neurons. In each round of training, the map neuron which is most active in response to an input pattern, the ‘winner’, increases each of its weights in proportion to the activity of each input (figure 6), thereby increasing its response to the same input on future presentations. The winning neuron influences adjacent neurons to perform an analogous update, and the magnitude of the modification declines with the distance from the most active neuron. This simple procedure produces a two-dimensional map of a complex, multidimensional input, in which the resulting topography reflects underlying similarities between input patterns. In one example, maps trained on short-time spectra of Finnish language sounds produce an ordered map of phoneme structure (figure 6; Kohonen 2003).

Figure 6.

A self-organizing feature map trained by Kohonen's winner-take-all algorithm. Graphs depict frequency contours over short time-frames (approx. 100 ms) for Finnish phonemes. (a) Points along the contour served as inputs to a map. (b) In a trained map, adjacent neurons map similar space. The phoneme in (a) is most similar to the ‘winning’ neuron at the centre of the concentric grey circles. In a training phase, the winning neuron would cause the neighbouring neurons to update their weights to be more similar to the winning weights. This neighbourhood function drives the map's ability to topographically map complex stimuli. Figure modified from Kohonen (2003).

Studies of these maps reveal interesting patterns. First, as in Hebbian learning, the maps tend to be biased towards the most common types of input patterns. Interestingly, the magnification of the representation is found to correspond to the frequency of a pattern raised to a power (Haykin 1994). That power typically varies from 1/3 to 1, indicating that as a rule, the number of neurons that represent a feature is an increasing but decelerating function of its frequency of occurrence (Kohonen & Hari 1999). How can one relate such parameters to natural patterns of allocation? I have mentioned that innervation density at the periphery tends to predict cortical area and behavioural discrimination. What is the quantitative nature of such relationships and how are they modified by mechanisms of map formation? How do changes in peripheral allocation alter the ability to represent information at higher levels? Such questions seem central to linking sensory drive to the compatible traditions of neuroethology.

In order to ask such questions, one must first identify a metric that describes how well the map preserves information relevant to identification of ecologically relevant patterns. This is not a trivial task, but assume for the moment that has been done. How could one incorporate it into evolutionary models? Using a genetic algorithm, one can encode the relevant learning parameters, the size of the map and perhaps the distribution of responses at the periphery (the ‘innervation density’). Based on a set of inputs, one could train networks with these parameters to produce a map of the input space. Using a fitness function analogous to the one described for peripheral allocation, one could select networks to form efficient representations that preserved meaningful distinctions from within that input space. How would the mechanisms of map formation—Hebbian, competitive or otherwise—interact with the demands of efficient coding to produce higher level allocations? Are there regularities one can predict between the costs of representation and the magnification of peripheral inputs? Can such considerations help us understand why some animals have multiple maps for complex representations, while others have lost all but the lowest level maps (e.g. the lack of secondary auditory and visual cortex in the brains of shrews, Catania 2005)? Such questions seem highly relevant to receiver design, but are unlikely to merit serious treatment by biologists focused on understanding human brain function.

4. Common themes, uncommon opportunities

This review has touched on a broad range of examples from both empirical and modelling studies. Running through this diversity are common threads worth articulating. The first is that the principles of sensory ecology have proved remarkably predictive of natural variation. This power has the limits of any theory that describes how animals ought to be designed—selection can only act locally, and the topology of the fitness landscape is likely to be influenced by the sub-optimal contingencies of genetics and history. A second thread is that treatments of receiver design need to explicitly address the costs of perceptual resources. Such costs are implicit in analytic models but are rarely dealt with directly. Beyond these themes are a number of more specific commonalities.

In principle, meaningful information could be extracted at either early or late stages of processing. Nevertheless, the periphery does attend selectively to certain kinds of information. This is evident not only in the peripheral tuning of sensory systems, but also in electrophysiological studies which demonstrate that biologically relevant stimuli are often more faithfully represented than arbitrary stimuli (e.g. Machens et al. 2001). Given that the allocation of neurons to higher order representations is shaped by the distribution of resources at the periphery, the earlier in the processing stream one can isolate information likely to be meaningful, the greater the efficiency of emergent representations. As knowledge of peripheral sensory structures becomes more advanced, more explicit and more elaborate evolutionary models for such processing seem to be on the horizon.

The efficacy of a nervous system must ultimately be measured by its ability to make decisions that contribute to an organism's fitness. In sensory ecology, as in the communications theory upon which much of the field is based, this is measured as an attempt to discriminate meaningful from unmeaningful stimuli. Dusenberry (1992) notes that there are multiple such measures, each having a different formulation, but all maximizing the ratio of the likelihood the stimulus is present to the likelihood it is not. In signal detection theory, this decision corresponds to a threshold (which might be further weighted by the values of various kinds of decisions, as by Green & Swets 1966). In information theory, this would likely take the form of the logarithm of this ratio, as in equation (2.3). In particular applications in sensory ecology, similar measures may be used without explicit reference to such theory. In an elegant study on dichromatic marine fishes, for example, Cummings (2004) measures the distribution of colours in the environment and in food-laden corals; using quantum flux calculations, she finds that for multiple species the receptor tuning is ideally suited to discriminate a coral target from background lighting. The same conceptual framework can describe the fitness functions of most evolutionary neural network studies. In our túngara frog example, we selected a network to maximize its output in response to a túngara call, and minimize its response to background noise. This is synonymous with maximizing how much information the output of the network conveys regarding the presence of a conspecific call. Similarly, in an elegant study on the coevolution of model signallers and receivers, Hurd et al. (1995) demonstrate that visual signals and feed-forward neural networks coevolve to maximize signal discriminability. Darwin's ‘principle of antithesis’ and the ethological ‘sign stimulus’ both emphasize the tendency of signals to evolve to extreme, easily detected forms (Darwin 1872; Tinbergen 1951). The ability to discriminate among biologically meaningful categories can be formalized in many ways, but the common crux is that detection is valuable, and receivers are active investors in information acquisition.

Thus far, discussion of perceptual allocation has focused on the ability to detect a target within a background noise—certainly a critical task, and the focus of much study in both sensory ecology and network evolution. However, it is worth noting that not only can there be multiple targets, as exemplified by the numerous mate choice and feeding examples discussed, but also there are tasks not readily described as target detection. A number of studies have, for example, investigated the tuning of visual receptors for wavelengths abundant in the natural environment, a task referred to as ‘luminance detection’ (Wyszecki & Stiles 1982). For example, the need to navigate in one's surroundings requires the ability to detect abundant and variable stimulus energies. Such background sampling is often at odds with a simple ‘matched filter’ strategy. A complete examination of perceptual allocation will need to address how background and target detection are accomplished simultaneously. The allocation of resources to potentially conflicting tasks is interesting in its own right, and more so when coupled to the interacting participants in sensory drive.

Another limitation of our first-order description of perceptual allocation is that not all tasks will have equal value. The optimal allocation to a task must balance the cost of each additional neuron with the value expected from the information gained. The most effective representation does not simply maximize the resolution of frequent stimuli—it maximizes the expected value of that information, which is the product of its frequency and value. Returning to the example of somatosensory cortex, over a lifetime the glans of the penis is not likely to receive much more frequent use than a corresponding area of skin on the adjacent body surface, and yet, for obvious reasons, its uses contribute much more to fitness, and its innervation density reflects that value. It is interesting to note that the importance of value on perceptual allocation is currently an exciting topic in sensory neuroscience. For example, playing tones to rats while stimulating ascending projections from the ventral tegmentum, a region encoding positive value, enhances the representation of those frequencies in auditory cortex (Bao et al. 2001; see also Kilgard & Merzenich 1998). Similarly, Suga and colleagues have used bats to show that making tones more meaningful by linking them to negative reinforcement causes an expansion of their representation in early auditory centres (Gao & Suga 2000). The view from behavioural ecology suggests that this is likely to be a rather general process.

Sensory ecology and neural network models have been successful in isolation. The underlying unity in their concern with sensory design suggests how neural network studies could gain from the clarity provided from the mathematical methods of sensory ecology. In turn, sensory ecology could stand to be sullied by the complexities of evolutionary processes captured in genetic algorithms and elaborate receivers. These complementary approaches make it possible to titrate the compromises faced by model organisms—to titrate the costs, histories and tasks to observe the complex interplay of ecological and evolutionary forces. The resulting hybrid promises not dilution and extinction, dilution and but diversity and rigour. By exploring the strengths and the weaknesses of sensory ecology through the neural network models, we are likely to gain a deeper understanding of its power, limits and its ability to direct novel experiments in a diversity of taxa.