Information socialtaxis and efficient collective behavior emerging in groups of information-seeking agents

Significance

From bacteria to humans, individual foraging has been a key example of complex behavior, often analyzed experimentally and theoretically, in terms of a trade-off between exploration and exploitation. Finding a source in a nonturbulent environment can be done by following the gradient of a signal, but in noisy environments, moving to maximize information about the source results in efficient searches. We present a model for group foraging, in which each agent aims to maximize its own information and minimize its informational overlap with others. The result is a highly efficient, robust, biologically plausible, and nearly optimal collective group behavior. This suggests information maximization from sensory and social sources as an optimization principle that can direct individual and group behavior.

Abstract

Individual behavior, in biology, economics, and computer science, is often described in terms of balancing exploration and exploitation. Foraging has been a canonical setting for studying reward seeking and information gathering, from bacteria to humans, mostly focusing on individual behavior. Inspired by the gradient-climbing nature of chemotaxis, the infotaxis algorithm showed that locally maximizing the expected information gain leads to efficient and ethological individual foraging. In nature, as well as in theoretical settings, conspecifics can be a valuable source of information about the environment. Whereas the nature and role of interactions between animals have been studied extensively, the design principles of information processing in such groups are mostly unknown. We present an algorithm for group foraging, which we term “socialtaxis,” that unifies infotaxis and social interactions, where each individual in the group simultaneously maximizes its own sensory information and a social information term. Surprisingly, we show that when individuals aim to increase their information diversity, efficient collective behavior emerges in groups of opportunistic agents, which is comparable to the optimal group behavior. Importantly, we show the high efficiency of biologically plausible socialtaxis settings, where agents share little or no information and rely on simple computations to infer information from the behavior of their conspecifics. Moreover, socialtaxis does not require parameter tuning and is highly robust to sensory and behavioral noise. We use socialtaxis to predict distinct optimal couplings in groups of selfish vs. altruistic agents, reflecting how it can be naturally extended to study social dynamics and collective computation in general settings.

Foraging is fundamental for survival and reproduction in numerous species and is often based on sparse and noisy sensory cues in complex environments. Understanding how organisms represent space and navigate and the utility functions and computations that underlie movement patterns and social behavior (1–6) critically depends on characterizing the nature of foraging. Theoretical models of foraging have been pivotal in economics, physics (7, 8), and machine learning (9) as a general framework for decision making, exploration and exploitation, optimization, and learning (10).

Climbing the gradient of a sensory signal is an efficient foraging strategy for finding a strong source in nonturbulent environments or on a microscopic scale, as demonstrated, for example, by theoretical and experimental studies of bacterial chemotaxis (1, 11). However, for weak sources, or on a macroscopic scale that is characterized by turbulent flows, the signal coming from a source is typically broken into random, sparse, and disconnected patches (12). Without a continuous gradient, a different strategy is needed to read and use information from these patches about the location of the source (13). Vergassola et al. (14) presented a foraging algorithm based on maximizing the information that an agent has about the location of a source and showed that it is highly efficient in finding a sparsely signaling source in a turbulent environment. These infotaxis search trajectories resemble the paths of foraging insects, suggesting a biological implication of this model.

In nature, many species, including bacteria (15), amoebae (16), insects (17, 18), fish (19), birds (3), and mammals (20, 21), display complex group behavior, including foraging. The mechanisms that underlie group behavior have been of great experimental and theoretical interest, focusing on the computation that each individual performs (22–24) and emergent collective behavior (24, 25). In terms of social behavior, such models have been used to describe exploration and exploitation of the environment, finding food or mates, decision making (26–29), and altruistic and opportunistic behavior, as well as the design of systems of conflicting components (30) and distributed algorithms (31).

We present a model of foraging in a group that combines infotaxis and social interactions, using information as a unifying principle. In this socialtaxis model, agents seek a signal-emitting source, using the information they receive through their own “senses” and information they receive from other agents or that they infer from the behavior of their conspecifics. We show that certain forms of socialtaxis allow a collection of opportunistic individuals to find the source considerably faster than a collection of independent agents and comparable to the case of agents sharing all their sensory information and decision making using a centrally managed strategy (which is exponentially more expensive). Moreover, we find that coordinated group behavior emerges in a collection of selfish individuals each maximizing their own goal. We further present biologically plausible versions of socialtaxis, in which agents share little or no information, and its robustness to various forms of noise. Finally, we show socialtactic optimal behaviors in game theory-like settings of competing and opportunistic agents, suggesting a wide range of applications for this model.

From Individual Infotaxis to Socialtaxis

The infotaxis algorithm (14) describes a strategy for individual foraging in space, where an agent (or animal) moves to maximize the information it has about the location of a stationary source that emits signals (or particles) in random directions at a low rate. In its discretized version, the search space is divided into “cells,” and the agent estimates at each time $t$ a probability map of the possible locations (r) of the source $P_t(\text{𝐫})$. The entropy of the map at time $t$, given by $S\left[P_t(\text{𝐫})\right]=-{\mathrm{\Sigma }}_{\text{𝐫}}{P}_t(\text{𝐫}){\log }_2{P}_t(\text{𝐫})$ (32), measures the uncertainty or missing information about the location of the source, in bits. At each time point, the agent may (i) find the source, (ii) detect a signal from the source (“hit”), or (iii) detect nothing. The agent then updates its probability map over the space based on one of these three events and uses the new map to decide whether to move to a neighboring cell or stay still. This decision is based on the expected information it would gain from each possible move, which is given by the expected change in entropy of each of the possible events in that location (finding the source, getting a hit, or not getting a hit), weighted by the probability of occurrence of each event (Materials and Methods). Fig. 1A shows the trajectories of two agents who are independently searching the source using infotaxis in a 2D arena, completely ignoring one another (Movie S1). The paths of these independent agents are typical of individual infotaxis searches (14), resembling trajectories of insects in nature.

Fig. 1.

Characteristic trajectories of pairs of agents searching for a source under different social interaction regimes. Shown are examples of pairs of agents, searching for a source located in the middle of the arena (purple square). Agents begin their search at the top left corner (purple triangles), and their trajectories are marked by blue and green paths, which are becoming lighter with time along the search; the circles on top of the trajectories indicate positions where hits were obtained (main text). The grayscale background represents the mean detection rate, decaying exponentially at large distances. (A) Independent agents, each performing infotaxis and ignoring one another. (B) Distance-dependent socialtaxis agents (Eq. 2) remain close to each other and are slightly more efficient than independent agents. (C) Information-based socialtaxis agents (Eq. 3) start off in different directions, seemingly dividing the search space and then quickly converging, resulting in highly efficient searches. (D) Fully cooperating pairs, who share their hits and coordinate their movements using a joint infotaxis algorithm (Materials and Methods), are highly efficient at finding the source.

If there are other agents (animals) around, then in addition to its own sensory information about the source, each agent may use information it receives from conspecifics or infers from their behavior. We therefore extended the infotaxis algorithm to consider the behavior of groups of agents. Here, each agent moved to minimize a combination of its own entropy (as in infotaxis) and some function of its relation to other agents. Thus, rather than minimizing entropy, each agent minimized a “free energy,” given by

$$F_i=S[P_i]+\beta G_i(\text{group}),$$ [1]

where $S[P_i]$ is the entropy of the probability map $P_i$ of agent $i$, as in the single-agent infotaxis above, $G_i$ is a social interaction term between agent $i$ and the rest of the group, and $\beta$ weighs the balance between the individual’s information and its coupling to others. The nature of this socialtaxis model is determined by $G_i$, or how individuals affect one another, and $\beta$, the coupling strength or inverse “temperature,” which balances individual information and social interactions: $\beta =0$ reduces socialtaxis to the standard individual infotaxis behavior of each agent, and $\beta \to \mathrm{\infty }$ implies pure socially driven behavior.

Following many models of collective behavior, a natural choice for the relation between individuals would be one that is determined by the physical distance between them. We consider here the case where the free energy of agent $i$ is given by

$$F_i^{dist}=S[P_i]+\beta {\mathrm{\Sigma }}_j{d}_{ij}^{\alpha },$$ [2]

where $d_{ij}$ is the Euclidean distance between agents $i$ and $j$, $\alpha >1$ is a tunable parameter that determines the scaling of the distance between agents, and the summation goes over all group members. We explored the behavior of groups of 2–20 agents searching for a single sparse source in a 2D arena and found that for each choice of $\alpha$, there was an optimal $\beta$ that was always positive and minimized the time needed for the first member of the group to find the source (Distance-Based Socialtaxis). However, even for the optimal values, the improvement in search time compared with independent agents was very small (more below). Fig. 1B shows an example of the cohesive trajectories of two coupled agents with their optimal ${\beta }_{opt}$ (Movie S1).

We then considered a socialtaxis model in which social interactions were based on the information that agents have and not their physical locations. Here each agent aims to minimize the combination of its entropy and the overlap in the information it has in common with the other agents,

$$F_i^{info}=S[P_i]-\beta {\mathrm{\Sigma }}_j{D}_{KL}[P_i||P_j],$$ [3]

where $P_i$ and $P_j$ are the probability maps of agents $i$ and $j$, and $D_{KL}$ is the Kullback–Leibler divergence (32) between probability maps, which measures in bits the dissimilarity between the distributions. For positive $\beta$ this social term would drive agents to make their probability maps different from one another (constrained by the individual infotaxis term), whereas for negative $\beta$ the social term would drive agents to merge their knowledge and make their individual maps more similar. Fig. 1C shows the trajectories of two agents under this model with strong coupling $\beta =10$, reflecting their seemingly diverging yet highly time-efficient paths toward the source (Movie S1). The agents’ trajectories are qualitatively different from the independent or distance-based algorithm and are similar to the trajectories of fully cooperating agents who use a joint probability map based on their joint hits, and individual movements are determined by a joint infotaxis strategy (Fig. 1D, Movie S1, and Materials and Methods).

We simulated the different socialtaxis models, under different parameter values, each with 3,000 randomly chosen initial configurations of the agents and the location of the source (to avoid any effects of specific choice of configuration, see also ref. 33). As the example trajectories in Fig. 1 reflect, the distance-based socialtaxis enabled the agents to find the source a little faster than in the case of independent agents, but they were much slower than the fully cooperating agents (Fig. 2A), even with an optimal choice of ${\beta }_{opt}$, which we had to find numerically. A spring-like interaction between agents that drove them toward a nonzero distance gave qualitatively similar results, whereas negative $\beta$ values, typically resulted in failure to find the source at all.

Fig. 2.

Socialtaxis efficiency becomes optimal with strong information social coupling. (A) Mean search time vs. coupling strength $\beta$ for two foraging agents using either information-based socialtaxis (red) or a distance-based one (purple, $\alpha =4$). Search times are shown for two independent (gray) and fully cooperating agents (blue), with search times rescaled such that the search time of two independent agents is set to 1. Results shown are averaged over $3$,$000$ randomly chosen configurations of the initial positions of the agents and the source; shaded regions and error bars represent SEM. Negative $\beta$ values led to failed searches. (B) Average distance between agents vs. time for different coupling strengths $\beta$. (C) Average distance between the source and the agent that finds it vs. time. Results in B and C are averaged over $1$,$000$ simulations of two agents, with the source in the center and agents starting near one another, close to one of the corners. (D) Trajectories of two independent infotactic agents are overlaid with marking of the points where a socialtaxis agent with the same information would choose a different direction; the direction of the socialtaxis agent is shown by an arrow; other details are as in Fig. 1.

In clear contrast, the information-based socialtaxis showed a clear and abrupt decrease in search time compared with that of independent agents (Fig. 2A). Importantly, the performance of agents improved monotonically as $\beta$ increased, and with large $\beta$ this model was on par with the case of fully cooperating agents. This optimal behavior did not require any tuning of model parameters, which we show analytically (Analytical Analysis of the Functional Dependence of the Search Times on Coupling Strength). Notably, negative $\beta$ values, which intuitively might seem to enable agents to average their information and filter noise, resulted in failure to find the source in almost all cases (compare Movie S2 for $\beta >0$ and Movie S3 for $\beta <0$).

The nature of trajectories of the agents in the information-based socialtaxis delineates its efficiency: Because of the social information term, agents “repel” each other early on, more strongly as $\beta$ is increased (Fig. 2B), which leads to an effective division of the search space. Later in the search, the social interaction is constrained by the individual knowledge, which results in faster and more accurate convergence toward the source. Accordingly, information coupling between agents reduced the characteristic long “overshoots” in the initial part of the trajectories seen in standard infotaxis that take them away from the source ($t\in [70,100]$ in Fig. 2C). This is further reflected by comparing the individual locations along the path, where infotactic and socialtactic agents would move in different directions (Fig. 2D). Notably, information-based socialtaxis is robust to different kinds of variations, such as the starting distances from source, the size and shape of the arena, strength of the source, and decay length of the signal from the source (Socialtaxis Is Robust to a Multitude of Environments and Wrong Modeling of the Environment and Fig. S1), as well as different forms of sensory, odometry, or computational noise (Figs. S2 and S3). It was also highly efficient in reducing search times in a 3D space compared with the case of independent agents (Fig. S1E).

Fig. S1.

Robustness of socialtaxis to various environmental parameters. (A) Mean search time vs. emission rate of the source $R_{real}$. (B) Mean search time vs. the correlation length of the source ${\lambda }_{real}$. (C) Mean search time vs. arena size. (D) Mean search time vs. emission rate in the model held by the agent $R_{model}$ when the real emission rate is $R_{real}=1$. (E) Mean search time vs. the correlation length of the source in the model held by the agent ${\lambda }_{model}$, when the real correlation length is ${\lambda }_{real}=50$. All results are for two agents with the source located in the middle, and the agents start close together on a circle of radius $R_{st}=0.4L$, with $L=110$ the arena size. Gray curves correspond to infotaxis agents and red curves to socialtaxis agents. (F) Mean search time vs. the coupling strength $\beta$ for three agents in a 3D search space. The search times for three independent agents and three cooperating agents are shown for comparison.

Fig. S2.

Effect of noise on independent and coupled agents. (A) Mean search time as a function of sensory noise, measured by the probability to miss a hit, for two independent and strongly coupled agents, with $\beta =10$, averaged over 3,000 random starting configurations. (B) Mean search time as a function of odometry noise. This noise is modeled as a perturbation in an agent’s updating of its probability map: At each update the agent’s estimate of its own location is randomly shifted from its true position within a square, whose side length is given by the “self-position error.” (C) Mean search time for two agents who share inaccurate information with one another. Here Gaussian white noise was added to the probability of each cell, after which the map was normalized. The noise was drawn from a normal distribution with mean zero and SD ${\sigma }_m{N}_r$, where ${\sigma }_m$ is the mean probability of all cells, and $N_r$ is the “relative probability error.” (D) The same as C, but using the Gaussian approximation scheme (Fig. 4C), with the peak position randomly shifted within a square, whose side length is given by the “maximal peak shift.”

Fig. S3.

Agents with different models of the environment. One of the agents had the correct model of the environment $R_1=R_{real}=1$, whereas the other one had a wrong model for the source emission rate $R_2$. Results are averaged over 3,000 random starting configurations, and error bars indicate SEM. (A) Mean search time vs. $\beta$ for different values of $R_2$. (B) Probability for the agent with the wrong model to find the source vs. $\beta$, for several values of that agent’s model emission rate $R_2$.

Distance-Based Socialtaxis

Extending the analysis of the distance-based socialtaxis introduced in Eq. 2 of the main text, we present in Fig. S2 the mean search time vs. $\beta$ for different values of $\alpha =2,8,16$ and different group sizes $N=2,5,10,20$. For all values of $\alpha$ and all group sizes explored, search time had a minimum value we denoted as ${\beta }_{opt}$, which was evaluated numerically over a large range of values. For weaker couplings, namely $\beta <<{\beta }_{opt}$, agents’ performance was similar to the case of independent agents, whereas for strong coupling, $\beta >>{\beta }_{opt}$, the social term dominated, driving the agents toward each other rather than toward the source.

We note that the optimal value ${\beta }_{opt}$ should occur when the contributions from the two terms in Eq. 1 are of the same order of magnitude, or when

$$\mathrm{\Delta }S\approx \beta \mathrm{\Delta }G,$$ [S1]

where $\mathrm{\Delta }$ refers to the differences between the possible directions the agent can move (for simplicity we consider the case of two agents). The difference in the social term from agent $i$ moving toward $j$ or away from it is given by

$$\mathrm{\Delta }G={(d_{ij}+1)}^{\alpha }-{(d_{ij}-1)}^{\alpha }\approx 2\alpha d_{ij}^{\alpha -1},$$ [S2]

where we used the binomial theorem and kept only the leading term. Substituting Eq. S2 into Eq. S1, and solving for $\beta$, we get an analytic approximation for the optimal value of ${\beta }_{opt}^a(\alpha )$,

$${\beta }_{opt}^a(\alpha )\approx \frac{\mathrm{\Delta }S}{2\alpha d_{ij}^{\alpha -1}}\approx \frac{{10}^{-3}}{2\alpha d_{ij}^{\alpha -1}},$$ [S3]

where we used $\mathrm{\Delta }S={10}^{-3}$ as a typical value for the difference in the reduction in entropy between possible movement directions. Substituting $d_{ij}=\lambda =50$ (37), we can find an approximate analytic value for ${\beta }_{opt}^a$ and compare it to the value found numerically, ${\beta }_{opt}^{numeric}$. Fig. S2D shows this method provides a reasonable estimate and can serve as a good value to initialize a numeric search.

Analytical Analysis of the Functional Dependence of the Search Times on Coupling Strength

To further understand the behavior of information socialtaxis we can rewrite the free-energy Eq. 3,

$$F_i=S_i-\beta {\mathrm{\Sigma }}_j{D}_{kl}[p_i||p_j]$$ [S4a]

$$=-{\mathrm{\Sigma }}_{\text{𝐫}}{p}_i(\text{𝐫}){\log }_2{p}_i(\text{𝐫})-\beta {\mathrm{\Sigma }}_j{\mathrm{\Sigma }}_{\text{𝐫}}{p}_i(\text{𝐫}){\log }_2\frac{p_i(\text{𝐫})}{p_j(\text{𝐫})}$$ [S4b]

$$=-(\beta +1)[{\mathrm{\Sigma }}_j{\mathrm{\Sigma }}_{\text{𝐫}}{p}_i(\text{𝐫}){\log }_2\left(\frac{p_i(\text{𝐫})}{p_j(\text{𝐫})}\right)+\frac{1}{\beta +1}{\mathrm{\Sigma }}_j{\mathrm{\Sigma }}_{\text{𝐫}}{p}_i(\text{𝐫}){\log }_2{p}_j(\text{𝐫})]$$ [S4c]

$$=(\beta +1)[-{\mathrm{\Sigma }}_{\text{𝐫}}{p}_i(\text{𝐫}){\log }_2{p}_i(\text{𝐫})+\frac{\beta }{\beta +1}{\mathrm{\Sigma }}_j{\mathrm{\Sigma }}_{\text{𝐫}}{p}_i(\text{𝐫}){\log }_2{p}_j(\text{𝐫})].$$ [S4d]

The first term inside the brackets in Eq. S4c is the Kullback–Leibler divergence (32) between $p_i$ and $p_j$, which obtains its minimal value when the two distributions are identical. The second term, or cross-entropy, obtains its maximal value when $p_i=p_j$. Thus, we can interpret Eq. S4c as a combination of a term driving the distributions to be different and another term, which we can view as a constraint, driving the distributions to be more similar. The weight of the constraint decreases monotonically with $\beta$, which explains why the search time has a monotonic dependence on $\beta$. Eq. S4d shows that choosing a social term $G\equiv -D_{kl}[p_i||p_j]$ with coupling strength $\beta$ is equivalent to choosing $G\equiv \mathrm{\Sigma }{p}_i{\log }_2{p}_j$ with a weight factor $\widehat{\beta }\equiv \frac{\beta }{\beta +1}$, which is bounded between 0 when $\beta =0$ and 1 when $\beta \to \mathrm{\infty }$, implying that the behavior of the agents should converge asymptotically as $\beta \to \mathrm{\infty }$. We point out that this result does not necessarily mean that the search time should decrease, but that it should converge asymptotically toward some value.

Socialtaxis Is Robust to a Multitude of Environments and Wrong Modeling of the Environment

In the main text, the analysis of the socialtaxis algorithm was shown for a source strength $R=1$, emitting one particle per time step, with correlation length of $\lambda =50$, as in ref. 14 (Materials and Methods). Here we present the analysis of the robustness of the socialtaxis model following ref. 37 where we manipulated source strength, the layout of the arena, signal transport, particles’ lifetime, and more, for the case of two socialtaxis agents with a strong coupling of $\beta =10$. We found that the mean search time for socialtaxis agents is more efficient than that for independent infotaxis agents, except for very low emission rates (Fig. S1A). Socialtaxis was also much more efficient than independent infotaxis for a wide range of correlation lengths (Fig. S1B).

Rodriguez et al. (33) demonstrated the considerable effect of the geometry of the arena on the search times in infotaxis. To avoid a bias resulting from an arbitrary choice of initial configurations the values presented in Figs. 2A and 4 in the main text were the search times for randomly chosen initial configurations, but the size of the arena was fixed to a square arena of side length $L=64$. Testing socialtaxis and infotaxis under different arena sizes and in rectangular arenas showed clearly the superiority of socialtaxis (Fig. S1C). In all of these cases, we assumed that the agents had a correct estimate (or prior) for the strength of the source. We found that both infotaxis and socialtaxis demonstrated a sharp increase in search times as the difference between the agents’ model of $R$ and the real emission rates increased, and socialtaxis agents (who found the source for any value of $R_{model}$ tested) were more efficient than infotaxis agents except for very low model emission rates (Fig. S1D). Similarly, if the agents had a wrong prior for the correlation length of the source, then for most values of ${\lambda }_{model}$, socialtaxis agents were more efficient in finding the source, where ${\lambda }_{real}=50$ (Fig. S1E). In Fig. S1F we plot the mean search time vs. $\beta$ in a 3D search arena (36) and the qualitative behavior is similar to the case of 2D. Here, the side length of a the cubed arena is $L=32$, and $\lambda =20$. We note, however, that the search times of socialtaxis do not reach the values obtained by fully cooperating agents in this case. We conclude that socialtaxis works efficiently in a wide range of conditions (compare ref. 37) and is more efficient than infotaxis in almost all cases.

Fig. 4.

Biological relevance and extensions of socialtaxis: diverse coupling strength, limited information sharing, and inference. (A) Mean search time of pairs of agents for different combinations of individually different couplings between them. Each line shows for a given coupling strength between 2 and 1, ${\beta }_{2\to 1}$, the search time as a function of the coupling from 1 to 2, ${\beta }_{1\to 2}$. (B) The probability that agent $1$ would be the first one to find the source is shown as a function of the coupling strength from 1 to 2, ${\beta }_{1\to 2}$, whereas ${\beta }_{2\to 1}$ is held fixed at different values. Results for A and B are averaged over 3,000 random starting configurations. (C) Mean search time vs. coupling strength for two socialtaxis agents that share their respective maps with one another as above (dark red) and the search time when they share just the peak of a Gaussian fit with a predetermined variance to their map (dark blue). For the normalized mean search time for agents that do not share information, but infer their respective maps from the movement of other agents, inference of the full map based on estimated hits (pink) gives very small improvement compared with the case of independent agents (gray), whereas inferring only the peak location using GLM gives a significant improvement (light blue). Search times are normalized by the search times of two independent agents (set to 1). Error bars shown are SEM.

Emergent Group Behavior of Opportunistic Socialtaxis Agents Maximizing Their Information Gain

An emerging spatial collective behavior in the information-based socialtaxis becomes apparent when we study the trajectories of agents in larger groups (Fig. 3A and Movie S4). For a given group size $N$, increasing the weight of the social term $\beta$ resulted in a faster finding of the source (Fig. 3B). For a given $\beta$ value, the dependency on group size showed monotonic improvement in search time, hinting at the possibility of a power-law scaling (Fig. 3B, Inset). Again, large groups of agents that relied on distance-based socialtaxis showed only a small improvement in search times compared with those of independent agents, even in the best case ($N\approx 15$), and required fine tuning of model parameters (Fig. S4).

Fig. 3.

Information socialtaxis in large groups and the emergence of collective behavior. (A) Examples of 30 agents performing infotaxis independently (Top) and information-based socialtaxis with $\beta =10$ (Bottom). Each panel shows a snapshot in time of the agents. Starting area of agents is shown by a purple triangle and the source by a purple square. For presentation clarity, source is shown to be larger than its actual size, which is a single lattice site. (B) Mean search time for different group sizes $N$, performing information-based socialtaxis. Results are for 250 different groups and error bars show SEM. B, Inset shows mean search time as a function of group size for two values of $\beta$. (C) Difference in information gain between agents in information socialtaxis and independent agents as a function of time. C, Inset shows the average entropy of agents in the two cases as a function of time. Gray line shows the mean residual entropy when the source is found. (D) Average probability of an agent to be in a close group (cluster) with other agents (dotted lines) or alone (solid lines) vs. time for infotaxis and socialtaxis (see Materials and Methods for details). (E) Distribution of distances of agents from the source when the first agent has found it, for independent infotaxis agents and information socialtaxis ones. Groups started from the same initial conditions in both cases, and distributions are averaged over 1,000 random simulations.

Fig. S4.

Performance of distance-based socialtaxis. (A–C) Mean search time vs. $\beta$ for different group sizes $N=2,5,10,20$ for $\alpha =2,8,16$. Note that the $x$ axes are plotted in a logarithmic scale, and the search time for $\beta =0$ is added for comparison (double line indicates discontinuity in the values). (D) Optimal coupling strength $\beta$ vs. the scaling value $\alpha$ (Eq. 2). Both numeric values from simulations and the analytic approximation (Eq. S4) are shown.

The socialtactic agents gain information about the source at a rate that is similar to that of independent agents in the initial parts of their search, but the social interactions boost the rate of information gain significantly later on (Fig. 3C). Consequently, when the first agent finds the source, all other agents have also gathered much more information. The information socialtaxis keeps agents consistently less clustered spatially than independent agents (Fig. 3D), yet when the first agent finds the source (faster), they are all much closer to it than in the case of independent agents (Fig. 3E). Thus, whereas there is no obvious residual N dependence of information sharing in terms of time to source of the first agent, there is clear benefit for the rest of the group: Even if the other agents did not know that the first agent had found the source, they would still find it faster as the coupling becomes stronger (Fig. S5 and Socialtaxis Is Efficient for the Group, Beyond the First “Finder”). Consequently, the group demonstrates a spatial “social structure” and coherent movements, despite the lack of an explicit spatial term in the model.

Fig. S5.

Benefits of information socialtaxis for other agents, after the first one has found the source. (A) Normalized time it takes the second agent in a pair to find the source, as a function of $\beta$. Here, once the first agent has found the source, it disappears and the second agent continues by infotaxis. (B) Residual search time (the number of steps after the first agent has found the source) of the second agent vs. $\beta$.

Information Socialtaxis Is Efficient for the Group, Beyond the First “Finder”

Whereas information socialtaxis is efficient for the group in terms of the time it takes the first agent to find the source, we then asked how it affects the rest of the agents. We recall that Fig. 3E showed that the other agents are closer to the source once the first agent has found it, compared with noninteracting groups. However, this does not necessarily mean they have more useful information at that point. We therefore simulated pairs of agents performing information socialtaxis, but when the first agent found the source, we made that agent “disappear,” and the other agent continued its search using the standard infotaxis search. Fig. S5A shows the second agent is also more efficient in finding the source when it used the social term in the beginning of the search, supporting the conclusion implied from Fig. 3E. Fig. S5B indicates that information socialtaxis is even more efficient for the group, showing the residual search time for the group, namely the time it took the second agent to find the source after it was found by the first agent, was also shorter for socialtactic groups.

Biologically Plausible Socialtaxis and Optimal Behavior of Selfish, Opportunistic, or Altruistic Agents

How relevant is socialtaxis for real animal groups? First, we considered a more biologically realistic case, where each individual may be coupled differently to its conspecifics—as these couplings may be innate or learned from experience. Focusing on pairs of agents, we find that for a given coupling value of agent 2 to agent 1, ${\beta }_{2\to 1}$, the mean time to find the source decreased monotonically as a function of the reciprocal coupling ${\beta }_{1\to 2}$ (Fig. 4A). Thus, even when each agent can have an individually chosen coupling to the others, strong couplings are beneficial for the group as a whole. However, the optimal coupling value that would benefit an individual’s chances of finding the source are different from what would be good for the group: For a given value of ${\beta }_{2\to 1}$ the optimal ${\beta }_{1\to 2}$ that would raise the probability for agent $1$ to find the source first has a peak at ${\beta }_{1\to 2}\approx 0.5$, regardless of the value of ${\beta }_{2\to 1}$ (Fig. 4B). Thus, optimizing group success or individual success requires very different coupling values. Still, even the most opportunistic choice of coupling values was more efficient for the group than the case of independent agents and would benefit the group.

The critical question of relevance of the socialtaxis model to real animals is the feasibility of information sharing between agents (34). Surprisingly, we found that even a very limited form of information sharing gives a highly efficient socialtaxis: If instead of sharing their full probability map at each time point (which is obviously biologically unrealistic), agents shared only the location of the peak of a Gaussian that approximates their current probability map, the search time of the group was comparable or even better than the case where the whole map was shared (Fig. 4C). Interestingly, the difference between the sharing of just the peak and that of their full maps was most apparent for weak coupling between agents; for strong couplings the different models showed stable asymptotic behavior (Fig. S6 and Exploring the Peak Inference Algorithm).

Fig. S6.

Analyzing the peak inference algorithm. (A) Mean search time vs. coupling strength $\beta$ for the same search schemes shown in Fig. 4C, now shown for stronger coupling values. Results are averaged over 3,000 random starting configurations, and error bars indicate SEM. (B) Distance of real to estimated peak vs. time, with the peak estimated using a GLM as described in the main text and in Materials and Methods. This is shown for independent agents ($\beta =0$), at the optimal coupling ($\beta =0.15$), and for strongly coupled agents ($\beta =30$).

Whereas this limited-bandwidth information sharing could be relevant for animals that actively communicate with one another, we further asked whether socialtaxis could work without any form of communication between individuals. We found that even if agents did not share any information, but instead tried to infer information about the source from their conspecifics based only on observing their behavior, they were able to perform a highly beneficial socialtaxis. Here, each of the agents used a simple generalized linear model (GLM) that estimated the peak position of the map of the other agent, based on its trajectory, which led to a substantial decrease in search time. This GLM-based inference model was less efficient than the information-sharing socialtaxis, but was still able to get the agents to find the source in $84\%$ of the time needed by independent agents. We emphasize that this model is only a lower bound on how well inference-based socialtaxis can work, and therefore conclude that such an approach can be highly efficient with low computational cost for individual agents and is biologically feasible, in terms of memory and computational simplicity.

Exploring the Peak Inference Algorithm

The most biologically plausible version of socialtaxis is probably the one displayed in Fig. 4C, where agents do not communicate at all, but infer the position of the peak of their conspecifics based on their trajectories. The inference is described in Materials and Methods. In brief, the agents estimate the location of the peak from the other agent’s trajectory, using a GLM, with the agent’s position as the features of the model. The inference, and the use of the social term, is done only in the interval $t\in [50,150]$, and the estimated peak position is updated every five steps. To better understand the nature of the algorithm we plot in Fig. S6A the mean search time vs. the strength of coupling for the same search schemes as in Fig. 4C, now presenting the behavior at strong couplings.

In Fig. S6B we present the accuracy of the peak estimation at the optimal coupling ($\beta =0.15$) and for strongly coupled agents ($\beta =30$). The error in estimating the location of the other agent’s peak is very similar for both coupling strengths. This suggests that the increase in search times as $\beta$ is increased is not a result of a less accurate peak estimation, but of giving too strong a weight to an inaccurate peak position.

Discussion

We have introduced a model and algorithm for group foraging in complex environments where each agent aims to maximize the information from its own sensory perception and minimize the overlap in information it has with group members. Increasing the information diversity between members made the group highly efficient, similar to fully cooperating, optimally managed agents, in clear contrast to the small improvement achieved by models of group behavior that rely on the physical distance between individuals. Importantly, information socialtaxis did not require parameter tuning and was efficient under various variations of the model and robust to a multitude of noise sources. Most interestingly, biologically plausible versions of the model, based on limited or no information sharing, proved to be similar to or even better than the full model, making it a compelling framework for considering behavior of real animals.

Modeling individual behavior in a group, as an optimization problem combining self-information and group information, unifies the ideas of gradient-based search using sensory signals; social interactions; and game theory notions of opportunism, individualism, and group benefits. The success of our model challenges the commonly used models of group behavior based on distance or explicit spatial organization—suggesting information as key sufficient substance of sensory processing and collective behavior. In addition to being a biologically plausible, robust, and efficient model of group behavior in a variety of settings, socialtaxis suggests a potential design for new algorithms for distributed searching and learning.

Our framework also naturally exemplifies the benefits of diversity in groups and the qualitatively different collective behaviors that may arise from individual traits. Whereas these ideas have obvious parallels in game theory and behavioral economics, we note the relation of information-based socialtaxis to the studies of decorrelation and redundancy reduction as an optimization principle in neural coding (35). We thus suggest that information socialtaxis can be readily extended to explore animal groups; artificial agents; or other systems in which group relations can be learned or dynamically modulated; and groups in which individuals have different innate behaviors, preferences, or motivations.

Materials and Methods

Model of the Environment.

A single source at an unknown location ${\text{𝐫}}_0$ in a finite environment is emitting particles with finite lifetime ${\tau }_p$ at a rate of $R$ particles per second in random directions, advected by a mean wind $V$ and diffusing Brownianly, with diffusivity $D$; for other conditions, see Effect of Noise on Independent and Social Agents. The average concentration field at position r is a solution of the advection–diffusion equation, which for a constant diffusion coefficient $\nabla \cdot D=0$ and incompressible flow $\nabla \cdot \overrightarrow{v}=0$ gives (with advection flowing in the negative $y$ direction) in the two-dimensional case a hit rate of

$$R_{hits}(\text{𝐫}|{\text{𝐫}}_0)=\frac{R}{ln\left(\frac{\lambda }{a}\right)}exp[-\frac{(y-y_0)V}{2D}]K_0\left(\frac{|\text{𝐫}-{\text{𝐫}}_0|}{\lambda }\right),$$ [4]

where $K_0$ is the monotonically decreasing modified Bessel function of the second kind of order zero, and $\lambda$ is the correlation length of the source. Here we focus on the case with no wind $(V=0)$, which has been shown to be the most difficult one (14, 36).

Infotaxis.

The infotaxis algorithm is described in the main text and in detail in ref. 14. Briefly, an agent searching for the source decides on its move at time $t$ based on the expected change in entropy of the probability distribution over the locations of the source, $p_t(\text{𝐫})$ from each of the possible moves from its location $\text{𝐫}(t)$. This is given by weighting the change of entropy from finding the source, getting a hit or “no hit” by their relative probabilities

$$⟨\mathrm{\Delta }S(\text{𝐫}(t)\to \text{𝐫}\prime )⟩=p_t(\text{𝐫}\prime )[-S]+(1-p_t(\text{𝐫}\prime ))[{\rho }_0(\text{𝐫}\prime )\mathrm{\Delta }{S}_0+{\rho }_1(\text{𝐫}\prime )\mathrm{\Delta }{S}_1],$$ [5]

where the first term on the right-hand side is the weighted value of the vanishing entropy from finding the source when moving to $\text{𝐫}\prime$ (“exploitation term”); and the second term is the sum of change in entropy from a hit and a no hit in that location weighted by their probabilities, where ${\rho }_k(\text{𝐫}\prime )$ is the probability to get $k$ hits at $\text{𝐫}\prime$. As in ref. 14, we neglect the case of $k>1$ due to sparseness of the source.

Estimating and Updating the Probability Map of an Agent.

At each time point, an agent estimates the expected number of hits at position r by $h(\text{𝐫})\equiv \mathrm{\Delta }t\int p_t({\text{𝐫}}_0)R(\text{𝐫}|{\text{𝐫}}_0)d{\text{𝐫}}_0$. Assuming independent hits the probability to obtain $k$ hits is given by a Poisson distribution, and then using the posterior map of a hit trace, the updated map is given by

$$p_{t+1}({\text{𝐫}}_0)=\frac{1}{Z}{p}_t({\text{𝐫}}_0)\{\begin{array}{cc}\hfill e^{-R(\text{𝐫}(t)|{\text{𝐫}}_0),}& \text{if}\hspace{0.17em}k=0\hfill \\ \hfill e^{-R(\text{𝐫}(t)|{\text{𝐫}}_0)}R(\text{𝐫}(t)|{\text{𝐫}}_0),& \text{if}\hspace{0.17em}k=1\hfill \end{array},$$ [6]

where $h=1(0)$ stands for the agent obtaining (not obtaining) a hit during this $dt$ time step, $Z$ is a normalization factor of the posterior probability, and the probability in visited sites is set to zero.

Model of Fully Cooperating Groups.

Following Masson et al. (37), we consider the case of fully cooperating groups where agents shared a single probability map, updated at each step based on the hits/no hits of all agents,

$$p_{t+1}^{sh}(\text{𝐫})=\frac{1}{Z}{p}_t^{sh}(\text{𝐫})\cdot {\mathrm{\Pi }}_{i=1}^N\{\begin{array}{cc}\hfill e^{-R({\text{𝐫}}_i(t)|{\text{𝐫}}_0),}& \text{if}\hspace{0.17em}{k}_i=0\hfill \\ \hfill e^{-R({\text{𝐫}}_i(t)|{\text{𝐫}}_0)}R({\text{𝐫}}_i(t)|{\text{𝐫}}_0),& \hfill \text{if}\hspace{0.17em}{k}_i=1\end{array},$$ [7]

where $N$ denotes the number of agents in the group, ${\text{𝐫}}_i$ denotes the position of agent $i$, $h_i=1(0)$ corresponds to agent $i$ obtaining (not obtaining) a hit during this time step, $Z$ is the normalization factor, and the superscript $sh$ indicates this map is shared by all agents. The decision on where to move is also coordinated as the agents evaluate all of the possible joint moves of the group. We note that the number of possible actions the group can take on a 2D square lattice is $5^N$, which limits the size of the group that could use this algorithm due to the exponential growth in computational cost.

Spatial Organization of Groups of Agents.

Analyzing behavior of large groups, “in cluster” and “isolated” agents were defined according to the number of conspecifics $n_{nbr}$ inside the neighborhood of an agent, which was defined as a circle of radius $d$ around it. In cluster was defined as $n_{nbr}\ge n_{cluster}$ and $n_{nbr}\le n_{isolated}$ was defined as isolated. Here we used $d=10$, $n_{cluster}=10$, and $n_{isolated}=3$, but different choices of parameters $d$, $n_{cluster}$, and $n_{isolated}$ gave qualitatively similar results.

Inference Model of Another Agent’s Probability Map Based on Its Trajectory.

Two inference models of the map of other agents were used to explore socialtaxis without sharing of information between agents. In the full-map inference model, agent $i$ iteratively used its current model of the map of agent $j$, ${\widehat{P}}_i^j(t)$, to assess the likelihood that the recent move of $j$ was the result of a hit, and from the inferred hits, it estimated the probability map of the other agent (in the common case where both events would result in the same movement we assumed that $i$ assigned a hit to $j$ with a probability based on the map that $i$ had at that time). Trying to infer hits using longer memory of past behavior of the other agents gave a small improvement of performance, but at an exponential computational cost. In the peak inference case, each agent tried to infer the location of the peak of the probability map of other agents, using a GLM. The $x$ and $y$ coordinates of the peak were estimated independently of each other with different models, where the past positions of the other agent were weighted and fed to a nonlinear function, to estimate the $x$ or $y$ location of the peak. We learned a separate model for a different starting region of the agents (space was split into 64 different 7×7 starting regions), and each model was trained using ${10}^5$ simulated trajectories. This procedure was performed separately for every coupling strength $\beta$. In that case, the inferred map that $i$ has of $j$, ${\widehat{P}}_i^j(t)$, was a Gaussian fit to the estimated peak position at $t$, with variance predetermined as in the regular Gaussian case described in the main text. To make this algorithm more stable and computationally compact, we used a limited temporal window for the social term, $t\in [\mathrm{50,150}]$, and the estimate of the location of the peak was updated only every five steps.

Effect of Noise on Independent and Social Agents

An animal (agent) searching for a source might be influenced by several sources of noise. These include sensory noise in perceiving the stimulus, inaccuracy in estimating its own location, or, in the case of socialtaxis, receiving inaccurate or noisy information from other group members. We therefore explored the effect of several noise sources on independent and socialtactic agents: We simulated the noise in the agents’ sensory perception by assigning a random probability for missing a hit, which we denote as p(missing a hit). In Fig. S3A we plot the mean search time for two independent and two socialtactic agents vs. the probability to miss a hit and find that for low noise socialtactic agents are more efficient at finding the source, whereas for high noise the inverse holds. The probability to miss a hit is equivalent to having a weaker source; hence these results are in agreement with those shown in Fig. S1A, which demonstrated that for very weak sources infotaxis is slightly more efficient than socialtaxis. Another form of potential noise that might affect the agents is odometry noise, meaning the agent’s estimate of its own location is inaccurate. Fig. S3B shows that for independent agents the search time increases with odometry noise, whereas for coupled agents, the information arriving from the other agent corrects these errors, and their search time is practically independent of the odometry noise we simulated.

Another source of noise may be social, which in this case would mean that the social information sent by the other agent or received from it may be corrupt. We simulated this scenario by adding Gaussian white noise to the probability map of agent $j$, ${\stackrel{\sim }{P}}_j$; noise was added independently to each cell. Thus, the social term of agent $i$ was now given by the $KL$ divergence from the noisy map, $G_i=-D_{KL}[P_i|{\stackrel{\sim }{P}}_j]$. We found that socialtaxis is more efficient up to very large noise levels (Fig. S3C). An analogous situation may occur when agents use the Gaussian approximation of the other agent’s map in the social term, captured here by randomly shifting the peak position. Fig. S3D demonstrates again that for low and moderate noise levels socialtaxis is more efficient, whereas at very high noise levels the use of an inaccurate social term increases search times. We note that we assumed here that agents trusted their conspecifics completely. Adding an uncertainty about the information coming from others would probably improve the results even more.

Another possible social noise may arise in the case where agents have different models of the environment. We focus on the case of two agents, where one of the agents has the correct estimate of the strength of the source $R_1=R_{real}=1$, whereas the other agent has a wrong estimate of the source emission rate $R_2$. In Fig. S4A we plot the mean search time vs. $\beta$ for different values of $R_2$ and note that in all cases the search time decreases monotonically with $\beta$. Furthermore, we note that if one of the agents overestimates the source strength, there is a decrease in the mean search time. In Fig. S4B we note that if an agent underestimates the strength of the source, it had a higher probability to find it, for the case of weak couplings between the agents (whereas overestimating the strength of the source decreases the agent’s probability to find it). For strong couplings, these differences vanish, and the probability to find the source becomes independent of the model emission rate $R_2$ for all values explored.