Individual incentives that promote collective intelligence

Significance

A group can often outperform an individual in complex problem solving, even when the group lacks a sophisticated central planner. We develop a simple theory to explain how collective intelligence emerges. The group tries to predict an outcome that depends on many random factors. Even though each individual can observe only a single factor, the collective prediction from averaging across individuals may nonetheless be accurate. We explore two schemes for rewarding individuals to foster collective intelligence. One approach rewards experts: those whose predictions are more accurate. The other approach rewards reformers: those whose predictions may not be accurate, but whose contributions improve the collective prediction. Both reward schemes can foster collective accuracy, but rewarding reformers is more robust and efficient.

Abstract

A collaborative group can often outperform a single individual in complex problem solving, even when information is limited. This phenomenon, called collective intelligence, can be achieved by engineering a central planner who assigns subtasks distributed across the group. But such algorithms cannot explain how natural populations, which often lack sophisticated central control, can nonetheless evolve collective intelligence. In fact, the process of social learning by imitating successful peers will typically reduce diversity and inhibit collective intelligence. Here, we consider a prediction task where the true outcome each round is a continuous quantity that depends linearly on a large number of random causal factors. Each individual can observe only one factor, and the collective prediction is generated by aggregating personal predictions across individuals. We propose two classes of reward structures that guarantee the emergence of collective intelligence through social learning. One scheme provides greater rewards to those individuals (called experts) whose personal predictions are more accurate. The other scheme provides greater rewards to those individuals (called reformers) whose predictions have greater potential to reduce the collective error, even though their personal predictions may be far from the truth. Although both of these payoff structures can provably maintain diversity and establish collective intelligence, we show that rewards based on collective error are more robust to diverse problem settings than rewards based on personal accuracy. Our results show that identifying reformers is more effective than identifying experts in promoting the emergence of collective intelligence.

A group of relatively uninformed individuals can often collaborate to accomplish a task that exceeds the capability of any single individual (1–5). For example, the average estimate of a quantity by a group is often remarkably close to its true value, whether in a single-shot guessing task or in more complex problem solving. This phenomenon in human groups is commonly referred to as collective intelligence.

A body of work in engineering describes how to achieve collective intelligence, at least for distributed computation nodes. Algorithms developed in control theory and machine learning can ensure that a group of nodes will effectively work together to accomplish a complex task. Such algorithms either divide the task into subtasks, each of which can be accomplished in parallel by individual nodes (6–9), or they adaptively optimize hyperparameters to dynamically control the contribution weights from individual nodes (10, 11). These algorithms require sophisticated central control, and they are designed for execution by computers. Such a sophisticated central planner is typically unavailable in natural populations, which nonetheless exhibit collective intelligence (1–5, 12, 13). And so the question remains: How can relatively uninformed individuals, without access to sophisticated central control, nonetheless develop collective intelligence through individual-based processes, such as social learning and peer-to-peer imitation (14–20).

A basic requirement for collective intelligence is a diversity of individual opinions, or strategies, within the population (12, 21–24). However, social influence and learning through imitation permeate the natural world and would seem to winnow the diversity that is required for collective problem solving (21). Consequently, an individual-based mechanism that incentivizes individuals to preserve diversity may be necessary for the emergence of collective intelligence.

Merely maintaining diversity among individuals may be insufficient to achieve collective intelligence. Many real-world problems are inherently complex in ways that diversity alone cannot reconcile (25–29). Moreover, in some settings when numerous factors are observable, which particular factors influence the outcome remains poorly understood. For such problems where solutions depend intricately on numerous factors, it may be impossible for any single individual to aggregate and analyze the requisite data. Even if a diversity of opinions is preserved in a population, and even if a population has access to a simple mechanism for aggregating and averaging individual opinions, it remains unclear what mechanisms will stimulate the correct diversity of individual strategies to produce accurate collective predictions.

A few prior studies have proposed individual incentive structures, such as rewarding accurate individuals (30) or rewarding minority opinions (31, 32), that might facilitate the emergence of collective intelligence in natural populations. These approaches are typically confined to simple value estimation tasks, without correlated factors, or to binary voting scenarios.

Here, we consider prediction tasks where the outcome is continuous and depends on a large number of causal factors, which may be independent or correlated. We consider a setting where each individual has limited information, so that he or she can observe only a single factor when making their individual prediction. A collective prediction is then generated by aggregating individual predictions, through one of several different aggregating rules. Individuals are rewarded with payoffs according to their performance, and they tend to imitate the strategies of others who have higher payoffs, in accordance with the standard assumption of evolutionary game theory that describes the social spread of behaviors (14, 15, 33–35). The key question, then, is what reward structures will incentivize individuals to adopt strategies that establish collective intelligence in the population.

Here, we propose two reward structures that can provably induce the emergence of collective intelligence in these naturalistic settings. We prove that collective intelligence is the asymptotically stable state for both of these structures. One reward structure, called niche expert, is based on the accuracy of each individual’s prediction. Individuals who predict more accurately, and especially those who are observing a factor that is otherwise poorly represented in the population, are rewarded with higher payoffs. The second reward structure, called feedback, is based on the error of the population’s collective prediction. The feedback structure does not reward experts, but rather those individuals, called reformers, whose individual predictions may not be accurate but are nonetheless beneficial for revising the collective prediction toward the truth. Although both of these reward structures provably induce accurate collective predictions, we find that the feedback is more robust to alternative forms of prediction tasks (e.g., task switching or correlated factors) and it exhibits more rapid convergence to collective intelligence. These results imply that rewarding reformers is more effective than rewarding niche experts, in promoting collective intelligence.

Model

We consider a scenario where the environmental outcome changes over discrete time, under the influence of $m$ factors denoted by $\mathcal{M}=\{1,2,\cdots ,m\}$. Let $Y$ represent the outcome of the environment and $x_i$ denote the value of factor $X_i$ at a given time. Throughout this study we assume a linear relationship between the environmental outcome $Y$ and the $m$ factors (36), expressed as

$$\begin{array}{c}\hfill Y={\alpha }_0+{\alpha }_1{X}_1+\cdots +{\alpha }_m{X}_m,\end{array}$$ [1]

where ${\alpha }_i$ ($i\in \mathcal{M}$) represents the weight of factor $X_i$’s contribution to the environmental outcome, and ${\alpha }_0$ is a constant. The values of the factors $x_i$ vary over discrete time steps, each sampled independently from a distribution with mean $0$ and variance ${\sigma }_i^2$, whereas the weights ${\alpha }_i$ and ${\alpha }_0$ remain constant (Fig. 1A). We will focus on the case of normally distributed random factors, but our results extend to any distribution with finite variance (see examples in SI Appendix, Fig. S1), as well as some cases in which the outcome is a nonlinear function of factors (SI Appendix, Fig. S2).

Fig. 1.

Individual and collective predictions. (A) In each time step the outcome of the environment, $Y$, is a linear combination of $m$ random factors ($X_1,\dots ,X_m$) sampled independently from a Gaussian distribution. The linear coefficients (${\alpha }_i$) are unknown to individuals. Each individual can observe the value of only a single factor before making their personal prediction for the outcome of the environment. The true environmental outcome $Y$ is observed only after all players have made their individual predictions. (B) An individual $A$ has a strategy composed of two components: which factor, denoted $e_A$, the individual has chosen to observe, and how the individual believes that factor is correlated with the outcome, denoted $c_A$ and called the individual’s belief. Thus individual $A$ makes personal prediction $Y_A=c_A{x}_{e_A}$. (C) The collective prediction of the entire population $\widehat{Y}$ combines the personal predictions of all individuals. There are several ways to combine individual predictions into a collective prediction, expressed by different aggregating functions $g(Y_1,Y_2,\cdots ,Y_n)$. We focus on two aggregating functions: averaging across the entire population (Eq. 2) or averaging across clusters of individuals who focus on the same factor (Eq. 5).

We now consider a population of $n$ individuals who are tasked with the problem of predicting the outcome of the environment, based on observations of the factors that determine it. Due to cognitive constraints or resource limitations, each individual can observe the value of only a single factor at each time step (Fig. 1B). In particular, when individual $A$ chooses to observe factor $X_{e_A}$, they obtain its value $x_{e_A}$. However, ${\alpha }_{e_A}$, which reflects the true weight of the factor’s influence, remains unknown to the individual. Individual $A$ then makes a personal prediction about the environmental outcome based on the observed factor $X_{e_A}$ and their belief about the correlation between the observed factor and the outcome. Let $c_A$ represent individual $A$’s belief: If $c_A>0$, individual $A$ believes the factor is positively correlated with $Y$, whereas if $c_A<0$ individual $A$ believes the factor is negatively correlated with $Y$. The magnitude of $c_A$ (i.e., $|c_A|$) reflects the believed strength of the factor’s influence on the outcome. Thus, the personal prediction made by individual $A$ is given by $Y_A=c_A{x}_{e_A}$. The strategy of individual $A$ is denoted by a pair $(e_A,c_A)$, which prescribes which factor the individual chooses to observe, and how they believe it correlates with the environment.

In addition, we account for the presence of stubborn individuals who do not observe any factors but instead provide constant predictions, even as the values of environmental factors are resampled. To formalize this, we introduce a virtual factor $X_0$, whose value $x_0$ always equals $1$. Stubborn individuals can be viewed as adopting the strategy $(0,c_A)$, so that their predictions are constant and expressed as $Y_A=c_A{x}_0=c_A$. Introducing stubborn individuals is important for obtaining accurate collective predictions, since the combination of stubborn individuals’ beliefs serves to predict the constant term ${\alpha }_0$ in the environmental outcome.

After all individuals observe one factor and provide their personal predictions, the collective prediction of the group is produced by aggregating the individuals’ opinions (Fig. 1C). Generally, the collective prediction $\widehat{Y}$ is a function of individual predictions, i.e., $\widehat{Y}=g(Y_1,Y_2,\cdots ,Y_n)$. There are many ways to aggregate individual opinions, with different aggregation rules corresponding to different aggregation functions. In this paper, we focus on two kinds of aggregating rules: averaging and clustering. Under the averaging rule, each individual’s prediction is treated as an estimate of the final outcome $Y$. The collective prediction is obtained by calculating the average of all individuals’ opinions:

$$\begin{array}{c}\hfill \widehat{Y}=\frac{Y_1+Y_2+\cdots +Y_n}{n}.\end{array}$$ [2]

An alternative way to express this aggregate prediction is in terms of the predicted coefficients, ${\widehat{\alpha }}_i$, for each factor:

$$\begin{array}{c}\hfill {\widehat{\alpha }}_i=\frac{1}{n}{\displaystyle \sum _{A\in {\mathcal{N}}_i}}{c}_A,\end{array}$$ [3]

where ${\mathcal{N}}_i$ represents the set of individuals observing factor $X_i$. The collective prediction $\widehat{Y}$ can then be expressed as the linear form, $\widehat{Y}={\widehat{\alpha }}_0+{\widehat{\alpha }}_1{x}_1+\cdots +{\widehat{\alpha }}_m{x}_m$.

In contrast to aggregation by equal-weight averaging, the clustering aggregation rule is based on the idea that each individual tries to predict only the contribution of the factor they observe (i.e., ${\alpha }_{e_A}{x}_{e_A}$) rather than the final outcome $Y$. Under the clustering aggregation rule, therefore, the predicted coefficient ${\widehat{\alpha }}_i$ is calculated as the average belief among the individuals observing the factor $X_i$ (rather than averaging over all individuals):

$$\begin{array}{c}\hfill {\widehat{\alpha }}_i=\frac{1}{n_i}{\displaystyle \sum _{A\in {\mathcal{N}}_i}}{c}_A.\end{array}$$ [4]

Here, $n_i$ denotes the size of cluster $i$, meaning those individuals who observe factor $X_i$: $n_i=|{\mathcal{N}}_i|$. The collective prediction $\widehat{Y}$ is then obtained by summing the predicted effects of all factors,

$$\begin{array}{c}\hfill \widehat{Y}={\displaystyle \sum _{i=0}^m}\frac{1}{n_i}{\displaystyle \sum _{A\in {\mathcal{N}}_i}}{Y}_A,\end{array}$$ [5]

where any factor with $n_i=0$ (i.e., no observers) is excluded from the sum.

After a single instance of sampling the environment and forming predictions, individual $A$ is rewarded by a payoff ${\pi }_A$ according to their performance. In the next section, we introduce several payoff structures that measure the performance of individuals in different ways. After many subsequent instances of prediction (i.e., repeated sampling of the environment), each individual obtains a stationary payoff (i.e., expected payoff $\mathbb{E}({\pi }_A)$). Individuals then update their strategies by imitating those with larger payoffs. In some parts of our analysis, we also include “mutation” or innovation so that individuals can explore new strategies not presently in the population.

More specifically, strategy imitation occurs as follows. In each discrete generation, two individuals $A$ and $B$ are selected uniformly at random. With probability $1-\mu$, $A$ imitates $B$’s strategy with probability (37, 38)

$$\begin{array}{c}\hfill e_{A\to B}=\frac{1}{1+exp(s(\mathbb{E}({\pi }_A)-\mathbb{E}({\pi }_B)))},\end{array}$$ [6]

or adopts a randomly generated strategy with probability $\mu$. Here, $s$ is called the intensity of selection, which quantifies the degree to which payoff affects imitation dynamics. This formulation is standard in evolutionary game theory to describe how agents with bounded rationality update strategies by social learning (39–42), as opposed to perfectly rational agents who would immediately compute Nash equilibria.

Using stationary payoffs to update strategies means that the timescale of repeated environmental sampling and prediction is much faster than that of strategic imitation. In SI Appendix, Fig. S3, we explore alternative relative timescales, including cases where the strategy updating takes place after each instance of prediction. The results demonstrate that our findings remain robust across different timescale configurations. Therefore, our subsequent theoretical analysis continues to assume stationary payoffs (rapid environmental sampling and prediction).

Individuals undergoing the imitation process described above will adjust their strategies over time in pursuit of higher payoffs, which in turn affects their prediction accuracy and subsequent rewards. We study what kind of reward structures can foster collective intelligence. In particular, we propose two reward structures that are based on different forms of information: one reward structure that evaluates an individual’s performance based on their personal success (i.e., the accuracy of their individual prediction) and another payoff that evaluates each individual by the contribution to the collective success (i.e., the error of the group’s prediction).

To evaluate the accuracy of the collective prediction, we define the following metric

$$\begin{array}{c}\hfill R=1-\frac{\mathbb{E}[{(Y-\widehat{Y})}^2]}{\mathbb{E}[{(Y-{\alpha }_0)}^2]}.\end{array}$$ [7]

Here, the expectation is taken over all possible values of the factors $x_1,\cdots ,x_m$. A larger value of $R$ indicates higher prediction accuracy. Specifically, $R=1$ implies that the collective prediction perfectly matches the real outcomes. Importantly, there is no lower bound for $R$, as prediction errors can grow arbitrarily large.

Results

Payoffs Based on Personal Accuracy.

We first examine payoff structures that are based on personal accuracy. A natural reward structure that might perhaps engender collective accuracy is to reward those individuals with larger payoffs when their personal predictions are more accurate. Given individual $A$’s personal prediction $Y_A$ ($Y_A=c_A{x}_{e_A}$), such a straightforward payoff structure for individual $A$ can be expressed as

$$\begin{array}{c}\hfill {\pi }_A=-{(Y_A-Y)}^2.\end{array}$$ [8]

Under this payoff structure, those individuals whose predictions are closer to the actual outcome $Y$ receive higher payoffs, and so we refer to this formulation as the “expert” payoff structure. After a sufficient number of predictions and environmental draws, the expected payoff for individual $A$ is given by the expectation of Eq. 8 over all possible values of the factors $x_1,\dots ,x_m$, namely

$$\begin{array}{c}\hfill \mathbb{E}({\pi }_A)=-c_A^2{\sigma }_{e_A}^2+2c_A{\alpha }_{e_A}{\sigma }_{e_A}^2+C,\end{array}$$ [9]

where $C$ is a constant (the same for all individuals). If $e_A=0$ (i.e., player $A$ is a stubborn individual), the expected payoff still has the same form as Eq. 9 by assuming ${\sigma }_0^2=1$. Under the expert payoff function, the expected payoff for each individual is independent of the strategies of other individuals. Moreover, among all individuals who choose to observe factor $X_i$ the one whose belief $c_A$ is closest to ${\alpha }_i$ will obtain the highest payoff, ${\alpha }_i^2{\sigma }_i^2+C$. Thus, within the entire population, the individual who observes the most valuable factor $X_{i^{\ast }}$ (i.e., ${\alpha }_{i^{\ast }}^2{\sigma }_{i^{\ast }}^2>{\alpha }_i^2{\sigma }_i^2$ for all $i\ne i^{\ast }$) and whose belief is closest to ${\alpha }_{i^{\ast }}$ will obtain highest payoff.

Over the course of strategic evolution by imitation, under the expert payoff scheme, all individuals will ultimately converge to adopt this most profitable strategy, $(i^{\ast },{\alpha }_{i^{\ast }})$. In the stationary state, then, all individuals choose to observe the same factor (factor $X_{i^{\ast }}$) and have the same belief coefficient. As a result, in the absence of mutations ($\mu =0$), no matter what aggregation function is employed, the predicted outcome $\widehat{Y}$ will incorporate no information from factors that are not being observed, so that the collective accuracy $R$ can never reach a high level. On the other hand, if the mutation rate is very high then most individuals will employ random strategies with poor estimates of the true coefficients, resulting again in low collective accuracy. For a narrow range of intermediate mutation rates the collective prediction under expert payoffs can reach relatively high accuracy, but perfect prediction is still impossible (SI Appendix, Fig. S4).

The key reason why the expert payoff scheme does not guarantee collective accuracy is that opinion diversity is lost during the process of evolution by social learning. To avoid a population that is entirely focused on a single factor, we propose the “niche expert” payoff structure by setting

$$\begin{array}{c}\hfill {\pi }_A=-{\rho }_{e_A}{(Y_A-Y)}^2.\end{array}$$ [10]

Here, ${\rho }_{e_A}$ is the proportion of individuals in the population who are observing the same factor as individual $A$, $e_A$. The niche-expert payoff depends not only on the accuracy of an individual’s prediction but also on the frequency of other individuals who are observing the same factor. When a large proportion of individuals observe the same factor $X_i$, this will decrease the payoffs of those who observe $i$, even though some of these individuals’ personal predictions may be very close to the real outcome, $Y$.

We can intuitively understand the equilibrium state for the niche-expert payoff by extending our analysis of the expert payoff case. Given the niche-expert payoff function (Eq. 10), if a factor $X_i$ is observed by a very small proportion of individuals, then choosing to observe factor $X_i$ will produce payoff near $0$, which is the highest payoff possible. As a result of this frequency dependence, more individuals will tend to switch to observe factor $X_i$. In equilibrium, for $N\gg m$, all factors will be observed by a substantial portion of individuals. And among the individuals who observe the same factor, $X_i$, those whose belief $c_A$ is closer to the true value ${\alpha }_i$ will obtain a higher payoff, as we saw in our analysis of the expert payoff function. As a result, in the absence of mutation, among the cluster of individuals with strategy $(i,c_A)$ observing factor $X_i$, the strategy $(i,c_{i\ast })$ whose belief $c_{i\ast }$ is closest to true coefficient ${\alpha }_i$ will eventually fix within the cluster, or subpopulation. In other words, in equilibrium all individuals observing factor $X_i$ will have belief $c_{i\ast }$. This is the unique stable equilibrium of the evolutionary system, under the expert payoff function.

For the niche-expert payoff, provided the initial composition of the population contains observers for all different factors, and also contains observers whose beliefs are close to the true coefficient ${\alpha }_i$ for each factor, then the equilibrium collective accuracy under clustering aggregation (Eq. 5) is guaranteed to be close to $R=1$. By extension, if the initial population contains at least one individual with perfectly accurate belief, for each factor, then the equilibrium collective accuracy will be perfect: $R=1$. Finally, in the presence of rare mutations, ($\mu \ll 1$), the equilibrium collective accuracy is guaranteed to be close to $R=1$, regardless of initial composition in the population. In SI Appendix, section S2.1, we provide more details about this equilibrium and its provable stability properties. Note that the niche expert does not succeed under the averaging aggregation; and so all subsequent results for the niche expert assume clustering aggregation.

We performed Monte Carlo simulations to verify the effectiveness of the expert and niche-expert payoff structures. All individuals are assigned random initial strategies (observe a random factor with a personal belief sampled from a Gaussian distribution). Fig. 2A shows that expert payoff equipped with the clustering aggregation rule cannot drive the population to produce accurate predictions; but the niche expert equipped with the clustering rule can induce accurate collective predictions. In the stationary state, under the expert payoff nearly all individuals are focused on the same factor (Fig. 2B). But under the niche-expert payoff, each factor is observed by an abundance of individuals and individuals’ beliefs are close to the real coefficients (Fig. 2C, and also see SI Appendix, Fig. S5).

Fig. 2.

The niche-expert payoff structure can facilitate collective intelligence. We consider two kinds of payoff structures that are based on the accuracy of individuals’ personal predictions: the expert (Eq. 8) and niche expert (Eq. 10). Both payoff structures are equipped with the clustering aggregation rule. Under the expert payoff function, an individual receives a higher payoff if their prediction is more accurate (closer to the real outcome); where under niche expert, an individual receives a higher payoff not only if their prediction is accurate but also if few individuals are focused on the same factor as them. (A) The expert payoff function can promote collective accuracy in early stages of behavioral evolution, but collective accuracy subsequently collapses. Whereas the niche-expert structure incentivizes individuals to focus on different factors, which guarantees long-term collective accuracy. (B and C) We show simulation results when the outcome is determined by $m=50$ factors, with coefficients ${\alpha }_k=-5+0.2k$. The stationary strategy distribution in the population for the expert payoff structure (B) is less diverse than for the niche-expert payoff structure (C). Each dot represents an individual’s strategy, including their belief and which factor they observe. (B) For expert payoff, nearly all individuals focus on the factors with the strongest effect on the outcome; but most factors are not observed by any individuals at all. (C) The niche-expert payoff function ensures a diversity of factors being observed, and it results in beliefs that are close to the real coefficient, for each factor. Parameters: $m=50$, $s=50$, $\mu =0$, ${\alpha }_i\sim U(-5,5)$ (uniform distribution) (A), ${\sigma }_i\sim U(0,3)$, $c_A\sim \mathcal{N}(0,5^2)$ (normal distribution), and $N=10,000$.

Payoffs Based on Collective Error.

For the expert and niche-expert reward structures, the core idea is that individuals with more accurate personal predictions receive higher payoffs. Thus, these payoffs are based on the accuracy of personal predictions alone. These payoff structures do not incorporate any consideration of the collective prediction $\widehat{Y}$ produced by the population. Next we consider a different payoff structure that is based on the error of the collective prediction (i.e., $Y-\widehat{Y}$) and how an individual contributes to it:

$$\begin{array}{c}\hfill {\pi }_A=Y_A(Y-\widehat{Y}).\end{array}$$ [11]

We call this payoff structure “feedback” because it rewards individuals whose contributions bring the collective prediction closer to the true value. In particular, when collective prediction is lower than the real outcome (i.e., $Y-\widehat{Y}>0$), individuals whose personal predictions are larger than the collective will receive higher payoff, and vice versa. The core idea of the feedback scheme is not to reward experts (individuals who predict more accurately), but rather to reward individuals whose predictions have more potential to improve the collective prediction, even if their personal predictions may deviate substantially from the outcome $Y$. We call such individuals “reformers.”

Unlike a payoff formulation based on personal accuracy, the feedback payoff structure depends on (and requires information about) the collective prediction $\widehat{Y}$. As a result, the feedback payoff function depends on the choice of aggregating rule used to form the collective prediction. We consider two versions of the feedback payoff arising from either the average aggregation rule (Eq. 2) or the clustering aggregation rule (Eq. 5).

For a feedback payoff structure (Eq. 11) we can compute the stationary payoff for individual $A$:

$$\begin{array}{c}\hfill \mathbb{E}({\pi }_A)=c_A({\alpha }_{e_A}-{\widehat{\alpha }}_{e_A}){\sigma }_{e_A}^2.\end{array}$$ [12]

Here, ${\widehat{\alpha }}_i$ is the predicted coefficient of the $i_{\mathrm{th}}$ factor, given by Eq. 3 under the average aggregation rule or by Eq. 4 under the clustering aggregation. It follows from Eq. 12 that all individuals have the same stationary payoff if and only if ${\alpha }_i={\widehat{\alpha }}_i$ for all factors $0\le i\le m$. So this is an equilibrium state of the evolutionary process, and the collective prediction will equal the real outcome in equilibrium ($R=1$). In SI Appendix, section S2.2, we construct a Lyapunov function $E[{(Y-\widehat{Y})}^2]$ for this system in the absence of mutation, which decreases monotonically as evolution proceeds until all individuals have the same payoff. Thus, for any initial strategy distribution, provided the real coefficients are in the predictable domain (i.e., given the strategies in the population, there exists at least one strategic composition with ${\widehat{\alpha }}_i$ equals ${\alpha }_i$ for all $i$), the equilibrium state $R=1$ is asymptotically stable. When mutations occur at a low rate, $\mu \ll 1$, the collective prediction is still accurate ($R=1$), and the predictable domain can even increase in size.

Unlike the niche expert, which requires clustering aggregation to produce collective accuracy, the feedback payoff can facilitate collective intelligence under both clustering and averaging aggregation (Fig. 3A). In equilibrium individuals’ beliefs need not be close to the real coefficient, as in the case of the niche-expert payoff, but rather the predicted coefficients ${\widehat{\alpha }}_i$ must be close to the real coefficients ${\alpha }_i$. Thus, the strategy distribution in equilibrium under feedback is not unique, but depends on initial conditions. The equilibrium strategy distribution under feedback payoff is also much more diverse than under the expert or even niche-expert payoff functions (Fig. 3 B and C).

Fig. 3.

Feedback payoff structures can facilitate collective intelligence. The feedback payoff function (Eq. 11) rewards individuals not based on their personal accuracy, but based on a comparison between individual beliefs and collective accuracy. The core idea of this payoff structure is to reward those, called reformers, whose beliefs make the collective prediction more accurate. (A) The feedback payoff has the general form ${\pi }_A=Y_A(Y-\widehat{Y})$, although the specific function depends upon the aggregation rule used to form the collective prediction $\widehat{Y}$ (averaging or clustering, Eq. 2 or Eq. 5 respectively). Both averaging feedback and clustering feedback guarantee the long-term accuracy of the collective prediction. (B and C) Unlike payoff functions based on personal accuracy alone (Fig. 2), for feedback structures the stationary beliefs of individuals observing the same factor $X_i$ need not converge to the true coefficient, ${\alpha }_i$, but rather are dispersed. Furthermore, the stationary strategy distribution is not unique, but depends on the initial distribution of strategies in the population. Parameters: $N=10,000$, $m=50$, $\mu =0$, $c_A\sim \mathcal{N}(0,{100}^2)$ (B), $c_A\sim \mathcal{N}(0,5^2)$ (C), ${\alpha }_i\sim U(-5,5)$, and ${\sigma }_i\sim U(0,3)$.

Although both clustering feedback and averaging feedback can generate accurate collective prediction, they require different scales of individuals’ beliefs (i.e., the values $c_A$). For a fixed population size $N$, with the averaging rule, the predicted coefficient is given by ${\widehat{\alpha }}_i=\sum _{A\in {\mathcal{N}}_i}{c}_A/n$. Thus, if there are more factors, there will be fewer individuals observing each factor, which implies that the magnitude of $c_A$ should be much larger than ${\alpha }_i$ in order that ${\widehat{\alpha }}_i={\alpha }_i$. For example, when the true coefficients lie in $[-5,5]$, the initial personal beliefs $c_A$ obey a normal distribution with SD $100$, which is much greater than the true coefficients (Fig. 3). When the environment depends on a larger number of factors ($m$ larger), the magnitudes of initial beliefs should be greater yet. Under clustering feedback, however, no matter how large the problem scale $m$, the magnitude of beliefs can remain the same as the true coefficients, because ${\widehat{\alpha }}_i=\sum _{A\in {\mathcal{N}}_i}{c}_A/n_i$ has the same scale as beliefs $c_A$ (SI Appendix, Fig. S6).

Comparison Between Niche-Expert and Feedback Payoffs.

Although both reward structures have the ability to foster collective intelligence, the niche-expert system has different properties, and outcomes, than the feedback system. To make a fair comparison between the two, we compare niche expert and feedback with the same aggregation rule, clustering aggregation, which requires the same magnitude of beliefs $c_A\sim {\alpha }_i$. (Feedback with average aggregation has similar properties as with clustering feedback, except for different requirements on the magnitude of beliefs, see SI Appendix, Fig. S6.)

Niche-expert and feedback payoff schemes respond differently to variation in the nature of the prediction task. We consider two scenarios. In the first scenario, we explore a form of task switching: After the population has already established collective intelligence, we assume that there is a shock so that all coefficients ${\alpha }_i$ are resampled, requiring that the population find a solution to a different prediction problem. Following the shock, we observe that a population evolving under feedback reward structure quickly recovers and reconverges to a state of perfectly accurate collective prediction. However, under the niche-expert reward structure, it is not guaranteed that the population can reestablish collective intelligence, when mutations are absent ($\mu =0$). The reason for this difference arises from the diversity of opinions supported under these two reward structures, shown in Figs. 2C and 3C. When the niche expert reaches equilibrium the belief diversity is lost. All individuals hold beliefs that are equal (or close) to the true coefficients. And so when a shock occurs and the coefficients are resampled from, e.g., a normal distribution, there will be no individuals whose beliefs are close to the new coefficients—so that imitation will not allow the niche-expert system to recover, except in the presence of innovation (Fig. 4A). Under feedback rewards, by contrast, the equilibrium state supports a diversity of beliefs (Fig. 4C). Even if the environment is changed, the new coefficients can still be expressed by individuals’ beliefs after subsequent imitation events.

Fig. 4.

Feedback payoffs are more robust than niche experts to variation in the nature of the prediction task. We consider two variant forms of prediction tasks for an environment that is a linear combination of $m$ factors: environmental shocks or correlated factors. We compare the performance of the collective predictions that emerge from feedback payoff versus niche expert, both under the clustering aggregating rule. (A) After reaching the stationary state of perfect collective accuracy, we introduce an environmental shock such that all coefficients of the $m$ independent factors are randomly redrawn. Under the clustering feedback payoff, the population rapidly adapts to recover near-perfect collective accuracy. The niche expert can also recover, although more slowly and only in the presence of innovations (mutations) during imitation. Panel (B) illustrates the average variance of beliefs within each cluster, which provides intuition for results in panel (A). For the niche expert, the beliefs within a cluster (individuals focused on the same factor) are all close to the true coefficient in equilibrium, so that the intracluster belief variance decreases toward zero over time. After an environmental shock, beliefs adapt to regain collective accuracy only through the accumulation of many mutations. For the clustering feedback, by contrast, belief diversity within clusters is retained in equilibrium, which guarantees that the new environment can still be expressed by individuals’ beliefs (even without mutation). (C) When the $m$ factors are correlated with covariance structure $({\sigma }_{\mathit{ij}})$, clustering feedback still produces collective accuracy, but the niche expert fails. (D) To understand the failure of the niche expert for correlated factors we focus on individuals within one cluster, $k=1$: Each blue dot represents such an individual’s belief. The blue line represents the predicted coefficient (average of beliefs ${\widehat{\alpha }}_1$) within cluster $k=1$, which converges to a value far from the true coefficient ${\alpha }_1$, so that collective prediction cannot be accurate. Note that predicted coefficient within cluster $k=1$ starts near zero, passes through the true value of ${\alpha }_1$, and then moves away from the true value, which also explains why the accuracy $R$ of niche expert first increases and then decreases (C). Parameters: $N=10,000$, $m=100$, $\mu =0.01$ ${\alpha }_i\sim U(-5,5)$, $c_A\sim \mathcal{N}(0,5^2)$, $({\sigma }_{\mathit{ij}})=(a_{\mathit{ij}}){(a_{\mathit{ij}})}^T$, and $a_{\mathit{ij}}\sim U(0,0.1)$.

We have also explored prediction tasks when the random factors are correlated with each other. In this case, the niche expert cannot maintain a high level of collective accuracy. However, clustering feedback still provides robust collective intelligence (Fig. 4B). In SI Appendix, section S3.1, we show that for niche-expert payoff, if the factors are correlated, among the individuals observing the same factor $X_i$ the individual with the highest payoff is not the one whose belief $c_A$ equals the coefficient ${\alpha }_i$. So in equilibrium, the individuals observing the same factor will have the same belief but not equal to ${\alpha }_i$, which means the predicted coefficient (Eq. 4) never equals the real coefficient (Fig. 4D, also see SI Appendix, Fig. S7). Under clustering feedback, by contrast, even though the expected payoff for an individual modified by correlated factors, ${\widehat{\alpha }}_i={\alpha }_i$ is still the stable equilibrium state of the system (SI Appendix, section S3.1); and so collective accuracy can still be ensured. Furthermore, if we assume that some factors that influence the outcome can never be observed, the performance of clustering feedback is again far superior to niche expert. In this case, strong correlations between factors will promote collective accuracy under clustering feedback, compared to the case of no correlation for clustering feedback; but correlations of any strength impede accuracy for niche expert (SI Appendix, Fig. S8). This is because feedback payoffs can incentivize individuals to infer partial information about unobservable factors through their correlation with observable factors.

The niche-expert system can be modified, by changing the aggregating rule (SI Appendix, section S3.1), to produce accurate predictions even with correlated factors. However, this form aggregation requires information about the covariance structure of the factors—which may in practice be unknowable—and it works well only if the factors are weakly correlated. Whereas clustering feedback does not require information about covariance structure and it is robust against even strong correlations between factors (SI Appendix, Fig. S9).

Niche-expert and feedback rewards also differ in their response to population size, $n$. In both cases, a large population is always beneficial for collective accuracy (Fig. 5). But the underlying mechanisms are qualitatively distinct. The effectiveness of the niche expert relies on the fact that there are some experts whose beliefs are very close to the real coefficients; and then other individuals will imitate those experts’ strategies. A larger population helps ensure that some individuals’ beliefs are close to the true coefficients, in the initial (randomly generated) composition of the population. Note that even though mutations can introduce new strategies whose beliefs may be close to the true coefficients, small populations remain unable to obtain accurate predictions, because there are very few individuals in each cluster, so that most strategy imitation occurs between different clusters without improving collective accuracy. By contrast, the feedback system does not require that someone’s belief be close to the real coefficient: Accuracy is achieved if the average belief for each factor (i.e., ${\widehat{\alpha }}_i=\sum _{A\in {\mathcal{N}}_i}{c}_A/n_i$) equals to the true coefficient. In a large population, the marginal change of ${\widehat{\alpha }}_i$ after each updating event is relatively small, which helps bring the predicted coefficient close to the real coefficient without overshoot events. And so the effect of population size has to do with the initial population composition for the niche expert; whereas it drives the dynamics of evolution for clustering feedback.

Fig. 5.

Large populations increase the accuracy of collective predictions. We present results for the niche-expert (A) and feedback payoff (B) structures (both with clustering aggregation), with initial beliefs in the population drawn uniformly at random. For a given population size, both payoff schemes perform better when the problem size is smaller (fewer factors, $m$). However, as the population size increases so does the collective accuracy, eventually reaching perfect accuracy. Evolution under the feedback payoff scheme requires a smaller population size to achieve perfectly accurate collective predictions, compared to the niche-expert scheme. Parameters: $c_A\sim \mathcal{N}(0,5^2)$, $\mu =0.01$, ${\alpha }_i\sim U(-5,5)$, and ${\sigma }_i\sim U(0,3)$.

Reward structures also have different behavior with respect to convergence speed—which grows with problem complexity $m$ under niche expert but is insensitive to problem complexity for feedback (SI Appendix, Fig. S10). Moreover, we have expanded our analysis to include situations when individuals can observe multiple factors at the same time (SI Appendix, section S3.2). In this setting as well, the feedback payoff structure engenders robust collective intelligence, whereas niche expert provides collective accuracy only when each individual observes either a few factors or nearly all factors (SI Appendix, Fig. S11).

Discussion

Collective intelligence, or the “wisdom of crowds,” is a pervasive phenomenon in real-world communities. Whether in decision-making through voting, scientific collaboration, technological innovation, or managing global economic systems, effective collaboration and organization among large numbers of individuals is indispensable for addressing complicated tasks (25–27, 43–45). A large body of evidence has shown that groups of individuals can often outperform single individuals in diverse tasks (1–5, 29). However, large population size alone is not sufficient to ensure the emergence of collective intelligence.

One approach to collective intelligence is to engineer algorithms for efficient distributed computation (6–11, 46, 47). Research in control theory and machine learning has produced remarkable solutions to distributed problem solving. These solutions are intended for execution by computers, though, and they do not describe how natural populations might achieve collective accuracy. Natural populations—even those under common governance—lack the degree of sophisticated central control required to implement solutions from engineering, which involve complex task allocation and dynamic feedback control. Instead, natural populations can access only very simple central authorities that might, for example, aggregate individual opinions and broadcast their average.

In this study, we have developed and analyzed two forms of reward structures (payoff functions) that effectively incentivize accurate collective predictions in a naturalistic population of individuals with limited information. The niche-expert scheme rewards individuals who not only make accurate predictions but also attend to information from underrepresented areas. This ensures a more comprehensive exploration of the problem space. The underlying mechanism of this payoff structure is reminiscent of division of labor in economics (48, 49), where the “invisible hand” balances the supply and demand of labor across various occupations. In equilibrium, switching to another occupation—or attending to another factor—yields no additional benefit.

The second type of reward structure, feedback payoff, differs fundamentally from the niche expert. Rather than rewarding individuals for the accuracy of their own predictions, this structure rewards individuals based on their contribution to reducing the collective prediction error. Reformers who can most effectively reduce the collective error—even if their own predictions are inaccurate—receive higher payoffs. This is roughly analogous to gradient descent optimization, where parameters are adjusted to minimize a global loss function (50, 51).

We have shown that the feedback scheme generates more effective collective accuracy across a wide range of prediction tasks, including environmental shocks or correlated factors. The primary distinction lies in the objective of the individuals under each payoff structure: Niche experts aim to optimize their own predictions, whereas feedback participants focus on improving collective predictions. The alignment of individual goals with collective objectives in the feedback scheme provides more robust performance.

In order to connect our theoretical results to empirical examples of collective intelligence, we must consider what real-world mechanisms could instantiate the niche expert or feedback reward structures we have analyzed. As an illustrative example, the niche-expert reward structure may naturally arise in the realm of scientific collaboration. Scientists are rewarded (by coauthorship on publications and grants) if they have expertise in one aspect of a larger problem (that is, in a single factor) and especially so if their expertise is rare within the scientific community so that they are in demand as coauthors. And so a form of niche-expert reward structure arises naturally in this setting, even without a central authority to measure the frequency of different strategies; it simply requires that individuals with rare expertise be disproportionally rewarded with scientific output.

Likewise, the feedback payoff structure is natural in the setting of a prediction market or a stock market. In this context, individuals buy and sell contracts whose prices reflect the group’s aggregate belief about a future event, such as the outcome of an election. A trader’s profit does not depend on whether his personal forecast is closest to the truth, but on whether his trades move the market price toward the eventual outcome. If a trader pushes the market price upward when the event indeed occurs, he profits; if he pushes the price downward in that case, he loses money. Here, a feedback payoff emerges endogenously from the trading mechanism: Individuals who act as reformers—by nudging the collective prediction closer to the truth—are automatically rewarded, even if their private estimates would not have been accurate in isolation. The same basic feedback operates in stock markets: When a stock is undervalued, traders who buy—even based on overly optimistic predictions—earn profits as the price moves upward.

Although these examples are illustrative of niche-expert and feedback rewards, we emphasize that our model provides only a simplified description of these real-world scenarios. The purpose of the simple model is not to produce a detailed, quantitative description of behavior among scientific collaborators or market traders, but rather to give intuition in a comprehensible framework for how reward structures can stimulate collective accuracy through social learning.

There is an extensive body of empirical literature on collective intelligence in natural populations (1–5, 29, 52–55). Corresponding theoretical studies have proposed various methods to improve collective intelligence in this setting (10, 30–32, 56, 57), and yet the field still lacks a systematic framework for studying how individual-level incentives promote collective accuracy. Most prior theory has focused on estimation tasks where the underlying truth is a constant or a single variable unaffected by external factors (10, 32, 56–58). These models, often grounded in opinion dynamics such as the DeGroot model (56, 58), assume that individuals’ opinions are influenced by their neighbors regardless of their performance. In contrast, our approach uses evolutionary game theory to model social learning through payoff-biased imitation, which describes individuals who are self-motivated and boundedly rational. While a few prior studies have also explored multifactor correlated truths, they are limited to binary voting tasks where both the underlying factors and environmental outcomes are binary (30, 31).

There are several shortcomings of our current analysis, and many directions for future research. We have assumed the population structure is well mixed, yet previous research has demonstrated that population structure can strongly influence information aggregation and behavioral spread by imitation (39, 59–63). There is some promise, then, that the reward structures we have developed may be more effective in structured populations, or they may require different aggregation rules that account for structure. We have also assumed that individuals give predictions based on the (true) value of observed factors. However, individuals might intentionally report false values of factors, in an attempt to achieve higher payoffs. Incentivizing individuals to provide honest predictions may require additional mechanisms, which remain to be explored. Finally, our study is limited to problems of collective prediction, as opposed to more general problem-solving tasks. Prediction of a continuous variable still encompasses a huge range of real-world problems that can be complex; but our analysis still assumes a linear relationship between the outcome and the underlying observable factors, which may be violated in real-world scenarios. Adapting our payoff structures and aggregation rules to account for nonlinear prediction problems is a significant challenge that remains an open problem for future research.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

There are no data underlying this work.