Adaptive social networks promote the wisdom of crowds

Significance

Under what conditions do groups outperform their individual members? This question is of paramount importance and has spurred many studies in management and organizational science, psychology, sociology, complex systems, and computer science. Here, we investigate the conditions under which the interaction network within a group can adapt to leverage the dispersion of individual abilities and promote collective intelligence. We find that network plasticity and feedback provide fundamental mechanisms for both improving individual judgments and inducing the collective “wisdom of the network,” allowing groups to outperform their best individuals working in isolation.

Abstract

Social networks continuously change as new ties are created and existing ones fade. It is widely acknowledged that our social embedding has a substantial impact on what information we receive and how we form beliefs and make decisions. However, most empirical studies on the role of social networks in collective intelligence have overlooked the dynamic nature of social networks and its role in fostering adaptive collective intelligence. Therefore, little is known about how groups of individuals dynamically modify their local connections and, accordingly, the topology of the network of interactions to respond to changing environmental conditions. In this paper, we address this question through a series of behavioral experiments and supporting simulations. Our results reveal that, in the presence of plasticity and feedback, social networks can adapt to biased and changing information environments and produce collective estimates that are more accurate than their best-performing member. To explain these results, we explore two mechanisms: 1) a global-adaptation mechanism where the structural connectivity of the network itself changes such that it amplifies the estimates of high-performing members within the group (i.e., the network “edges” encode the computation); and 2) a local-adaptation mechanism where accurate individuals are more resistant to social influence (i.e., adjustments to the attributes of the “node” in the network); therefore, their initial belief is disproportionately weighted in the collective estimate. Our findings substantiate the role of social-network plasticity and feedback as key adaptive mechanisms for refining individual and collective judgments.

Intelligent systems, both natural and artificial, rely on feedback and the ability to reorganize (1, 2). Such systems are widespread and can often be viewed as networks of interacting components that dynamically modify their connections. Cell reproduction relies on protein networks to combine sensory inputs into gene-expression choices adapted to environmental conditions (3). Neurons in the brain dynamically rewire in response to environmental tasks to enable human learning (4). Eusocial insects modify their interaction structures in the face of environmental hazards as a strategy for collective resilience (5). Fish schools collectively encode information about the perceived predation risk in their environment by changing the structural connectivity of their interaction (2). In the artificial realm, several machine-learning algorithms rely on similar concepts, where dynamically updated networks guided by feedback integrate input signals into useful output (6). The combination of network plasticity and environmental feedback is a widespread strategy for collective adaptability in the face of environmental changes. This strategy provides groups with practical and easy-to-implement mechanisms of encoding information about the external environment (2, 5).

The emergent ability of interacting human groups to process information about their environment is no exception to this use of feedback to guide reorganization. People’s behavior, opinion formation, and decision making are deeply rooted in cumulative bodies of social information, accessed through social networks formed by choices of who we befriend (7), imitate (8), cooperate with (9), and trust (10, 11). Moreover, peer choices tend to be revised, most frequently based on notions of environmental cues (such as success and reliability) or proxies such as reputation, popularity, prestige, and socio-demographics (12–15). The ability of human social networks to reorganize in response to feedback has been shown to promote human cooperation (9, 15, 16) and allows for cultural transmission over generations to develop technologies above any individual’s capabilities (17, 18).

However, there is substantial evidence showing that social influence increases the similarity of individual estimates (19–22), thereby compromising the independence assumption (i.e., that individual errors are uncorrelated, or negatively correlated) that underlies standard statistical accounts of “wisdom-of-crowds” phenomena (23). Furthermore, while it is commonly assumed that groups of individuals are correct in mean expectation (22, 24), humans’ independent estimates can be systematically biased (25, 26). However, although these independence and collective unbiasedness assumptions rarely hold in practice, the wisdom of crowds does emerge in human groups. Moreover, numerous studies have offered conflicting findings: On the one hand, there is evidence that social interaction can either significantly benefit the group and individual estimates (21, 27, 28); or, on the other hand, it can also lead them astray by inducing social bias, herding, and group-think (19, 20, 22). Notable efforts have focused on providing a resolution to these inconsistent conclusions. Studies have found that these divergent effects are moderated by placing well-informed individuals in prominent positions in the network structure (21, 22, 29); those individuals’ self-confidence (27, 30–32); the group’s ability to identify experts (33); dispersion of skills (34–36); quality of information (25); diversity of judgments (36, 37) or lack thereof (38); social learning strategies (39, 40); and the structure of the task (35, 40). In other words, whether social interaction is advantageous for the group depends on the environment in which the group is situated (41). Given that people often do not have access to all of the parameters of their environment (or the environment can change), it is advantageous to find an easy-to-implement mechanism that performs well across shifting environments.

Theoretical and experimental work on collective intelligence (including the reconciliation efforts mentioned above) has been predominantly limited to frameworks where the communication network structure is exogenous, as well as when agents are randomly placed in static social structures, such as dyads (27, 30), fully connected groups (20, 28, 42), or networks (21, 22). However, unlike what is explicitly or implicitly assumed in most existing work, the social networks we live in are not random, nor are they imposed by external forces (43). Rather, these social networks are shaped by endogenous social processes and gradual evolution within a potentially nonstationary social context. The present study builds on the observation that agent characteristics, such as skills and information access, are not randomly located in a network structure. Intuitively, groups can benefit from awarding centrality to—and amplifying the influence of—individuals with particular attributes (e.g., skillful or well informed). Consequently, the distribution of agents is often the outcome of social heuristics that form and break ties influenced by social and environmental cues (12–14, 44). Therefore, neither the emergent (“macroscopic”) structure nor individual (“microscopic”) properties can be decoupled from the characteristics of the environment (“context”). Hence, we hypothesize that dynamic social-influence networks guided by feedback may be central to context-adaptive collective intelligence (41), acting as core mechanisms by which groups, which may not initially be wise, evolve wisdom, adapting to biased and potentially nonstationary information environments.

Study Design

In this paper, we test the hypothesis that adaptive influence networks may be central to collective human intelligence with two preconditions: feedback and network plasticity. To this end, we address the following four research questions: 1) Do groups of interacting individuals outperform groups of unconnected individuals? 2) How does the plasticity of a network and the quality of the feedback affect the performance of that network? 3) Does the best individual in the group also benefit from social interactions? 4) What are the mechanisms that drive the system’s dynamics?

To answer these questions, we developed two web-based experiments (i.e., $E_1$ and $E_2$) and a simulation model to identify the role of dynamic networks and feedback in fostering adaptive “wisdom of crowds.” In the two experiments, the participants ($N_{E_1}=719;N_{E_2}=702$) recruited via the online labor market Amazon Mechanical Turk (SI Appendix, Section 1) engaged in a sequence of 20 estimation tasks. Each task consisted of estimating the correlation of a scatter plot, and monetary prizes were awarded in proportion to the participants’ performance at the end of the experiment. The participants were randomly assigned to groups of 12. Each group was randomized to one of three treatment conditions in $E_1$, where we varied the network plasticity; or four treatment conditions in $E_2$, where we varied the quality of feedback. To evaluate the generalizability of the findings and tune intuition about the underlying mechanisms, we also simulated a model of interacting agents in a separate context that was similar to that of our experiments.

Estimation Task: Guess the Correlation Game.

Earlier work demonstrated that estimating the correlation in scatter plots is an intuitive perceptual task that can be leveraged to investigate various aspects of our visual intelligence (45). We chose this judgment task for the following two reasons. First, this task can be rapidly performed without requiring participants to have specialized skills (45). Second, the task structure is both simple enough to vary systematically and rich enough to manipulate the quality of the information provided to the participants. In particular, we used scatter plots with three levels of signal quality (varying the number of points and adding outliers or nonlinearities; Fig. 1A). At every round, all plots seen by the participants shared an identical actual correlation. Still, the quality of the signal among them could differ (confer ref. 46). The design also allowed the simulation of a shock to the distribution of information among the participants. Specifically, each participant experienced a constant signal-quality level across the first 10 rounds; then, at round 11, we introduced a shock by reshuffling signal qualities to new levels that remained constant. The participants were not informed about the signal quality they or their peers faced (see SI Appendix, section 1 and Figs. S1 and S2 for further detail).

Fig. 1.

Experimental design. (A) Examples of the experimental task and the signal-quality levels. Provided are examples of the scatter plots used in the experiment. For any given round, all participants saw plots that shared an identical true correlation, but signal quality among them varied. Task signal quality was systematically varied at the individual level by varying the number of points, linearity, and the existence of outliers. The participants were not informed about the signal quality they or other participants were facing. (B) The overall experimental framework. Shown is an illustration of the experimental design. In experiment 1, the feedback level was fixed (i.e., full-feedback), and network plasticity was manipulated (i.e., a static network vs. dynamic network). In experiment 2, plasticity was fixed (i.e., always dynamic network), and feedback was manipulated (i.e., no-feedback, self-feedback, and full-feedback). The colors of the nodes represent the signal quality. Each participant experienced a constant signal quality level across the first 10 rounds; then, at round 11, we introduced a shock by reshuffling signal qualities to new levels that remained constant for the remaining 10 rounds.

Experiment 1 ($E_1$): Varies Network Plasticity; Holds Feedback.

In the first experiment ($E_1$, $N=719$), each group was randomized to one of the following three treatment conditions: 1) a solo condition, where each individual solved the sequence of tasks in isolation; 2) a static network condition, in which 12 participants were randomly placed in static communication networks with a fixed degree (i.e., each participant had exactly three neighbors); and 3) a dynamic network condition, in which participants at each round were allowed to select up to three neighbors to communicate with. Across all conditions, at each round, the participants were initially asked to submit an independent guess. Then, those in static and dynamic network conditions entered a social-exposure stage, where they could observe the answers of their network peers, update their own, and see their peers’ updated beliefs in real time. After submitting a final guess, the participants in all conditions were provided with performance feedback. Finally, those in the dynamic network condition were allowed to revise which peers to follow (up to three neighbors) in subsequent rounds (see SI Appendix, Fig. S3 for the experimental design and SI Appendix, Figs. S4–S8 for the online platform screenshots).

Experiment 2 ($E_2$): Varies Feedback; Holds Dynamic Network.

In the second experiment ($E_2$, $N=702$), each group was randomized to one of the following four treatment conditions: 1) a solo condition, where each individual solved the sequence of tasks in isolation; however, this time no performance feedback was provided; 2) a no-feedback condition, in which the participants were placed in a network, but were not shown any performance feedback; 3) a self-feedback condition, in which the participants were placed in a network and shown their own performance feedback; and 4) a full-feedback condition, in which the participants were placed in a network and shown performance feedback of all participants (including their own). The participants in all conditions in $E_2$ were allowed to revise which peers to follow (up to three total neighbors) in subsequent rounds, except for the solo condition, which acted as our baseline. Fig. 1B illustrates the overall experimental design.

Simulation: Varies Environmental Shock and Rewiring Rates.

Finally, we simulated interacting agents that update beliefs according to a DeGroot process (47) and rewire social connections according to a performance-based preferential attachment process (48) (see SI Appendix, section 2 for further detail). Using this model, we explored the effect of plasticity and the quality of feedback to compare with our experimental findings, examine the robustness of our results under different parameter values, and tune our intuition about the underlying mechanisms. In these simulations, we also explored the interaction between network adaptation rates—a network’s sensitivity to changes in agents’ performance—and the rate of environmental changes.

Results

Individual and Collective Outcomes.

We first compared individual- and group-level errors across all conditions. We observed that networked groups across studies and conditions significantly outperformed equally sized groups of independent participants. This result is consistent with previous research on complex problem-solving tasks (49, 50) and estimation tasks (21). Fig. 2 shows the individual and group error—using the arithmetic mean as group estimate—relative to baseline errors (i.e., subtracting the average error of the solo condition). Overall, we found that the participants in dynamic networks with full-feedback achieved the lowest error in both experiments. The dynamic networks, in the presence of feedback, gradually adapted throughout the experiment. The performance edge was more substantial in those periods when networks had adapted to their information environment (i.e., rounds $\left[\mathrm{6,10}\right]\cup \left[\mathrm{16,20}\right]$; hereafter referred to as adapted periods).

Fig. 2.

Individual and collective outcomes. (A–D) Groups connected by dynamic influence networks and provided with feedback incur substantially lower individual errors (A and C) and lower collective errors (B and D). The reduction is notably larger and more significant in the periods where networks had adapted to the information environment (i.e., rounds $\left[\mathrm{6,10}\right]$ and $\left[\mathrm{16,20}\right]$). Errors are relative to the errors in the solo condition. Error bars indicate 95% CIs. (A and B) Experiment 1: varying plasticity. (C and D) Experiment 2: varying feedback. E shows the basic overall (across the 20 rounds) mean absolute individual and collective errors and SDs for all conditions.

In particular, in $E_1$, as compared to the participants in static networks, dynamic networks had, on average, 17% lower individual error ($\beta =-0.038$, $z=-4.64$, $P<10^{-5}$; mixed model described in SI Appendix, Table S1) and 18% lower group error ($\beta =-0.03$, $z=-3.25$, $P=0.001$). The mixed model was conducted based on the absolute errors, and the $\beta$ coefficient refers to the difference between conditions. The percentages refer to the reductions in the absolute errors. In the adapted periods, dynamic networks reduced individual error by 36% ($\beta =-0.05$, $z=-6.56$, $P<10^{-6}$) and group error by 40% ($\beta =-0.04$, $z=-4.44$, $P<10^{-5}$).

Our simulation results corroborate this experimental result. We found that, in the presence of feedback, dynamic networks adapted to changes in the information environment by shifting influence to agents with better information, substantially decreasing individual and group error compared to unconnected groups (SI Appendix, Fig. S10).

In $E_2$, as compared to the no-feedback condition, having self-feedback marginally reduced the overall individual error (7%; $\beta =-0.015$, $z=-1.38$, $P=0.17$) and significantly reduced it in the adapted periods (21%; $\beta =-0.037$, $z=-3.52$, $P=0.0004$). On the group level, self-feedback had, on average, 11% lower error ($\beta =-0.02$, $z=-1.66$, $P=0.096$) and 29% in the adapted periods ($\beta =-0.038$, $z=-2.94$, $P=0.003$).

On the other hand, the full-feedback condition had on average 20% lower individual error ($\beta =-0.03$, $z=-3.3$, $P=0.001$) and 16% lower group error ($\beta =-0.02$, $z=-2.25$, $P=0.024$), as compared to the participants in the self-feedback condition. In the adapted periods, full-feedback reduced individual error by 32% ($\beta =-0.04$, $z=-4.99$, $P<10^{-5}$) and group error by 29% ($\beta =-0.03$, $z=-2.99$, $P=0.0028$; SI Appendix, Table S2).

As the full-feedback condition in $E_2$ and the dynamic network condition in $E_1$ were identical (i.e., both dynamic network and full-feedback), we considered them to be a replication of the same condition across two studies. Indeed, no statistically significant differences between the two conditions were observed (SI Appendix, Table S3).

In agreement with the experimental findings of $E_2$, simulation results confirmed that high-quality feedback is necessary for enabling beneficial group adaptation through social rewiring. As we added more noise to the peer-performance feedback, the collective performance of adaptive networks deteriorated until it converged to that of the simple wisdom of crowds (i.e., the solo condition; SI Appendix, Fig. S11).

The Performance of the Best Individual.

We found that even the best individual benefited from network interaction. The best individual within each group was determined based on ex post revised estimate performances across all rounds [prior research (51) refers to this as the “average best member”]—that is, based on the quality of the postsocial interaction estimates. In particular, we found that the best individuals in the dynamic and static conditions from $E_1$ reduced their overall error by approximately 20% ($P<10^{-4}$). In $E_2$, the best individuals in the full-feedback condition reduced their error by 30% ($P<10^{-4}$), while the best individuals in the self-feedback condition reduced their error by 21% ($P=0.009$) and 15% in the no-feedback condition ($P=0.057$). Again, these improvements were more apparent in the adapted periods, except for the no-feedback condition, where the best individual was not able to adapt (SI Appendix, Table S4).

As the best individuals were determined based on the postsocial interaction estimates, it is possible that those same individuals were not the best based on the initial estimate performance. Indeed, we observed that only 60% of the best presocial interaction individuals were also the best individuals after social interaction. However, even the best individual based on the presocial interaction performances benefited from group interaction. In particular, in $E_1$, the best individuals in the dynamic and static conditions reduced their overall error by about 7% ($P<10^{-4}$). In $E_2$, the best individuals in the full-feedback condition reduced their error by 12% ($P<0.004$). In contrast, the best individual in the no-feedback and self-feedback conditions did not improve.

Mean-Variance Trade-Off of Select Crowds.

The results of our experiments showed that the collective performance of groups was not bounded by that of the best individual. In a further exploratory analysis, we generalized the use of group means as collective estimates and the definition of best individual to analyze the performance of top-k estimates—that is, aggregate estimates where only the guesses of the $k$ best-performing group members in our experiments (ranking based on cumulative performance from prior rounds) were averaged. In particular, top-12 estimates corresponded to the group mean (i.e., the whole-crowd strategy) and top 1 to the estimate of groups’ best-performing individual (i.e., best-member strategy). Fig. 3 reports the mean and SD of estimation errors incurred by the entire range of $k$ (i.e., top-k or the select $k$ crowd strategy) estimates during the adapted periods. Ideal estimates would minimize both mean error and variability (i.e., toward the [0,0] corner). The qualitative shape of top-k curves reveals that, as we removed low-performing individuals (from $k=12$ to $k=1$), estimates initially improved in both mean and SD. Then, as we further curated the crowd (roughly—beyond $k\approx 4$), top-k estimates of the trade-off between decreasing mean error and increasing variability finally regressed in both objectives as $k\to 1$. As reported in a previous study, selecting an intermediate number of the best members strikes a balance between using the best estimates, on the one hand, and taking advantage of the error-canceling effects of averaging, on the other (35). Interestingly, we found that the full-group average in dynamic networks yielded 16% lower error ($P=0.02$; 500 bootstraps) and 39% less variability ($P<10^{-5}$; 500 bootstraps) than the best individual in the solo with feedback condition (i.e., dynamic top 12 vs. solo top 1).

Fig. 3.

Mean-variance trade-off. Mean and SD of absolute errors incurred by top-k estimates during the adapted periods (experimental data). Top-12 estimates correspond to the full-group mean and top 1 to the group’s best individual. Error bars indicate 95% CIs.

Adaptive Mechanisms.

The results supported our primary hypothesis that network plasticity and feedback may provide adaptiveness that can benefit both individual and collective judgment. To explain these results, we explored two mechanisms: global adaptation and local adaptation.

Global adaptation: Network structure.

The first mechanism we explored was a global (or structural) mechanism, where, in the presence of high-quality feedback, dynamic networks adaptively centralize over high-performing individuals. This behavior was predicted by cognitive science and evolutionary anthropology studies showing that people naturally engage in selective social learning (13, 17, 35, 44)—i.e., the use of cues related to peer competence and reliability to selectively choose who we pay attention to and learn from. Fig. 4 A and B show that the participants in dynamic networks used peers’ past performance information to guide their peer choices. Specifically, the overall correlation between popularity (i.e., number of connections) and performance (i.e., cumulative score) was strongest for dynamic networks with full-feedback ($r=0.62$) as compared to the self-feedback ($r=0.3$, $z=-0.319$, $P<0.001$) and no-feedback ($r=0.2$, $z=-0.383$, $P<0.001$) conditions. As rounds elapsed, performance information accrued, and social networks evolved from fully distributed into centralized networks that amplified the influence of well-informed individuals. Upon receiving an information shock, the networks slightly decentralized ($\beta =-0.046$, $z=-4.042$, $P<10^{-4}$; SI Appendix, Table S5), entering a transient exploration stage before finding a configuration adapted to the new distribution of information among the participants (see Fig. 4D for an example of the network evolution).

Fig. 4.

Mechanisms promoting collective intelligence in dynamic networks. A shows that the network becomes more centralized with time (Freedman’s global centralization $\in \left[\mathrm{0,1}\right]$—i.e., how far the network is from a star network structure). B depicts the relation between performance (i.e., average error) and popularity (i.e., number of followers). C shows the correlation between the accuracy of initial estimate and confidence (i.e., the degree to which participants resisted updating their initial estimate toward their peer’s estimates; Materials and Methods). Error bars indicate 95% CIs. D shows an example of the network evolution in one experimental trial. The circle color represents performance. The size of each circle represents the number of followers (i.e., popularity). The dashed orange line is the distribution of estimates before social influence; the solid blue line is the distribution of postsocial influence estimates. The dashed vertical line is the true correlation value. Corr, correlation.

Furthermore, using simulation, we explored the interaction between network global-adaptation rates—a network’s rewiring sensitivity to changes in agents’ performance—and the arrival rate of environmental shocks. The results of these simulations indicated that networks with higher adaptation rates are suitable for environments with frequent information shocks. Conversely, networks with slower adaptation rates could leverage more extended learning periods, eventually achieving lower error in environments with infrequent shocks (SI Appendix, Fig. S12). This short-term vs. long-term accuracy trade-off implies that optimal network-adaptation rates depend on the pace of changes in the environment, analogous to notions of optimal adaptation rates in natural systems (52) and learning rates in artificial intelligence algorithms (6).

Local adaptation: Confidence self-weighting.

However, network centralization over high-performing individuals cannot account for all of the results. First, a centralization mechanism alone would suggest that group members may merely follow and copy the best individual among them, hence, bounding collective performance by that of the group’s top performer, which we found to be untrue (i.e., even the best individual, on average across rounds, benefits from group interaction). Second, we found that, even in the absence of feedback, there was still a correlation between popularity and performance (Fig. 4B) that was similar to the self-feedback condition, but weaker than the full-feedback condition. Therefore, the first mechanism alone would not explain why the participants in the self-feedback condition in $E_2$ (but not those in the no-feedback condition) were able to adapt.

However, previous research on the two-heads-better-than-one effect indicated that, in the more straightforward case of dyads, even the best individual can benefit from social interaction (27, 30) and that the critical mechanism enabling this effect is a positive relationship between individuals’ accuracy and their confidence. This is a plausible local mechanism that can work in conjunction with both dynamic and static networks, as well as with or without feedback. Fig. 4C shows that, overall, the participants in the networked conditions had a positive correlation between the accuracy of their initial estimates and their confidence (measured behaviorally using the “weight on self” measure; Materials and Methods). The participants were likely to rely on private judgments whenever these were accurate and on social information otherwise. This positive correlation of confidence and accuracy is consistent with reported findings (21, 31). This mechanism allows the group members to exploit the fact that the “average best performer” is usually not the best on every single task item. That is, on some task instance, one or more other members may be the best. Therefore, even the group’s best performers can benefit from interacting with others if they are less resistant to social influence on the instances where they are incorrect. This mechanism also allows the individuals in the no-feedback conditions to overcome the absence of feedback by exploiting the round-by-round covariation between their own internal decision confidence (i.e., how certain they are) and the answers of their peers (53, 54). That is, if an individual is confident that she or he is right (according to his or her inner confidence), she or he can be equally confident that any neighbor who disagrees with him or her is wrong and, accordingly, down-weight his or her estimate in future rounds (or break ties). Therefore, even in the absence of explicit feedback, confidence is used as a signal to learn about the reliability of peers and could explain the positive correlation between popularity and performance. Moreover, Fig. 4C also shows that, as rounds elapsed, the participants in the full- and self-feedback conditions used the explicit performance feedback to calibrate their accuracy–confidence relationship further and were able to gradually readapt upon the shock.

Another plausible mechanism for the superior performance of the connected groups in our experiment is group-to-individual transfer (55). That is, group interaction might enable the members to come up with better individual (i.e., presocial interaction) estimates, and averaging these improved estimates leads to better group performance as compared to the solo estimates. However, since there were no significant differences between the quality of initial estimates across conditions, we found no evidence for this mechanism (SI Appendix, Fig. S15).

Discussion

Previous research on collective intelligence and social influence has not considered several aspects widespread in natural situations: 1) the rewiring of interaction networks; 2) the role of performance feedback; and 3) changing environmental conditions (i.e., shocks). In this study, we demonstrated that dynamic influence networks could adapt to biased and nonstationary environments, inducing individual and collective beliefs to become more accurate than the independent beliefs of best-performing individuals. We also showed that the advantages of adaptive networks are further amplified in the presence of high-quality performance feedback. Taken together, our results suggest that details of interpersonal communications—both in terms of the structure of social interactions and the mechanism of its evolution—can affect the ability of the system to promote adaptive collective intelligence. This provides evidence that dynamism of the networks has profound effects on the processes taking place on them, allowing networks to become more efficient and enabling them to better adapt to changing environments.

Although laboratory experiments are abstractions of real-world situations that have been deliberately simplified in order to test theories, drawing conclusions of immediate practical relevance would require performing far more realistic and complicated follow-up experiments. We also acknowledge that—as for most social systems—even subtle changes in environmental conditions (e.g., how a situation is framed, incentive structure, or the participants’ identities) can yield different outcomes (56). Said differently, even the results of a well-designed and highly realistic experiment may not generalize reliably beyond the specific conditions of the experiment. For instance, we speculate that network structure distortions to individuals’ local observations—such as information gerrymandering (57) and majority illusion (58)—may discourage the participants from searching for peers with high signal quality in a dynamic network with limited feedback. Therefore, robust knowledge of the conditions in which dynamic networks would lead to the wisdom of the crowd (as opposed to the madness of the mob) will require orders-of-magnitude more experiments where the available time, group size, task type, incentive structure, level of dispersion in abilities, group-interaction parameters, and many other potentially moderating variables would be manipulated. Although such a program is currently infeasible (or logistically challenging), “virtual laboratory” experiments, combined with numerical simulations of the type conducted in the present study, offer a promising route forward. We hope that the adaptive systems and environment-dependent views on collective intelligence will spur the establishment of connections with a variety of fields and advance an interdisciplinary understanding of the design of social systems and their information affordances.

Materials and Methods

The study was reviewed and approved by the Committee on the Use of Humans as Experimental Subjects at the Massachusetts Institute of Technology. All participants provided explicit consent. The experiment was developed by using the Empirica platform (59).

Statistical Tests.

All statistics were two-tailed and based on mixed-effects models that included random effects to account for the nested structure of the data—details of the statistical tests are in SI Appendix, Tables S1–S5.

Measuring Confidence.

Our behavioral measure of confidence was inspired by the weight on the self (WOS) measure frequently used in the literature on advice taking (60, 61). WOS quantifies the degree to which people update their beliefs (e.g., guesses made before seeing the peers’ guesses) toward advice they are given (the peers’ guesses). In the context of our experiments, it is defined as

$$WOS≔\left(u_2-m\right)/\left(u_1-m\right),$$

where $m$ is the neighbors’ average initial guess of the correlation, $u_1$ is the participant’s initial guess of the correlation before seeing $m$, and $u_2$ is the participant’s final guess of the correlation after seeing $m$. Note that the denominator takes into account where the participants fall in the distribution of estimates that they see. The measure is equal to 0 if the participant’s final guess matches the neighbors’ average guess, 0.5 if the participant averages their initial guess and the neighbors’ average guess, and 1 if the participant ignores the neighbors’ average guess.

Network Centralization.

To quantify the network centralization, we used Freedman’s centrality (62) that calculates the sum in differences in centrality between the most central node in the network and all other nodes relative to the theoretically largest such sum of differences (i.e., a star network—in which all nodes are directly connected to a common central node—of the same size):

$$C_x=\frac{\sum _{i=1}^N{C}_x\left(p_{*}\right)-C_x\left(p_i\right)}{max\sum _{i=1}^N{C}_x\left(p_{*}\right)-C_x\left(p_i\right)},$$

where $C_x\left(p_i\right)$ is the in-degree (number of followers) of individual $i$, $C_x\left(p_{*}\right)$ is the in-degree of the most popular individual, and $max\sum _{i=1}^N{C}_x\left(p_{*}\right)-C_x\left(p_i\right)=\left[\left(N-1\right)\left(N-2\right)\right]$ is the theoretically largest sum of differences. Freedman’s centralization is defined to be in the range of 0.0 (most decentralized structure—regular graph) to 1 (most centralized structure—star network).

Data Availability

Replication data and code are available at the Harvard Dataverse, https://doi.org/10.7910/DVN/EOYZKH (63).