Significance
We consider social norms as collections of beliefs or superstitions about events occurring in nature. These events have no inherent meaning, but individuals can choose to believe that they do. Specifically, individuals can interpret events as prescribing behaviors for themselves along with expectations of the behaviors of others. We show how evolutionary competition between such beliefs can give rise to both prescriptive and descriptive self-enforcing norms that can allow populations to coordinate behavior. Our model provides an evolutionary account for how normative meaning emerges in an inherently meaningless world.
Keywords: social norms, correlated equilibrium, cooperation, coordination, superstitions
Abstract
Social norms regulate and coordinate most aspects of human social life, yet they emerge and change as a result of individual behaviors, beliefs, and expectations. A satisfactory account for the evolutionary dynamics of social norms, therefore, has to link individual beliefs and expectations to population-level dynamics, where individual norms change according to their consequences for individuals. Here, we present a model of evolutionary dynamics of social norms that encompasses this objective and addresses the emergence of social norms. In this model, a norm is a set of behavioral prescriptions and a set of environmental descriptions that describe the expected behaviors of those with whom the norm holder will interact. These prescriptions and descriptions are functions of exogenous environmental events. These events have no intrinsic meaning or effect on the payoffs to individuals, yet beliefs/superstitions regarding them can effectuate coordination. Although a norm’s prescriptions and descriptions are dependent on one another, we show how they emerge from random accumulations of beliefs. We categorize the space of social norms into several natural classes and study the evolutionary competition between these classes of norms. We apply our model to the Game of Chicken and the Nash Bargaining Game. Furthermore, we show how the space of norms and evolutionary stability are dependent on the correlation structure of the environment and under which such correlation structures social dilemmas can be ameliorated or exacerbated.
Cooperation and coordination in social settings are fundamental to the functioning of human and animal communities. Considering purely self-interested actors, cooperation and coordination should be rare if there are conflicts of interests between players and if no external incentives (or coercion) are at hand to resolve these conflicts. Yet, we observe many examples of cooperation without external incentives mediated by informal mechanisms, such as social norms (1–3). It is apparent that, while humans and animals do tend to make decisions to improve their payoffs, individual decision making is pervasively affected by both prescriptive beliefs on their own behavior and expectations of others’ behavior (4). These prescriptions and expectations may or may not be rooted in reality or accurately describe others’ behaviors, but regardless, they have a real effect on behavior (5–8). Importantly, they can induce regularities in a population that allow self-interested agents to cooperate and coordinate with each other efficiently. These beliefs can endow events and cues in the world that are inherently meaningless with normative meaning to voluntarily coordinate individual behavior. We address how social norms that give rise to such beliefs can arise through an evolutionary game theory model that combines rationality, superstition, and evolutionary dynamics to coordinate behaviors on external events without design.
Although social norms coordinate behavior, any individual or the society at large generally does not explicitly design them; rather, they are a collective phenomenon that emerges from the interaction of individuals. Thus, game theory is a common tool to model the emergence of social norms (9–11). Many different game theoretic conceptions of social norms exist. Some norms can be seen as mechanisms that select from (the typically numerous) Nash equilibria in repeated games (12). Alternatively, some social norms consist of conditional behavioral rules (3) that convert social dilemmas into coordination games: for example, by external enforcement (either through institutions or by others in a group) (13, 14). Our focus here is on social norms that lack external enforcement but still are followed, because agents, given their beliefs about the world (also induced by social norms), find it in their interest to do so. In other words, we are interested in both descriptive norms that tell individuals what behaviors to expect from others and prescriptive norms that tell individuals how to behave and can be predicated on descriptive norms (15). More specifically, we model social norms that act as “choreographers” (16, 17) that induce correlated beliefs in agents, allowing them to coordinate on a correlated equilibrium (18, 19) of the game.
A correlated equilibrium is generated by a correlating device [the choreographer (16, 17)] that suggests pure strategies to the players. If the suggestions to each player are correlated in a certain way and the agents know these correlations, it can be optimal for the players to follow the suggestions of the choreographer. Such behavior is called a correlated equilibrium. The Game of Chicken is an exemplar for the coordinating effects of correlated equilibria, as it can feature a scenario in which coordination would lead to a socially optimal yet game-theoretically unstable state. Consider two cars approaching an intersection: one northbound and the other westbound. The drivers may choose to drive through the intersection or stop. The payoff matrix for this game is
| [1] |
where is the payoff to a player playing the row strategy vs. an opponent playing the column strategy. The cooperative strategy is to stop, which is to “chicken out,” and both players stopping is the socially optimal state. The problem is that this is not a Nash equilibrium of the game. Yet, if somehow a convention emerges where the westbound car has the right of way, it can coordinate interactions and avoid collisions. This convention is analogous to the Bourgeois Strategy of evolutionary game theory (8, 20), where a norm of respecting current ownership of a resource may coordinate fighting over it. Another analogous example is when animals use conspicuous natural landmarks in the environment as focal points for territory boundaries (21, 22). In all of these cases, an event or cue in the world that has no inherent meaning for the interaction ends up having normative meaning, because individuals in the population conventionally condition their behavior on it.
In the classical conception of correlated equilibrium, there is a third party, the choreographer, that assigns the meanings to the cues and decides on the correlations between the suggestions, which then are assumed to be common knowledge between individuals. The city putting a traffic light at our intersection would be an example of this. Here, we are concerned with the case where there is no central party that decides on the normative meaning, and players do not know correlations between cues. Instead, we ask how prescriptive and descriptive individual beliefs about events in the world compete against each other decentrally. In effect, we ask whether natural selection can serve as a “blind choreographer” that shapes individuals’ beliefs about stochastic events in the world such that they play correlated equilibria without knowledge of the correlation structure of the world or external enforcement.
Our work belongs to a body of game theory that has studied how correlated equilibria can emerge in a decentralized way through evolutionary or learning dynamics (23–32). The first strand in this body starts with the seminal paper of Selten (23) who proved that evolutionarily stable strategies conditional on cues in the world have to be pure strategies. Later, Cripps (24) and Shmida and Peleg (25) showed that these correspond to correlated equilibria of the underlying game (31). Building on these results, Metzger (32) modeled the evolution of the cue distribution itself through a centralized mechanism where all players have to agree to change the distribution of signals. A second strand of literature (26–30) is concerned with learning in repeated games, where the history of the interaction (26–28) or selection of players (29) or an exogenous correlation device (30) acts as the choreographer, and it shows that various learning dynamics can converge to correlated equilibria after repeated play.
Our paper is closest in spirit to the first strand. We consider nature as providing a collection of correlated events in the world and ask how behavioral strategies conditional on these event spaces might evolve. Our main contribution is that we explicitly consider prescriptive and descriptive beliefs that individuals might impute from such events. We model the evolution of such beliefs under the assumption that, given their beliefs, agents make individually rational decisions. This is analogous to the “indirect evolution” approach to the evolution of utility functions (33–36). In this case, we model how evolution drives individual behavior indirectly by shaping normative beliefs in a population. Our approach is aimed at explaining not only norms as correlated behaviors but also, the belief structures that support them in the absence of central authorities that impose or justify these beliefs. We show how belief structures about events in the world can emerge and be both individually rational and evolutionarily stable, although the events are inherently meaningless to the interaction. We are particularly interested in this phenomenon with respect to noncooperative games that feature a social dilemma where the socially optimal solution is evolutionarily unstable. As illustrative examples, we apply our general framework to the Game of Chicken and the Nash Bargaining Game.
Model
Consider two players meeting to play a game. Each player privately observes an event, which is irrelevant to the game being played but may be correlated with their opponent’s event and other events in ways unknown to the players. We consider social norms that assign to these events a set of labels, prescriptions, and descriptions of opponent behaviors.
Players can assign a private label to these events to differentiate between them. The events and labels have no inherent association with one another. As a function of the observed label, players’ social norms prescribe a pure strategy recommendation. We assume that players cannot observe the events that their opponents see or the prescriptions that they may have received. Instead, each players’ norm gives them a (private) conditional probability distribution of prescriptions that their opponent may be receiving given the recommendation that they themselves have been given. This conditional probability distribution may or may not correspond to the actual conditional distribution of prescriptions or behavior of a real opponent. It simply represents a player’s subjective expectation given by its social norm. The players then may obey the norm’s prescription or play a default strategy from which they will earn a payoff. The prescription of the norm is only obeyed if it represents a best response to the description of the opponent behavior given by the norm.
Fig. 1 illustrates our model graphically in a world with four events and two players having differing norms. An event is chosen where each player observes a cat. In this case, Player 1 observes the black cat, and Player 2 observes the white cat. Player 1’s norm prescribes cooperation when it observes black or white cats and defection for gray and tabby ones. On the descriptive side, Player 1’s norm specifies a conditional distribution of behaviors of Player 2. Since this distribution is conditional on the labels, Player 1’s expectation of Player 2’s behavior is the same when observing black and white cats. Observing a black or white cat, Player 1 believes that Player 2 will cooperate with probability and defect otherwise. Notice that a norm’s prescriptions can be different from a description of those prescriptions. This discord is illustrated by Player 2’s norm, which could well be described as impudent; they believe that they should defect regardless of which cat they see and expect their opponent to always cooperate.
Fig. 1.
A pictorial representation of the model. Here, each vertex on the graph corresponds to the observation of a different colored cat. As an example, Player 1 and Player 2 meet, and they observe a black (B) cat and a white (W) cat, respectively. Player 1’s norm recommends that they cooperate (represented in white) and given that Player 1 has been told to cooperate, claims that Player 2 will cooperate with probability . Player 2’s norm recommends that they defect (represented in black) and claims that Player 1 will always cooperate. G, gray; T, tabby.
Mathematically, we model the world of events as an undirected graph, , where the edges, , represent interactions between two players via a game and the vertices privately observed events for those games. This construction is essentially equivalent to specifying a join probability distribution as in refs. 18, 19, and 23. However, we separate player interactions and events so that events may be aggregated under particular labels, allowing changes in the observed (i.e., label) distribution. Players then play each other a large number of times such that they observe all events. When a game is played, each player is equally likely to be privately assigned either vertex of the edge (this model may be extended to incorporate vertex assignment dependent on “types,” although we do not consider this case here). In the event that the edge is a self-loop, both players are assigned the same vertex (although this is still privately observed). The probability of selecting edge is , a weighted symmetric adjacency matrix. We limit the number of ways in which a player can distinguish vertices by defining a set of labels, , to apply to each vertex. If two vertices have the same label, then the player cannot differentiate between the two. In matrix notation, we define this as follows. Given a set of labels , let be the label matrix, which partitions the vertices into labels (i.e., if has label ; otherwise, ).
Norms prescribe pure strategy recommendations to labels. We can represent the behaviors that players are prescribed by the prescriptive behavior matrix, , where if the norm prescribes the player to play given . If , players can be prescribed a particular strategy for each vertex individually. In the case where the norm has no prescription for a label, the players play a default strategy, such as a symmetric (mixed) Nash equilibrium. The expected behavior of an opponent is determined by the column stochastic descriptive behavior matrix, , which is the expected behavior of an opponent given a label observed by the focal player. Given these constructions, a norm is defined by the matrix triplet .
We examine the effects of a variety of competing norms on a given event graph. In playing an opponent with a differing norm, a player will still follow its own norm and thus, will not necessarily receive the expected payoff that their norm would dictate. Norm change can be facilitated by such disparity between expectations generated by descriptive norms and reality (37). However, we assume that players do not reflect on—or at least do not do so as to modify—their norms directly but can observe the fitness of competing norms. Thus, norm change is generated by imitation dynamics, whereby players imitate the norms of those with higher fitness (38). The expected payoff of following norm using playing norm with is a function of the prescriptions for each player at each interaction multiplied by the payoff from that strategy pairing. Mathematically, this is
| [2] |
The payoff playing the whole population is thus the sum of this expected payoff over all norms weighted by their frequency. Note that this expected payoff does not directly depend on , the descriptive behavior matrix, since only describes the beliefs that a focal individual has about the behavior of an opponent, not the actual behaviors. Indirectly, can affect the payoff, since it can determine whether a focal individual obeys the prescriptive norm or not.
Applying this equation to the example in Fig. 1 with equally weighted edges, we have
| [3] |
Results
A Classification of Norms.
Our definition of social norms captures the idea that norms imbue an inherently meaningless world with prescriptions on self-behavior and expectations about others’ behaviors. However, we do not assume that the prescriptions of a social norm are to be blindly obeyed. In particular, we assume that a norm will only be obeyed when—given the expectations from the descriptive aspect of the norm—individuals cannot improve their payoffs by deviating from the social norm’s prescriptions. In this section, we will first define the rationality condition and then, classify several important norms (depicted as a Venn diagram in Fig. 2). In Evolutionary Stability of Norms, we will address evolutionary concerns.
Fig. 2.
A Venn diagram of the set of norms. All norms can be partitioned into null and rational norms. Empirical norms can be partitioned into consistent (which implement a correlated equilibrium when playing itself) and inconsistent norms. Inconsistent norms may implement a correlated equilibrium with complementary inconsistent norms but not with themselves. Best-response norms (BR) are a subset of evolutionary stable norms (ES).
Let be a social norm. The expected payoff for obeying the prescription at label is . It is rational for players to obey their norm’s prescriptions if each prescription at a label is a best response to an opponent’s expected behavior at that label: that is,
| [4] |
for every label and prescription . If a norm does not obey [4], then we assume that players who have it will disregard both the recommended prescriptions and the descriptions to play a default strategy at all events (SI Appendix, section 2 has a discussion of alternative assumptions for this case). We will call such norms null norms. If the game features a dominant strategy, we assume that this is the default. However, if the game features a symmetric mixed Nash equilibrium, we assume that that is the default. In the case of multiple such strategies, we have a family of null norms.
Although the descriptive aspect of a norm does not have to accurately describe opponents’ behavior, there exists a class of norms that does fit the behavior of (possibly hypothetical) opponents. In particular, a norm is empirically validatable if its prescriptions are rational against the behavior of a rational norm. This class is of interest, since two norms that are empirically validatable with respect to one another implement a correlated equilibrium when they play one another (SI Appendix, Lemma S1).
We are also interested in a particular class of norms where everyone would obey their norm given that everyone else also obeys the same norm (39). The prescriptive part of such norms must be rational vs. the behavior of opponents that follow the prescriptions of the same norm. In other words, the norm is empirically validated by itself. We call these norms consistent norms. In contrast, the prescriptive part of an inconsistent norm does not follow from a description of its own behavior but rather, is rational against a player obeying a different set of prescriptions. For example, consider the event pair {black cat, white cat}, in which the two cats are completely correlated with one another. Suppose that, when a black cat is observed, a norm prescribes defection and expects cooperation from the opponent. If this norm is consistent, it will prescribe cooperation and expect defection when a white cat is observed. However, an inconsistent norm can hypocritically prescribe its carrier to also defect and expect cooperation in the event that the carrier sees a white cat. Mathematically, if a norm is empirically validatable for , then the social norm is consistent. Note that, if a norm is consistent, it necessarily implements a correlated equilibrium playing against itself. However, if a norm is empirically validatable but is not rational for , then the norm is inconsistent. The left-hand norm in Fig. 1 is consistent, and the right-hand norm in Fig. 1 is inconsistent. It is worth a reminder that players do not explicitly know the event graph and therefore, cannot directly evaluate whether the norm is consistent or inconsistent. Nonetheless, as we will see in the next section, consistent and inconsistent norms have fundamentally different evolutionary properties.
We label a subset of consistent norms where the prescriptions for each vertex of an edge are best responses to one another as best-response norms. A best-response norm always exists if the game admits a single-strategy Nash equilibrium. However, for a two-strategy pure Nash equilibrium, a best-response norm exists if and only if the graph is bipartite. Since we may assign to each partition one of the strategies, each vertex along an edge is a best response to each other. It follows that the fitness of a best-response norm playing itself is the mean of the payoffs of the pure Nash equilibria (SI Appendix, Lemmas S2 and S3).
Evolutionary Stability of Norms.
Similar to the concept of an evolutionary stable strategy (40), an evolutionary stable norm is a norm that cannot be invaded by an initially rare norm (SI Appendix, section 1 has the formal definition). We are concerned with the evolutionary history initiated at a state without prescriptions where everyone plays a null norm using a mixed Nash equilibrium. As long as the prescriptions are not obeyed ([4] is not satisfied), they may accumulate in the population by drift, since they remain playing the mixed Nash strategy. However, on being rational, they are obeyed. If such a norm is a consistent norm, it may then invade if its prescriptions are in the hull of the mixed Nash strategy. Any strategy within the support of a mixed Nash playing the mixed Nash strategy receives the mixed Nash payoff. Furthermore, as a consistent norm implements a correlated equilibrium, its payoff playing itself is greater than any alternative norm under the same event labeling. Since the null norm’s behavior is identical across all events, the labeling is inconsequential and thus, receives a lower fitness playing the consistent norm. Therefore, the consistent norm can invade. More generally, the same arguments applied to a subset of events show the emergence of obeyed prescriptions from null prescriptions. In other words, the evolutionary dynamics of norms tend to manufacture beliefs that coordinate interactions to correlated equilibria out of inherently meaningless events (SI Appendix, Lemma S4 has mathematical details of these arguments).
If the null norm plays a mixed Nash, evolutionarily stable norms must be consistent norms and thereby, implement correlated equilibria (SI Appendix, Lemma S6). In this way, evolutionarily stable norms in our model allow the decentralized emergence of not only coordinated behaviors but also, coordinated beliefs about the correlation device.
Since consistent norms induce a correlated equilibrium, they can be evolutionarily stable against norms that share the same relative labeling of events (i.e., as long as events are labeled such that they are differentiated in the same ways). Norms with different labels may invade a consistent norm that is evolutionarily stable against norms with the same labeling unless the resident norm is a best-response norm (SI Appendix, Lemma S7). Unlike consistent norms, inconsistent norms cannot be evolutionary stable, because they can be invaded by a norm that validates their descriptions, which by definition, is different from themselves (SI Appendix, Lemma S5). However, two such inconsistent norms that empirically validate one another form a stable polymorphism (SI Appendix, Lemma S8).
We may have many other types of polymorphisms depending on the underlying graph and labeling. Monomorphisms of consistent norms are not necessarily stable, since [4] only holds for a norm’s labeling. An implication of this is that a consistent norm that can invade an inconsistent norm will not necessarily fix, and a polymorphism may thus result. This is because the expected payoff of an inconsistent norm against the consistent norm may be higher than the consistent norm against itself if they have different labelings. However, if the consistent norm’s labeling is complete (i.e., each vertex has a unique label), then the invading consistent norm will fix (SI Appendix, Lemma S9). Another example of a polymorphism is that of weakly evolutionarily stable consistent norms with differing prescriptions, which can exist as long as, on average, the pairings of prescriptions are the same or trivially, if the null norm and consistent norms’ behaviors are identical (SI Appendix, section 4).
Examples.
The Game of Chicken.
The existence and evolutionary stability of norms for a game are dependent on the underlying event space. Consider the bipartite graph in Fig. 3 for the Game of Chicken. There is one behavior that a null norm can play, namely the mixed Nash equilibrium, where they cooperate at each vertex with probability . The set of consistent norms is composed of five norms represented graphically as in Fig. 3 (note that and require an appropriate labeling to be rational). , , and are evolutionarily stable, while the other two norms are not.
Fig. 3.
Several graphical representations of the prescriptions of consistent norms. White corresponds to a prescription to cooperate and black corresponds to defect in graph . , , and are evolutionary stable norms. provides a higher fitness than and , which are best-response norms. In contrast, the only evolutionarily stable norms for graph are best-response norms. Black corresponds to demanding and white corresponds to demanding in graph for the Nash Bargaining Game.
Continuing with the example above, we may consider all prescriptions as a strategy from which we have a 12-dimensional replicator equation, the symmetric Nash equilibria of which are the fixed points (38). There are 25 fixed points of this model, 3 of which are stable (the monomorphic states of the three stable consistent norms). For monomorphic populations, has a higher fitness than the null norm and the two best-response norms, and its inverse. Contrast this result to another bipartite graph, . The evolutionary stable norms of graph are all best-response norms (the one depicted in Fig. 3 and symmetries of it), which all have fitness , lower than the fitness of of 5.
The evolution of fairness in the Nash Bargaining Game.
The Nash Bargaining Game provides us with another interesting example (41). Each player’s strategy is a demand of a portion of a dollar. If the sum of the demands is less than or equal to a dollar, then each player receives their demand. If, however, the sum exceeds a dollar, then they both receive nothing. A variety of solutions (corresponding to varying definitions of fairness) have been found (41–43). We are interested in how a fair (50/50) division of the dollar can emerge. It has been shown that a single-population replicator equation model can converge to such a fair distribution for a symmetric linear Nash Bargaining Game (44). However, the history of play matters, and the basin of attraction to a suboptimal polymorphic stable equilibrium is not negligibly small. A fair division of the surplus between tenants and landlords is also stochastically stable if memory is large and noise is small (45). Here, players play the linear symmetric Nash Bargaining Game at each edge.
Let us first consider fairness across event space (i.e., the variance of the expected payoff) for evolutionarily stable consistent norms prescribing either or . If the graph is bipartite, then best-response norms may exist. Therefore, prescribing any can be evolutionarily stable. Examples include the ring graph with an even number of vertices and the square grid graph. The ring graph with an odd number of vertices (such as in Fig. 3) and the complete graph are counterexamples. For these graphs, the range of for consistent norms is constrained by a tradeoff between the distributions of each strategy and the demands. For example, for the complete graph, the proportion playing is approximately (SI Appendix, section 5, Lemma S10). However, the only distribution that can reach the optimal expected payoff is , the fair distribution. Since each event is equally correlated with every other event, this result can be interpreted as an expression of the Veil of Ignorance (46).
To explore the effects of the graph type and number of vertices on the evolution of fairness in the Nash Bargaining Game, we used trait substitution simulations on the complete graph, ring graphs, and small world graphs (47) (details are in SI Appendix, section 5), where the dollar can be divided into either quarters or dimes. These simulations permit both monomorphisms of consistent norms and polymorphisms of inconsistent norms. Fig. 4 depicts the results for the complete and degree 2 ring graph (remaining results are in SI Appendix, section 5). The fair distribution is frequently reached and is primarily a monomorphic state of a consistent norm. Polymorphisms of inconsistent norms can emerge and stabilize, leading to suboptimal mean fitness. In general, monomorphic states are more likely to lead to higher fitness than polymorphisms. For example, if the dollar can only be split into quarters on the complete graph, we have 1/3 of the population demanding of the dollar, and the remaining demanding , from which we have a mean fitness of . Going from quarters to dimes decreases the frequency of the fair outcome, as the cost of miscoordination becomes smaller.
Fig. 4.
Mean fitness frequency in the Nash Bargaining Game for the complete (A and B) and ring (C and D) graphs and for dollar refinements of quarters (A and C) and dimes (B and D). Red corresponds to monomorphic states, and blue corresponds to polymorphic states.
Discussion
In this paper, we show how social norms that coordinate behaviors can be bootstrapped from random superstitions about irrelevant events in the world. A key to our model is the synthesis of prescriptive and descriptive norms. Inconsistencies between prescriptive and descriptive norms have been experimentally shown to weaken norm adherence (48, 49). As such, our rationality and empirical validation conditions are key to aligning prescriptive and descriptive norms and inducing their obeyance. We show that empirically validatable norms align not only the behaviors of the players but also, their beliefs about others’ behaviors in a correlated equilibrium. Such social norms act as choreographers of the players’ behaviors, potentially ameliorating social dilemmas, a well-studied product of social norms (50). However, our players are blind to the true correlations between events in the world, and as such, this coordination is unintentional. Rather than being designed, the choreography of play through consistent belief emerges from the accumulation of random superstitions of the environment. In alignment with empirical work (51, 52), our norms start out at the individual level as subjective norms ascribing meaning to events in the world: whether they be seeing a colored cat or something equally superstitious. If these subjective norms are rational to follow given the beliefs that they induce and yield a high payoff, they can win in evolutionary competition and acquire conventional normative meaning. In this way, normative meaning invented at the personal level provides the material for conventional meaning to emerge through the joint action of individual rationality and evolutionary dynamics.
Supplementary Material
Acknowledgments
Funding was provided by Defense Advanced Research Projects Agency Next Generation Social Science Program Grant D17AC00005 and Army Research Office Grant W911NF-12-R-0012-03.
Footnotes
The authors declare no conflict of interest.
This article is a PNAS Direct Submission.
This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1817095116/-/DCSupplemental.
References
- 1.Ostrom E. Collective action and the evolution of social norms. J Econ Perspect. 2000;14:137–158. [Google Scholar]
- 2.Fehr E, Fischbacher U, Gächter S. Strong reciprocity, human cooperation, and the enforcement of social norms. Hum Nat. 2002;13:1–25. doi: 10.1007/s12110-002-1012-7. [DOI] [PubMed] [Google Scholar]
- 3.Bicchieri C. The Grammar of Society: The Nature and Dynamics of Social Norms. Cambridge Univ Press; Cambridge, UK: 2006. [Google Scholar]
- 4.Gintis H. A framework for the unification of the behavioral sciences. Behav Brain Sci. 2007;30:1–16. doi: 10.1017/S0140525X07000581. [DOI] [PubMed] [Google Scholar]
- 5.Jacobs RC, Campbell DT. The perpetuation of an arbitrary tradition through several generations of a laboratory microculture. J Abnorm Soc Psychol. 1961;62:649–658. doi: 10.1037/h0044182. [DOI] [PubMed] [Google Scholar]
- 6.Cialdini RB, Kallgren CA, Reno RR. A focus theory of normative conduct: A theoretical refinement and reevaluation of the role of norms in human behavior. In: Zanna MP, editor. Advances in Experimental Social Psychology. Vol 24. Academic; New York: 1991. pp. 201–234. [Google Scholar]
- 7.Etzioni A. Social norms: Internalization, persuasion, and history. Law and Soc Rev. 2000;34:157–178. [Google Scholar]
- 8.Gintis H. The evolution of private property. J Econ Behav Organ. 2007;64:1–16. [Google Scholar]
- 9.Voss T. Game theoretical perspectives on the emergence of social norms. In: Hechter M, Opp KD, editors. Social Norms. Russell Sage Foundation; New York: 2001. pp. 105–136. [Google Scholar]
- 10.Gintis H, Helbing D. Homo socialis: An analytical core for sociological theory. Rev Behav Econ. 2015;2:1–59. [Google Scholar]
- 11.Paternotte C, Grose J. Social norms and game theory: Harmony or discord? Br J Philos Sci. 2012;64:551–587. [Google Scholar]
- 12.Binmore KG. Game Theory and the Social Contract: Just Playing. Vol 2 MIT Press; Cambridge, MA: 1998. [Google Scholar]
- 13.Bowles S, Choi J-K, Hopfensitz A. The co-evolution of individual behaviors and social institutions. J Theor Biol. 2003;223:135–147. doi: 10.1016/s0022-5193(03)00060-2. [DOI] [PubMed] [Google Scholar]
- 14.Tavoni A, Schlüter M, Levin S. The survival of the conformist: Social pressure and renewable resource management. J Theor Biol. 2012;299:152–161. doi: 10.1016/j.jtbi.2011.07.003. [DOI] [PubMed] [Google Scholar]
- 15.Chung A, Rimal RN. Social norms: A review. Rev Commun Res. 2016;4:1–28. [Google Scholar]
- 16.Gintis H. Social norms as choreography. Polit Philos Econ. 2010;9:251–264. [Google Scholar]
- 17.Gintis H. Individuality and Entanglement: The Moral and Material Bases of Social Life. Princeton Univ Press; Princeton: 2016. [Google Scholar]
- 18.Aumann RJ. Subjectivity and correlation in randomized strategies. J Math Econ. 1974;1:67–96. [Google Scholar]
- 19.Aumann RJ. Correlated equilibrium as an expression of bayesian rationality. Econometrica. 1987;55:1–18. [Google Scholar]
- 20.Smith JM. Evolution and the Theory of Games. Cambridge Univ Press; Cambridge, UK: 1982. [Google Scholar]
- 21.Eason PK, Cobbs GA, Trinca KG. The use of landmarks to define territorial boundaries. Anim Behav. 1999;58:85–91. doi: 10.1006/anbe.1999.1133. [DOI] [PubMed] [Google Scholar]
- 22.Mesterton-Gibbons M, Adams ES. Landmarks in territory partitioning: A strategically stable convention? Am Naturalist. 2003;161:685–697. doi: 10.1086/374203. [DOI] [PubMed] [Google Scholar]
- 23.Selten R. A note on evolutionarily stable strategies in asymmetric animal conflicts. J Theory Biol. 1980;84:93–101. doi: 10.1016/s0022-5193(80)81038-1. [DOI] [PubMed] [Google Scholar]
- 24.Cripps M. Correlated equilibria and evolutionary stability. J Econ Theory. 1991;55:428–434. [Google Scholar]
- 25.Shmida A, Peleg B. Strict and symmetric correlated equilibria are the distributions of the ess’s of biological conflicts with asymmetric roles. In: Albers W, Guth W, Hammerstein P, Moldovanu B, van Damme E, editors. Understanding Strategic Interaction. Springer; Berlin: 1997. pp. 149–170. [Google Scholar]
- 26.Foster DP, Vohra RV. Calibrated learning and correlated equilibrium. Games Econ Behav. 1997;21:40–55. [Google Scholar]
- 27.Hart S, Mas-Colell A. A simple adaptive procedure leading to correlated equilibrium. Econometrica. 2000;68:1127–1150. [Google Scholar]
- 28.Hart S. Adaptive heuristics. Econometrica. 2005;73:1401–1430. [Google Scholar]
- 29.Ianni A. Learning correlated equilibria in population games. Math Soc Sci. 2001;42:271–294. [Google Scholar]
- 30.Arifovic J, Boitnott JF, Duffy J. Learning correlated equilibria: An evolutionary approach. J Econ Behav Organ. 2019;157:171–190. [Google Scholar]
- 31.Kim C, Wong K-c. Evolutionarily stable correlation. Korean Econ Rev. 2017;33:55–94. [Google Scholar]
- 32.Metzger LP. Evolution and correlated equilibrium. J Evol Econ. 2018;28:333–346. [Google Scholar]
- 33.Güth W. An evolutionary approach to explaining cooperative behavior by reciprocal incentives. Int J Game Theor. 1995;24:323–344. [Google Scholar]
- 34.Dekel E, Ely JC, Ylankaya O. Evolution of preferences. Rev Econ Stud. 2007;74:685–704. [Google Scholar]
- 35.Akçay E, Van Cleve J, Feldman MW, Roughgarden J. A theory for the evolution of other-regard integrating proximate and ultimate perspectives. Proc Natl Acad Sci USA. 2009;106:19061–19066. doi: 10.1073/pnas.0904357106. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Alger I, Weibull JW. Homo moralis—Preference evolution under incomplete information and assortative matching. Econometrica. 2013;81:2269–2302. [Google Scholar]
- 37.Bicchieri C. Norms in the Wild: How to Diagnose, Measure, and Change Social Norms. Oxford Univ Press; Oxford: 2016. [Google Scholar]
- 38.Hofbauer J, Karl S. Evolutionary Games and Population Dynamics. Cambridge Univ Press; Cambridge, UK: 1998. [Google Scholar]
- 39.Bicchieri C. Norms of cooperation. Ethics. 1990;100:838–861. [Google Scholar]
- 40.Smith JM. The theory of games and the evolution of animal conflicts. J Theor Biol. 1974;47:209–221. doi: 10.1016/0022-5193(74)90110-6. [DOI] [PubMed] [Google Scholar]
- 41.Nash J. Two-person cooperative games. Econometrica. 1953;21:128–140. [Google Scholar]
- 42.Kalai E, Smorodinsky M. Other solutions to nash’s bargaining problem. Econometrica. 1975;43:513–518. [Google Scholar]
- 43.Harsanyi JC. Rational Behaviour and Bargaining Equilibrium in Games and Social Situations. Cambridge Univ Press; Cambridge, UK: 1977. [Google Scholar]
- 44.Skyrms B. Stability and explanatory significance of some simple evolutionary models. Philos Sci. 2000;67:94–113. [Google Scholar]
- 45.Young HP. An evolutionary model of bargaining. J Econ Theor. 1993;59:145–168. [Google Scholar]
- 46.Rawls J. A Theory of Justice. Harvard Univ Press; Cambridge, MA: 1971. [Google Scholar]
- 47.Watts DJ, Strogatz SH. Collective dynamics of ‘small-world’networks. Nature. 1998;393:440–442. doi: 10.1038/30918. [DOI] [PubMed] [Google Scholar]
- 48.Pillutla MM, Chen X-P. Social norms and cooperation in social dilemmas: The effects of context and feedback. Organ Behav Hum Decis Process. 1999;78:81–103. doi: 10.1006/obhd.1999.2825. [DOI] [PubMed] [Google Scholar]
- 49.Smith JR, et al. Congruent or conflicted? The impact of injunctive and descriptive norms on environmental intentions. J Environ Psychol. 2012;32:353–361. [Google Scholar]
- 50.Van Lange PAM, Joireman J, Parks CD, Van Dijk E. The psychology of social dilemmas: A review. Organ Behav Hum Decis Process. 2013;120:125–141. [Google Scholar]
- 51.Sherif M. The Psychology of Social Norms. Harper; New York: 1936. [Google Scholar]
- 52.Valsiner J. Comparative Study of Human Cultural Development. Fundación Infancia y Aprendizaje; Madrid: 2001. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.




