How institutions shaped the last major evolutionary transition to large-scale human societies

Abstract

What drove the transition from small-scale human societies centred on kinship and personal exchange, to large-scale societies comprising cooperation and division of labour among untold numbers of unrelated individuals? We propose that the unique human capacity to negotiate institutional rules that coordinate social actions was a key driver of this transition. By creating institutions, humans have been able to move from the default ‘Hobbesian’ rules of the ‘game of life’, determined by physical/environmental constraints, into self-created rules of social organization where cooperation can be individually advantageous even in large groups of unrelated individuals. Examples include rules of food sharing in hunter–gatherers, rules for the usage of irrigation systems in agriculturalists, property rights and systems for sharing reputation between mediaeval traders. Successful institutions create rules of interaction that are self-enforcing, providing direct benefits both to individuals that follow them, and to individuals that sanction rule breakers. Forming institutions requires shared intentionality, language and other cognitive abilities largely absent in other primates. We explain how cooperative breeding likely selected for these abilities early in the Homo lineage. This allowed anatomically modern humans to create institutions that transformed the self-reliance of our primate ancestors into the division of labour of large-scale human social organization.

1. Introduction

Life on the Earth has undergone a series of major evolutionary transitions in which individuals at a lower level of biological organization came together to form higher level units [1]. Examples include replicating molecules coming together to form protocells, single-celled individuals evolving into multicellular organisms and solitary insects transitioning into eusocial colonies. The final transition proposed by Maynard Smith & Szathmáry [1] is the origin of human societies. Yet, while the other major evolutionary transitions are starting to become well understood [2,3], there is a lack of a cohesive theory that can explain the transition from primate social organization based on kinship and personal exchange to human societies with large-scale impersonal exchange and division of labour between unrelated individuals.

Human societies do indeed largely meet the criteria for a major evolutionary transition [3]. For example, just as epigenetic inheritance (a novel inheritance mechanism) allows the cells in a multicellular organism to differentiate and profit from a division of labour, so language (a novel cultural inheritance mechanism) allows human individuals to coordinate and specialize in different tasks, and so also to profit from a division of labour. Similarly, while by most measures, a multicellular organism is more complex than a single cell, so human chiefdoms are more complex than hunter–gatherer bands in terms of the number of hierarchical levels of organization [4]. And just as multicellular organisms with division of labour and sterile somatic cells gradually evolved from single-celled ancestors, so cultural phylogenies (based on language trees) point to states evolving gradually from chiefdoms, which in turn evolved gradually from hunter–gatherer macro-bands and tribes [4].

We propose to subdivide the major transition to large-scale human societies into four distinct, smaller transitions (figure 1). (i) The origin of the human hunter–gatherer niche, characterized by large but hard to acquire food packages, allomaternal care and egalitarian social structure. (ii) The origin of language, a novel unlimited inheritance system that strongly facilitates cumulative cultural evolution and negotiation between individuals. (iii) The Neolithic revolution, which involved the shift to agricultural and sedentary populations with hierarchical social organization. (iv) The origin of states, where interactions regularly occur between non-kin who may never meet again.

Figure 1.

Sub-transitions of the major transition from small- to large-scale human societies, with the major triggers for subsequent changes indicated, as well as crude estimates of the timing of these transitions. Solving the collective action problems inherent in large-scale agriculture may or may not have involved coercion depending on the society (e.g. over management of irrigation [5–7]). However, the surpluses provided by agriculture eventually led to hierarchal institutions that were prone to collapse and be reformed [8], culminating in the first states ca 4 kya.

We will assume that the first transition, from a largely vegetarian primate living in fission–fusion societies in woodland landscapes, to a savannah-living partly carnivorous cooperative hunter type of living, was made possible by changes in social organization not unlike those seen in other lineages that ended up adopting a combination of cooperative breeding and hunting (e.g. [9]). Our focus here, then, is on explaining the transitions in social organization subsequent to the emergence of language. Current estimates place the origin of modern-like language at either less than 100 kya or at around 500 kya, with the older date being the most plausible [10].

From an economic point of view, the major transition is from an initial state of autarky in which group members do not typically exchange resources with each other, to one of catallaxy where there is extreme division of labour and hence extreme interdependence between group members. In non-human primate social systems, each individual itself produces most of the resources and technology it needs to survive and reproduce. By contrast, while hunter–gatherer individuals can typically still each produce their own technology, they are reliant on the sharing of food with other individuals in order to survive. Finally, in large-scale human societies, individuals rely on trade with non-kin for nearly all of their vital resources, and an individual will not always itself possess the entire knowledge necessary to produce any single piece of technology.

Here, we develop the hypothesis that the human capacity to form institutions was a key driver of the transition to large-scale societies (and may indeed be necessary for their formation). Institutions (sensu [11]) are human-devised mechanisms for generating the rules of social interactions. Through communication and negotiation, humans can transform the rules of their ‘game of life’. The game of life depends on two kinds of constraints. The first kind consists of exogenous biotic and abiotic factors that cannot be changed by individuals at the time they are interacting [11,12]. These factors include the laws of physics and the current environment which comprises, for example, the current total resource endowment and the individuals' state of technology. The second type of constraints is behavioural in nature and so can potentially be modified by the individuals themselves [11,12]. This includes restraining or expanding behavioural options. By creating institutions, individuals can change the rules aspect of their social interactions, thereby increasing some possibilities without foreclosing others, and potentially tipping the balance from a situation where defection is individually advantageous into one where it pays to cooperate [12–16]. As we shall discuss below, theoretical work in economics has formally demonstrated conditions under which this can occur even in arbitrarily large groups of unrelated individuals where participants meet very infrequently [14–16].

Humans can create these institutions because they possess various cognitive features that are lacking in other primates and that are necessary to devise and enforce institutional rules. These include shared intentionality, strong inhibitory control and a willingness to seek out mutual opportunities. We explain below how these skills evolved as a result of the adoption of cooperative breeding early in the Homo lineage. Once in place, they could then be co-opted for institution formation.

In §2, we first define the term institution more precisely, before delineating their costs and benefits, and discussing the cognitive prerequisites necessary for their evolution. In §3 we then discuss how the institutional-path hypothesis can explain the key steps of social evolution from hunter–gatherers with language to large-scale states.

2. Institutions

(a). What is an institution?

In general, the outcome(s) of an individual's behaviour, in terms of its fitness consequences and/or material rewards, depends upon the behaviour of other individuals as well as on exogenous biotic and abiotic factors. In game theory [17,18], a game form defines the behavioural options—the ‘strategies’—available to each individual, and the relationship between strategies and outcomes. The game form thus specifies the rules of social interactions or, in other words, the ‘rules of the game’, which are usually and casually referred to in evolutionary biology as a game. More particularly, in game theory, a game consists of a game form and the preferences of individuals over alternative outcomes, and thus allows the equilibrium strategies to be determined [17,18]. In evolutionary biology, strategies are often (but not always) assumed to be genetically or culturally inherited, in which case it is directly the evolutionary process and not the preferences that determine equilibrium strategies.

When individuals can communicate with each other, and when the strategies consist of messages, a game form is often called a mechanism [11]. We follow Hurwicz [11] in considering that an institution is a mechanism whose outcome is a game form. The hallmark of an institution is a sequence of at least two sets of social interactions:

• (i) Active genesis of institutional rules through communication and bargaining by the individuals in a group (or subset thereof).

• (ii) Economic interactions whose outcomes are material, and which are affected by the institutional rules.

An institution thus consists of a political game form, which determines the rules of the subsequent economic interactions (figure 2). These two types of interaction are likely to take place on very different time-scales. In particular, the political game form is likely to be played much less frequently than the economic game form. For example, the economic game form will likely be played many times in a single generation, while the institutional rules may only change once every several generations. The rules of the political game can also themselves be set by rules generated by another game form [11], referred to as a ‘constitutional’ game form by Ostrom [19, p. 59]. This constitutional game form will, in turn, be played even less frequently. Finally, the rules of the constitutional game form will themselves be set by a ‘meta-constitutional’ game form, but this series of rule-generating game forms eventually begins with the unchangeable rules of the biophysical world and terminates with the economic game form that generates material pay-offs [19]. Because our focus is on the distinction between generating rules versus playing the economic game form, for simplicity we consider only one political game form and one economic game form.

Figure 2.

An institution is a mechanism of communication whose output is the rules for economic interactions. As such, one is in the presence of a political game form, where the preferences of group members for institutional rules are expressed, followed by an economic game form. The political game form could give equal weight to the preferences of all group members, as in egalitarian institutions, or could give more weight to dominant individuals, as happened with the origin of agriculture. The result of the political game form is the rules (or game form) for the subsequent economic interactions. The economic interactions determine the fitness or material rewards to individuals and may, for example, be a variant of a public goods or coordination game, or an exchange economy where goods are traded.

A more all-encompassing and formal definition of institutions than that given above exists (most notably the one by Hurwicz [11, p. 128]), but for our purposes it is enough to see an institution as a mechanism involving communication whose outcomes are rules of interactions. Non-linguistic animals are probably unable to produce institutions involving many individuals, even though they play economic games, because they are unlikely to autonomously generate institutional rules through communication.

The assemblies in modern hunter–gatherers that discuss resource allocation rules or what would be adequate norms of behaviour provide a good example of an institution as defined above. We stress that the institution comprises the negotiation process as well as the resulting norms or rules of behaviour [11, p. 128]. This is in contrast to the cultural evolution literature, which equates institutions with equilibrium norms of behaviour in an economic game form [20], rather than with a political game form that generates rules for the economic game form.

(b). The benefits of institutions

The formation of institutional rules can transform the ‘Hobbesian’ rules (or default rules) of the game of life into different rules that lead to more cooperative outcomes, but why is this? Since interactions are localized, it is important to realize that social life in hominins largely consists of a repetition of interactions that involve coordination or cooperation problems. For repeated interactions, the fundamental folk theorem of game theory [17,18,21] tells us that cooperation can ultimately be sustained in an equilibrium by conditional strategies that respond to players' past actions (reciprocity).

Specifically, in an indefinitely long sequence of interactions where individuals value future pay-offs and cannot completely hide their actions, any strategy that guarantees a pay-off at least as great as the minimax pay-off in the underlying stage game can be an equilibrium [17,18,21]. The minimax pay-off is the largest pay-off that an individual can receive if its opponent tries to minimize the individual's pay-off—in the Prisoner's Dilemma, it would be the pay-off received when the opponent defects. Therefore, if any individual deviates from the equilibrium strategy, then its pay-off can be reduced to the minimax pay-off by its co-players. Consequently, it does not pay an individual to deviate from a strategy that gives more than its minimax pay-off. This logic applies even to groups of infinitely large size where an individual does not interact twice with the same partner, provided that there is a way to transmit sufficient information about the past behaviour of partners [14,22], i.e. reputation. It also applies to interactions where N-players interact simultaneously [23], such as repeated collective action problems.

There are potentially three kinds of issues that can limit the application of the folk theorem to sustain equilibria with high individual material payoffs. Each of these can be addressed by institutions. The first potential problem stems from the fact that there are infinitely many equilibria with some level of cooperation [21,24]. But many of these equilibria will give pay-offs that are hardly any better than the minimax pay-off, while others will result in much greater pay-offs. If individuals act independently, then they have no means to guarantee that they will coordinate on an equilibrium that gives high individual pay-offs, and are likely to settle on the ‘default equilibrium’ determined by the default Hobbesian rules of interactions (figure 3). Institutions can resolve this problem, because they provide a means for individuals to amalgamate dispersed information about resources and wants, and hence coordinate their actions to reach an equilibrium that gives higher pay-offs than the default equilibrium (figure 3). By devising rules of interactions individuals settle on an equilibrium, transforming the social contract (in the sense of [21,24]) from one that gives only the pay-off of the Hobbesian equilibrium, to one where the benefits of cooperation are achieved.

Figure 3.

In the absence of institutions, individuals that engage in repeated social interactions are likely to receive only the pay-off corresponding to the default or Hobbesian equilibrium of the game of life (see also [12]). However, when group members can communicate and negotiate an agreed coordination mechanism (i.e. create an institution), they can coordinate in the economic game form on an equilibrium that potentially increases their mutual pay-off relative to the Hobbesian equilibrium (shaded area, see also [21,24]). The resulting equilibrium strategies are self-enforcing, in the sense that it is both individually advantageous to follow them when others are doing so, and individually advantageous to sanction group members that do not follow them. Although this figure shows a group of two individuals for illustration, the size of the shaded area and thus the benefit of having institutions actually increases with the number of interacting individuals. This is because the problems of coordinating on an equilibrium without institutions increase with group size [25]. (Online version in colour.)

The second issue is that individuals need to value future pay-offs, and the game needs to be indefinitely repeated. Institutional rules can help to make these conditions hold. For example, Casari [26] describes the development of institutional rules to govern the use of common agricultural land in the Italian Alps, between AD 1200 and 1800. The rules which most villages ended up adopting tied families and their future descendants into the group, by requiring that the sale or purchase of rights to use the communal land was subject to a majority vote among the other villagers. This ensured that individuals would then care about their future pay-offs and that there was no simple way to end the game.

The third issue is that individuals need to have sufficient information about the past behaviour of other individuals, a problem which becomes all the more pressing as group size increases. Institutional rules can help to alleviate these problems by facilitating the spread of information between group members. For example, extant groups managing common-pool resources from irrigation systems to shared grazing lands make agreements to appoint individuals to act as monitors, and regularly hold assemblies of all group members to share information [13,26]. Institutional rules that resolve social dilemmas also typically create centralized repositories for storing information about the reputation of group members, which was common for merchants in mediaeval Europe [14,16]. The right institutional rules, then, can create an environment in which the Folk Theorem can sustain equilibria that give high individual material pay-offs [27].

Institutional rules are typically not imposed externally, but are the result of a political game form. Experiments have repeatedly found that individuals placed in social dilemmas and allowed to communicate achieve better outcomes than if they are not allowed to communicate [28]. Those using communication both to agree on a joint investment strategy and to choose their own sanctioning system achieve results close to the group's optimum ([28], see also [29]). Field studies have illustrated how institutional rules, designed by resource users themselves, allow for the self-organized management of irrigation [5,30,31], fishing and harvesting systems [13]. For example, in the Spanish huerta irrigation systems, institutional rules specify how much water each user may take at a given time, how responsibilities for maintenance of the system are shared and what the sanctions are for individuals who break the rules. These rules are not imposed externally but are created by assemblies of the irrigators themselves, and indeed have been for a thousand years [13].

Critically, and contrary to cultural group selection arguments (e.g. [32–34]), institutional rules in these situations create an economic game form in which monitoring and sanctioning are not altruistic (sensu evolutionary biology [35]). Rather, field studies have demonstrated that successful institutional rules create conditions that provide direct benefits (sensu evolutionary biology [35]) to individuals that actively monitor and enforce them [13,14,16,27,36]. As such, in contrast to altruistic punishment [33,37], they do not require high genetic or cultural relatedness between group members.

For example, Ostrom describes how extant small-scale societies incentivise group members to monitor each other, by allowing individuals that discover a cheater to keep a proportion of the fine levied on that cheater [13]. And as an example in larger scale societies, in mediaeval Europe the Law Merchant system of institutional rules was developed, where individuals could pay a cost to register non-cooperative acts by their partner with a judge. They could also pay a cost to query the system to see whether their trading partner had any disputes registered against them before transacting [14,16]. Judges could impose a fine on cheaters, but had no means to force individuals to pay this fine. Nevertheless, if a fine was imposed then it was in the trader's own interests to pay it in order to maintain a good reputation with the Law Merchant, and so be able to reap the benefits of cooperation with other individuals in future. Consequently, this system of sanctioning was self-enforcing, even though traders could not be compelled to pay a fine, and had to finance the Law Merchant system themselves [14].

(c). The costs of institutions

Creating self-enforcing institutional rules is a costly process. First, there are costs to setting up a self-enforcing system of monitoring and sanctioning, such as paying judges in the Law Merchant system. Second, time and energy must be spent on negotiating the rules. While this can be done in face-to-face discussions after sunset in hunter–gatherer groups, negotiation becomes much more costly as group size increases. Indeed, it cannot be overstated how difficult it is to agree on something in a group. Arrow's impossibility theorem [38] says that there is no satisfactory way of making social decisions once individuals have sufficiently different preferences. As a result, institutional arrangements that need a high level of consensus between group members may be inherently unstable whenever individuals' endowments and allegiances shift over time, or when there is a turnover of players. Finally, some individuals may exert disproportional influence in the political game form, driving the creation of rules that favour themselves at the expense of others. The cooperation and coordination achieved under the institutional rules needs to provide sufficient benefits to offset all of these costs, and thus improve on the pay-off from the Hobbesian equilibrium (figure 3). Nevertheless, the fact that we see cooperation-promoting institutions in the real world implies that this condition can in principle be met.

(d). The uniqueness of institutions in humans

We emphasize here the uniquely human genesis of institutional rules: the explicit and coordinated construction of group-wide rules that regulate social interactions and that are enforced by other group members. This contrasts in a fundamental way to the usual mechanisms for social interactions considered in evolutionary biology. Other organisms can indeed condition their behaviour on the actions of other individuals (e.g. reciprocity), and they can modify their environment over time (niche construction, [39]). Other animals also perform social learning, imitating traditions of other group members such as bird songs or techniques to open nuts. But crucially, we are aware of no other species that over one individual's lifetime can construct arbitrary rules to regulate social activity, and then enforce these rules by coordinated sanctioning (see also [40] for a similar argument about the uniqueness of human culture).

For example, consider the institutional rules of marriage, the details of which are particular to any one society. At first sight, the reproductive strategy of monogamy adopted by many animals may seem to be the same. But this is not so, because the institutional rules of marriage regulate what counts as marriage, what the necessary preconditions for it to occur are (e.g. the payment of dowries), who may marry whom, how a marriage may be terminated, etc. These rules are necessarily recognized and followed by many individuals, and violations are enforced by coordinated sanctioning. In other words, they define what is normative, and they change the economic game form by changing the mapping between individual strategies and the corresponding outcomes, i.e. the pay-off matrix. By contrast, monogamy in the animal world is simply an individual unilateral reproductive strategy that is not regulated by rules and enforced by societal sanctioning, and so which does not change the economic game form.

(e). The cognitive requirements of institutions

It is difficult to see how individuals could play the political game form without certain cognitive faculties that are unique to humans. Institutions involve individuals bargaining over rules to structure their social interactions. This means that they first need to be able to foresee alternative social contracts, and then communicate and negotiate over them in order to improve over the default Hobbesian rules. This requires at least three types of advanced cognitive features. (i) To devise alternative rules of interactions, individuals need to be able to create virtual worlds. This requires planning, imagination, causal understanding, large working memory and the ability to anticipate future rewards. (ii) To communicate and bargain efficiently over their rules of interactions, individuals need language and a motivation to seek out information and knowledge, have shared intentionality, and a fully developed theory of mind. (iii) To reach consensus, individuals need a strong willingness to seek out mutual opportunities, as well as strong inhibitory control.

These abilities are only partially present in other primates. Why is this? After all, other primates have large brains [41] and relatively well-developed cognitive faculties. The answer is that many of the traits require at least some degree of prosocial motivations. Prosocial motivations are lacking in extant great apes, from which we can infer that they were also lacking in the common ancestor of the great apes and Homo. In the next section, we present a hypothesis for why prosocial motivations evolved in our lineage, and hence why our lineage evolved a social cognition that could later be co-opted for the formation of institutions.

3. From primate autarky to human catallaxy

(a). The hunter–gatherer niche before language and institutions

Elements of the hunter–gatherer, or forager, niche were gradually assembled over the past 2 Myr or so, but details necessarily remain sketchy. Because language must have affected this niche (see §3b), we will try to reconstruct what it looked like before language arose, based on comparative evidence and on the archaeological and fossil records.

It is known that by the time good documentation of Homo erectus is found, at ca 1.8 Ma, the basic elements of hunting and gathering were in place [42]. We can infer aspects of the social system [43], including bonding among males (collective defence against large carnivores and subsequent collective acquisition of meat) and male–female friendships (as found in primates in very large groups, e.g. [44]). Large meat packages inevitably meant wider sharing, including with females and immatures. The latter would have increasing difficulty supporting themselves, given the increasing reliance on technology or endurance running, and thus probably required energy inputs from others. Finally, their large brain size, well above the so-called grey ceiling for hominoids [45], suggests energy inputs for reproducing females. In other words, H. erectus showed many elements of extensive allomaternal care [9], i.e. cooperative breeding.

Comparative studies show that cooperative breeding changes the psychology of primates, and indeed other mammals such as elephants and African wild dogs, when compared to their non-cooperatively breeding sister taxa [46]. These studies imply that cooperative breeding selects for a high social tolerance and prosocial motivations, leading to a marked increase in socio-cognitive abilities [46,47]. What is unique in Homo is that cooperative breeding and the consequent prosocial psychology were added on top of an already existing large-brained ape-like cognitive system, inherited from our earlier hominin (australopithecine) ancestors. This created the potential for a more advanced social cognition than that seen in other cooperatively breeding species. One especially pertinent feature of an ape cognitive system is an ability to understand mental states in other individuals. In great apes, this ability seems to only be used in competitive contexts [46]. But when prosocial motivations co-evolved with cooperative breeding in the Homo lineage, this existing ability to grasp mental states could start to be used in cooperative contexts. Ultimately, this culminated in the evolution of shared intentionality [46], i.e. the sharing of psychological states between individuals. Shared intentionality, in turn, underlies many of the other cognitive prerequisites for institution formation, including the use of language to share information [48].

(b). Hunter–gatherers after the advent of language

The origin of language is a complicated and well-studied area, which we do not address directly here. However, we note that once it evolved, language had two key consequences for hunter–gatherer social evolution. First, language made teaching more effective, which provided more scope for cumulative cultural evolution and hence the development of new technologies [49,50]. Second, once in place, language enabled individuals to negotiate their rules of social interactions; that is, to start to create institutions for the first time. These two elements came together to produce greater cooperative division of labour among post-language hunter–gatherers.

With the advent of new complex technologies, such as poison-tipped arrows, nets and traps, it became possible for individuals to hunt large game in much smaller groups than before. Because hunting large game is inherently unpredictable, having multiple hunting parties within a social unit would provide benefits to wider food sharing as an insurance scheme. Even the best hunters benefit from sharing because this reduces the variance in their own and their offspring's daily calorific intake [51]. The improvements in safety and hunting ability led to the break-up of the old group into smaller subgroups, now called bands or camps. But crucially the reputational effects of language allowed bonds to remain strong among camps of the same community, as shown by frequent moves between them. And increasing returns to scale would provide an advantage to sharing with a larger number of individuals, for example by allowing individuals to overproduce food items that they found easy to obtain and exchanging them for other items. Thus, fewer hunters per party supported by a sharing system could massively reduce variance in an individual's yield [52]. Language made it possible for the sharing networks to become larger and therefore more stable.

Anthropological studies have shown that individuals in extant hunter–gatherer groups consistently devise and use institutional rules to regulate this kind of food sharing. Indeed, extant hunter–gatherers spend much of their time discussing rules of sharing and gossiping about violations of these [51]. In other words, they negotiate institutional rules and enforce these themselves. Examples includes rules that specify who receives what part of a kill and what quantity [40,53], with defectors being punished by a variety of means from public ridicule through to ostracism and execution [54]. While some other primates do practise some degree of food sharing, they do not have non-dyadic systems of food sharing which are regulated and enforced by impartial rules that apply to everyone in the group [52]. This suggests that institutional rules are necessary to regulate such systems of sharing [51], and hence that the supporting institutional rules co-evolved with extended food sharing in hunter–gatherers after the advent of language.

Hill [40] gives many more examples of institutional rules in extant hunter–gatherer societies. These include rules concerning access to mating partners within the groups (prohibitions and prescriptions on the basis of age, kin or ritual membership), polygyny (degree allowed and who may practice it), regulation of violent conflict within and between groups, and rules regulating political power (rules of turn-taking in discussions, and rules governing who will be the leader for different social activities). Institutional rules also affect life history, by specifying who must give resources to juveniles.

During the Upper Palaeolithic, we also see the emergence of long-distance trade and division of labour beyond food sharing. Evidence for this includes the remains of materials that had been transported hundreds of kilometres from their origins, and the development of new tools that were specialized in performing specific tasks [55]. Trade would have been strongly favoured by the presence of institutions, because already back then it required a mechanism by which individuals could make faithful promises to invest in labour that would only become useful when the finished product was exchanged. While doing this, the individual would produce less food, which would necessitate the pre-existence of rules for food sharing. Finally, the efficiency of long-distance trade would have been greatly enhanced by an institution for using some type of (pre-numismatic) money. The existence of art and other forms of symbolism suggest that Upper Palaeolithic humans had the cognitive abilities to do this [55].

An important question concerns how institutional rules were formed in Palaeolithic hunter–gatherers. In other words, what did the political game form look like? Modern hunter–gatherer groups spend much time discussing institutional rules and violations of these around the camp fire after sunset [54]. Furthermore, observations of these groups show egalitarian political interactions. Individuals typically take turns to give their opinions during group discussions [54]. The role of leaders seems to be to help the group to reach a consensus, rather than to force their own opinion upon others, or to benefit materially. Moreover, ethnographic evidence suggests [54], and archaeological evidence confirms [56], that status was not hereditary in these ancient mobile hunter–gatherers. This egalitarianism likely evolved and was maintained by a combination of high degrees of social interdependence in obtaining and defending resources, and the creation of lethal weapons that reduced the effects of physical differences in strength between would-be dominants and subordinates [57]. Moreover, it would be difficult for any one individual to monopolize meat from large game. Thus, although each individual should be expected to try to craft institutional rules that benefit itself, the egalitarian social structure would have prevented any one individual from being able to benefit itself too much at the expense of the rest of the group. Consequently, the political game form was likely to take the shape of a mechanism that aggregated the preferences of all group members [58] without resulting in too much dissent.

(c). The origin of agriculture, large social groups and hierarchy

The origin of agriculture was likely made possible by many factors [59], including the stabilization of the Earth's climate during the Holocene. However, successful agriculture would have necessitated the expansion of the domain of regulation by group institutions. It would require new property rights, to ensure that an individual could not simply have his plants, animals, land or stored food taken by others [60]. Agriculture would also require institutional rules to prevent the overexploitation of land and other common-pool resources [13,26]. Groups would also face new social dilemmas brought about by new, shared technology, such as the construction and usage of irrigation systems [6]. The existence of institutions therefore placed humans in a unique position to benefit from agriculture.

It is plausible that institutions aimed at solving these problems co-evolved with the demographic expansion of human groups brought about by agriculture. If cooperation problems were solved, then larger group sizes could potentially benefit individuals through both economies of scale (increasing returns in material pay-off as a function of population size; [61]) and economies of scope (increasing returns in material pay-off due to variety, not size; [62]). The logic of this has been demonstrated in a formal model of the coevolution of demography with institutions to regulate irrigation [63]. The results were that groups with institutional rules that successfully solved collective action problems grew to a larger size, and spread their institutional rules to other groups through excess migration.

However, as humans shifted to intensive modes of subsistence, the political game form itself started to change [64]. With the advent of storage technologies, it became possible for some individuals to start to build up a surplus of resources and form patrilineal clans for their defence. Permanent agriculture, especially irrigation systems, would have tied individuals to their group, making it hard to escape a despotic leader. The result was that agriculture triggered a shift from egalitarianism back to despotic social structure. Despotic leaders that commanded surpluses of resources would then be able to influence institutions for their own good at the expense of other group members, in a way that they could not have done previously in an egalitarian structure. For example, leaders could dominate the political game form and skew the economic rules in their favour by enforcing (with coalitional support) what proportion of surplus resources from irrigation farming they could keep for themselves rather than share with the rest of the group [65,66]. Consequently, the shift to intensive food production heralded a transition to coercive and non-egalitarian institutions, or so-called extractive institutions [67].

(d). The origin of states and large-scale markets

Agriculture ultimately led to the emergence of multiple levels of hierarchical organization—states. In a state, the individuals just below the leader in the hierarchy each specialize in just a subset of the tasks of the ruler [68]. The creation of specialized authority roles represents a new institution, i.e. a new political game form that determines the rules of the economic interactions of commoners.

The archaeological evidence shows that the first states arose by the aggregation of previously independent groups, rather than by one group simply expanding in size and displacing its neighbours [69]. At least two types of between-group interactions are implicated in driving this aggregation: warfare and trade [69]. The role of warfare is quite intuitive: aggregation can happen by one polity forcing another to become subordinate to it. The role of trade is often seen as creating ties between chiefs, through the procurement and exchange of prestige goods (e.g. [70]; see [71] for a review). However, there is also evidence that staple goods were traded over long distances during the Neolithic [72]. Indeed, institutional rules regulated trade during the Neolithic [71]. Trade would be most reliable, and have lower transaction costs [73], with those others that were playing by the same institutional rules. Institutions could therefore provide a pressure for groups to aggregate into a larger polity in order to reap the gains of economies of scale and scope from trade.

In modern states, division of labour is so pronounced that individuals are critically dependent on others outside their family and close friendship circles for food, as well as for protection from the myriads of hazards encountered during their lifespan. These vital elements for reproduction and survival are often supplied by decentralized competitive markets. This arrangement results in a ‘mutual dependence among strangers’ [74] where there is a remarkable level of trust among interacting individuals, which appears to be as uniquely human as language.

The central problem behind the functioning of any market, and more generally any large-scale society, is that no one has complete information [75]. The rewards of competitive exchange thus seem unachievable without institutional rules that guarantee at least secure property rights and that enforce the various structural characteristics of information flow in markets [21]. Furthermore, not all markets can be efficient because resource allocations made at the individual level often result in externalities, i.e. the effects of an individual's action can spill over into the environment and negatively affect other individuals, creating a tragedy of the commons situation [76]. This forces groups to design institutional rules that regulate any spillover on the environment, such as the mediaeval Law Merchant system that facilitated trade between strangers by spreading information about their past behaviour [14]. Indeed, the quality of institutional rules has been proposed as being the single main determinant of whether modern nations will succeed or fail. Acemoglu & Robinson [67] distinguish between institutional rules that are inclusive, meaning that they provide incentives to individuals that reward them for their productivity, in contrast to extractive rules that reward only a few individuals and that fail to adequately protect property rights.

4. Discussion

Institutions, the individually devised communication processes that produce rules to structure social interactions, are evolved (extended) phenotypes that fundamentally rely on language. They are key determinants of, and may be necessary to explain, the last major evolutionary transition.

In economics, institutions are often thought of as being formed by individuals with unbounded levels of cognition; that is, individuals following the neoclassical rational choice assumptions ([73], ch. 3). But preferences for forming institutions, and for institutional rules, can also evolve by processes of cultural evolution [63]. Consequently, institutions can be formed by individuals that have only bounded rationality (like probably any hominoid), as long as institution formation increases an individual's own pay-off, or their inclusive pay-off or fitness. Our hypothesis for the emergence of large-scale societies relies on self-created and self-enforcing institutional rules, which regardless of the exact level of rationality/cognition of the individuals involved, provide direct benefits from cooperation and coordination. Under self-enforcing institutional rules, cooperation, and monitoring and sanctioning are adaptive at the individual level.

There are at least three alternative hypotheses for the evolutionary origin of large-scale human societies. The first rests on individuals performing biased social learning, especially conformity-biased learning, whereby individuals tend to imitate the most frequent behaviours within their group. This creates high cultural relatedness within groups, and thus enables cultural group selection [77]. Although the conditions under which this really works remain unclear [78–80], a very low level of rationality is implicitly assumed, since individuals are assumed to be unable to compute when it would actually be advantageous to express shirking behaviours instead of to conform [81]. Consequently, in contrast to the institutional-path hypothesis, the biased-social-learning hypothesis entails that cooperation is often maladaptive at the individual level. Yet although experiments show that human infants develop a propensity for unconditional helping by the age of 2 years, by the age of 3 years they start to become influenced by the past behaviour of their partners [82]. In other words, as they develop, children do start to take account of the expected benefit when deciding whether to cooperate. Such individually beneficial cooperation is expected under the institutional-path hypothesis. Interestingly, this is the same age at which children start to normatively enforce the rules of artificial games in the laboratory [83].

Moreover, because the cultural evolution literature has essentially ignored the possibility of humans playing a political game form, it has implicitly assumed that the rules of the economic game form cannot be changed by a process operating within groups. As such, the cultural evolution literature has concluded that rules can only change by a slow and external process of between-group competition, rather than being variable in the short term through internal negotiation. But this conclusion does not fit well with the large brains and advanced planning and negotiation skills of our species.

The second alternative hypothesis presupposes the formation of coercive hierarchy, which results from strong asymmetries in physical strength or power within groups (figure 1). Coalitions of powerful individuals (elites) are able to coerce others when surpluses, as produced by agriculture, are large enough to be exploited. They may increase this coercion as groups expand in size through monopolizing resources gained through conflict with other groups [84,85]. This ultimately results in elites creating coercive states through conquest [86]. Under this hypothesis, individuals may still behave in their self-interest when coerced, but the social equilibrium will be far from one that gives a high pay-off to the majority of individuals.

The third alternative hypothesis (the ‘interdependence hypothesis’) is based on the idea that cooperation in early humans was mutualistic, with individuals becoming highly dependent on each other through the scavenging of the carcases of large game, which later extended into cooperative hunting [87]. This required the development of shared intentionality, and then other advanced socio-cognitive features such as language, in order to ensure successful coordination in high risk Stag–Hunt game situations. The high interdependence of individuals, combined with the possibility of partner choice, provided an incentive for individuals to use reputation when deciding whether to cooperate with an individual. However, this kind of cooperation was threatened as group size expanded, partly due to the problem of knowing the reputation of other group members. It is hypothesized that this problem lead to the adoption of group-wide norms and conventions, and symbolic markers as proxies for reputation [87].

There are clearly strong connections between all these hypotheses, and several elements of them are not mutually exclusive. Both the biased-cultural-transmission and the institutional-path hypotheses rely fundamentally on cultural evolution, and thus involve social learning. The main difference is the conception of rationality with which individuals are endowed. Under the institutional-path hypothesis, individuals are assumed to have high levels of cognition and rationality (see §2e), enough at least to respond adaptively to their social environment and reinforce individually beneficial actions under most circumstances. But it does not require conformity- or prestige-biased transmission at all. While conformity is surely important in humans and other primates, we also know that humans are flexible with their investment in cooperation depending upon the context [81,88,89] and that there is strong within-culture variation in the social learning strategies that individuals employ [90]. The institutional-path hypothesis better fits with these findings, by not requiring within-group homogeneity of behaviour or preferences.

Having institutions also does not exclude hierarchy and dominance. If hierarchical command is an efficient mode to solve economic problems as group size increases [25], then the voluntary creation of hierarchy and leadership is exactly what we expect to see in the long-run under the institutional-path hypothesis. The political game form can then subsequently change into one of dominance, where the new leaders take advantage of the costs of resisting or dispersing to create institutional rules that benefit themselves at the expense of others [65,66], paving the way to extractive institutions. However, coercive hierarchy seems to be inherently unstable [8] and costly to maintain, given the possibility for subordinates to form coalitions. Moreover, extant small-scale societies demonstrate that egalitarian institutions can resolve social dilemmas in irrigation and other agricultural problems, and often do so more effectively than coercive institutions [13].

Finally, both the institutional-path and interdependence hypotheses agree that human cooperation first emerged through direct and indirect individual benefits in small groups. But the interdependence hypothesis argues that the mechanisms supporting this must have broken down in large groups, leading to the use of conformity, group-wide norms and conventions, and symbolic markers as proxies for reputation. However, this hypothesis does not provide an account of how particular group-wide norms and conventions would be adopted. By contrast, under the institutional-path hypothesis, institutional rules continue to provide direct benefits to cooperating even in large groups and are created by a political game form.

We conclude that the key puzzle about large-scale human societies is not how to explain the existence of altruistic cooperation that is costly and fitness reducing over an individual's lifetime, as has been widely suggested [91]. Instead, the puzzle lies in understanding how the institutional rules that provide lifetime direct benefits to cooperation and coordination are generated and sustained over both short and long time-scales. From a theoretical perspective, there is a need for further modelling work on the evolution of institutional rules. From an empirical perspective, future work should investigate further how the cognitive prerequisites for creating institutions evolved, and what the exact level of rationality required is. It should also examine the role of the coevolution of trade and warfare with institutions, and the concomitant rise of large-scale societies.

Authors' contributions

S.T.P., C.P.v.S. and L.L. conceived the study. S.T.P. and L.L. conducted research. S.T.P., C.P.v.S. and L.L. drafted the manuscript.

Competing interests

We have no competing interests.

Funding

L.L. and S.T.P. were funded by Swiss NSF grant no. PP00P3-146340. C.P.v.S. was funded by Swiss NSF grant no. 310030B_160363/1.