Social complexity as a proximate and ultimate factor in communicative complexity

Abstract

The ‘social complexity hypothesis’ for communication posits that groups with complex social systems require more complex communicative systems to regulate interactions and relations among group members. Complex social systems, compared with simple social systems, are those in which individuals frequently interact in many different contexts with many different individuals, and often repeatedly interact with many of the same individuals in networks over time. Complex communicative systems, compared with simple communicative systems, are those that contain a large number of structurally and functionally distinct elements or possess a high amount of bits of information. Here, we describe some of the historical arguments that led to the social complexity hypothesis, and review evidence in support of the hypothesis. We discuss social complexity as a driver of communication and possible causal factor in human language origins. Finally, we discuss some of the key current limitations to the social complexity hypothesis—the lack of tests against alternative hypotheses for communicative complexity and evidence corroborating the hypothesis from modalities other than the vocal signalling channel.

1. Introduction

Imagine two congeneric species that face similar physical environmental selection pressures—for example, similar seasons and weather conditions, similar territory and food resource bases and similar parasites and predators. In the first species, the typical social unit is a female–male pair. The average individual in the first species spends the majority of its adult life interacting with a single individual of the opposite sex, with the occasional interaction with an individual or two at a territorial boundary, and with a part of the year devoted to rearing offspring. In the second species, the typical social unit is a larger grouping of several female–male pairs and one or two generations of the offspring of those different pairs. The average individual in the second species spends the majority of its adult life interacting with adults that are more dominant to it, with adults that are more subordinate to it, with younger generations of conspecifics, and with large numbers of individuals of other such groupings at territorial boundaries. In this scenario with these two species, we might imagine that in the second species compared with the first species, there is a greater need for individual discrimination based on communicative signals, or for conveying a broader diversity of messages about individual behavioural tendencies and environmental contexts, or for expressing a wider range of emotional and motivational states, or some combination of these three needs. Thus, because of the greater social complexity in the second species compared with the first, we would expect there to be a greater diversity of communicative signals in the second species—greater communicative complexity.

In this article, we argue that complexity in the type of social interactions that occur among animals selects for greater communicative complexity, in both a proximate and ultimate sense. This argument is the ‘social complexity hypothesis’ for communicative complexity. The central question that underlies this hypothesis is why a species that lives in larger or more complex social groups is unable to use the same communication system to manage its social world as one that lives in a less complex society. The social complexity hypothesis is not novel to us, and in the §2 we provide historical background to this argument. We end this section with some explicit predictions of the hypothesis. Next, in §3 we briefly discuss some of the empirical evidence in support of the hypothesis. In §4 we briefly discuss the social complexity hypothesis in the context of language origins in our own hominid ancestors. In §5 we describe some of the current limitations of testing the hypothesis—the relative paucity of data critically testing the hypothesis against alternatives, and of data testing the hypothesis in non-vocal communicative channels.

2. The social complexity hypothesis for communicative complexity

(a). Historical arguments

Over 200 years ago, Lamarck provided us with the earliest complete theory of evolution of species, and one of the earliest explicit arguments for how social complexity leads to communicative complexity. In a section of the Philosophie Zoologique [1], in which he described his argument for a key way in which our human lineage was distinct from that of non-human animals, namely language, he wrote:

‘The individuals … having largely increased their needs according as the societies which they formed became larger, had to multiply their ideas to an equivalent extent, and thus felt the need for communicating them to their fellows. We may imagine that this will have compelled them to increase and vary in the same degree the signs which they used for communicating these ideas … ’ [1, p. 172]

This section in the Philosophie Zoologique represents an early argument for the notion that certain communicative events might serve as referents for certain environmental contexts or stimuli, or certain internal states or processes, and that more complex social groups or networks may demand greater signalling abilities because of the greater need to transmit such information. This passage also represents an early argument about communication as a process of information transfer [2–4].

Among Darwin's many contributions that serve as the foundations for fields in the life sciences and social sciences, he made a similar argument to Lamarck with regard to the complexity of social interactions in The Expression of the Emotions in Man and Animals [5]:

‘With social animals, the power of intercommunication between the members of the same community,—and with other species, between the opposite sexes, as well as between the young and the old,—is of the highest importance to them. This is generally effected by means of the voice, but it is certain that gestures and expressions are to a certain extent mutually intelligible.’ [5, p. 60]

‘Naturalists have remarked, I believe with truth, that social animals, from habitually using their vocal organs as a means of intercommunication, use them on other occasions much more freely than other animals.’ [5, p. 84]

In this monumental work on emotion, expression and the principle of antithesis in communication, Darwin approaches our concern from the standpoint of a need for social species to convey and assess a wider range of emotional states in their signals, compared with less social or solitary species. This passage, therefore, represents an early argument for communication not as a transfer of information, but rather as a means of signalling about emotional or motivational state [6,7].

Students of animal communication realize that these two approaches to communication—one as a process of information transfer and one as a process of expression of emotion or arousal—represent very different views of what communication is and how we should go about studying, interpreting and describing it (e.g. recent commentaries in the studies [8–10]). Nonetheless, what is interesting about these two early passages from Lamarck and Darwin is that they both point to a similar end regarding socially complex species—individuals in those species should have a richer, more diverse system of communication.

Early twentieth century researchers from a range of fields have made similar arguments. In an early sociology text, Bernard [11, p. 299], for example, writes that, ‘The ever-expanding social environment calls constantly for new organizations of the old simple innate or acquired activity units into new character complexes to serve as adjustment mechanisms. These character complexes are not themselves inherited but they are built anew for each individual who needs them for adjustment purposes’. Tolman [12, p. 238], in a text that stands as a foundation for the cognitive psychology that would follow some four decades later, wrote, ‘The success of this speech-tool depends, of course, upon the degree of the speaker's means-end-capacities and his discriminanda- and manipulanda-capacities relative to the logical, rhetorical, and social means-ends-dimensions involved’. Translating the jargon of that field and time, his basic argument here is that the capabilities of a language/communication system are related to signal perception and production abilities of those communicating, and also to the social and environmental pressures impacting them. In his classic paper on the establishment and maintenance of dominance hierarchies in birds, the zoologist Schjelderup-Ebbe [13] discussed complications in the social life of birds stemming from sex, age, familiarity, ‘sympathy and antipathy’, and social position, and how these factors can influence the ways individuals physically interact with and communicate with one another. The comparative psychologist Carpenter [14] and the ethologist Thorpe [15], despite the methodological and theoretical differences of their respective fields (though not so much of their own research), drew similar connections between the complexity of social groups in animal species and the complexity of their systems of communication.

More recently, researchers in anthropology, biology and psychology have established and tested the social intelligence hypothesis (also called the Machiavellian intelligence hypothesis [16–19] and the social brain hypothesis [20,21]), which has relevance to our more general notion that sociality drives communicative complexity. The basic argument of the social intelligence hypothesis is that, whereas individuals of all species must solve difficult problems in their physical environments, individuals of heavily social species must solve additional—and potentially even more difficult—problems related specifically to the social worlds they inhabit. These individuals must balance ‘selfish’ individual needs with needs stemming from the other members of the group to which they belong. There is a need in these heavily social species for increased social cognitive processing ability. These abilities include recognizing and remembering past interactions with individuals, responding appropriately to third-party relationships within groups (e.g. an individual reacting differently to the relative of a recently aggressed ‘peer’ than it would to the relative of a non-aggressed ‘peer’), and assessing and managing the behaviour of group members in flexible and effective ways. The primary means of accomplishing this last point is through communicative interactions with group members [2,6,7,22,23]. Thus, one of the key planks to the general arguments of the social intelligence hypothesis is that there is likely to be a need for increased communicative complexity in heavily social species, compared with less social or more solitary species. Although communication represents a key component of the social intelligence hypothesis, we do note that there has been relatively little work that has directly assessed communication in light of the social intelligence hypothesis.

(b). Definitions

(i). Social complexity

We do not mean to be flippant, but ‘complexity’ is a term that is itself complex, and rarely explicitly defined in behavioural papers that discuss it. As such, we feel some definition of social complexity is needed (box 1). Complexity of signals is discussed in §2b(ii). Classically, and intuitively, sociality is synonymous with group living. Animals living in troops, pods, flocks, herds, etc. are those in which the consistency and regularity of interactions among the same set of individuals can lead to the formation of complex social dynamics (e.g. dominance hierarchies, alliances, different functional roles). We stress the notion of regularity of interactions among group members, as there are obvious differences between the stable structure of a baboon troop and an enormous aggregation of ungulates (the importance of ‘bondedness’ raised in Dunbar [29]). Whereas both a baboon troop and an aggregation of ungulates represent social groupings, and individuals in both can benefit from that sociality, the social structure and network of relations with the baboon troop make them far more socially complex than the much larger groupings of ungulates. Thus, there are social complexity distinctions that can be made between different social groups. What about individuals that do not occur in social groups? It would be wrong to say that all non-group living animals consequently lack social complexity. Consider a territorial animal in which the primary ‘social unit’ is the surrounding network of territorial neighbours. Not only must this animal keep track of those territorial neighbours (which is the basis of the dear enemy phenomenon: [30]), but this animal will also often interact with some or all of those individuals repeatedly on a daily basis. Consider also lekking species and those animals in which males spend considerable time and effort courting choosy females. Perhaps, the social dynamics of these animals are not as sophisticated as those occurring within a troop of chimpanzees, but territorial lizards or lekking grouse (for example) have a complex social structure in their own right. That is, definitions of social complexity are in essence dependent on the taxon in question. Furthermore, broad comparisons across major, disparate taxonomic groups (e.g. primates versus birds versus lizards) are arguably less informative for testing the social complexity hypothesis than comparisons among closely related species that vary in some aspect of their sociality (e.g. level of territoriality or type of mating system).

Box 1. Definitions of complexity, social complexity and communicative complexity.

Complexity

• Complexity ‘is the length of a concise description of the regularities of a system’ [24, p. 227]

• Complexity relates to ‘the phenomena which emerge from a collection of interacting objects’ [25, p. 3–4]

• Complexity ‘just means number of part types or degree of differentiation among parts’ [26, p. 7]

• Complexity ‘arises when the dependencies among the elements [of a system] become important’ [27, p. 9]

• Complexity ‘lies between order and randomness’ [and] ‘cannot be easily described, evolved, engineered, or predicted’ [28, p. 32]

Social complexity

Complex social systems are those in which individuals frequently interact in many different contexts with many different individuals, and often repeatedly interact with many of the same individuals over time.

Communicative complexity

Complex communicative systems are those that contain a large number of structurally and functionally distinct elements (e.g. large display repertoire sizes) or possess a high amount of bits of information.

We adopt a less exclusive definition of social complexity based on the frequency, nature and consistency of interactions among conspecifics. Our view is basically an extension of system theory [31] in which any system of elements can be described by the number of elements, the different types (‘species’) of element and the different relationships among the elements. Complexity in social systems, therefore, relates to the number of interacting individuals, the different types (social roles) of those individuals and the nature and diversity of interactions among those individuals. By this definition, simple social systems are those in which animals rarely interact, and when they do, interactions tend to be in one context (e.g. for reproduction) and almost never repeated with the same individual. Conversely, complex social systems are those in which individuals frequently interact in many different contexts (e.g. reproduction, aggression and foraging), and often with many of the same individuals. Two groups with an identical number of social roles and similar relationship structure may differ from one another in the number of individuals, with larger groups being more complex than smaller groups. Two groups with an identical number of individuals and similar relationship structure may differ from one another in the number of social roles, with groups having more social roles being more complex than groups with fewer social roles. Finally, two groups with an identical number of individuals and identical number of social roles may differ from one another in their relationship structure, with groups having more dyadic, triadic and higher level relations among individuals being more complex than groups having fewer such relations. We do note that comparing complexities of groups that differ in two or all three of these parameters is less straightforward—recent advances in social network analysis are providing the metrics needed to make such comparisons, however (see §2b(ii)).

Whereas some might find our definition of social complexity overly simplistic, we wish to highlight that with this definition, the social complexity hypothesis becomes relevant for understanding the evolution of communication in any animal that has cause to interact with a conspecific, regardless of the particular context of that interaction. It is obvious from the diversity of social signals that have evolved across the tree of life that the origin of social communication was not preceded by the formation of cohesive groups specifically, but originated in organisms that frequently interacted with members of their own species more generally.

(ii). Communicative complexity

There have been two typical ways in which researchers have tried to measure communicative complexity within a signalling system of a particular species. The first and most common is the number of distinct displays or signals in a signalling repertoire [32]. For example, males of some species of songbird have very large repertoires of different songs, whereas males of other species of songbird may sing only one song type. Certain mammalian species may have two or more acoustically distinct calls used in different contexts related to the detection of a predator, whereas other species may have only one ‘alarm’ type of call used when any type of potential threat is detected. Taking this approach, then, the argument is simply that larger repertoires of distinct signals or displays are more complex than smaller repertoires of distinct signals or displays. We note that it is important for repertoire sizes of organisms to be determined by studies both of signal production in different contexts and of responses to (experimental presentations of) signals. Simple description of variation within a repertoire of signals is an important first step in beginning to determine repertoire size. It is necessary to go beyond this, however, into empirical study and understanding of how the signals are used by signallers and how those signals in turn affect receiver behaviour (e.g. various theoretical approaches such as assessment–management analysis [7] and message-meaning analysis [33]), because this study and understanding may in turn shed light upon the nature and structure of the social group itself.

There is often graded variation among the signals of a repertoire. For example, two visual displays in a non-human primate species may be used by individuals in functionally different contexts and may be reliably distinguished by both human observers and conspecifics. However, the two displays may share certain facial muscles and underlying neurophysiological bases and, depending on affective or motivational state, may enable an individual signaller to move gradually from one display to the other. Similarly, reliably distinguished vocal signals with vocal repertoires of birds and mammals may nonetheless grade from one to the other. Furthermore, although signals are often described in typological ways, there is often considerable within-signal variation. Graded variation within and between signals can provide an additional dimension in communicative complexity [34–37].

The second major way researchers have tried to measure communicative complexity stems from information theory and the mathematical theory of communication [38]. Formally, complexity of a signalling channel is measured in terms of its entropy or uncertainty—bits of information. The greater the diversity of elements within a particular signalling system, the greater the uncertainty of any particular signalling event, the greater the reduction in uncertainty once the event is produced, and so the more potential information or complexity in the signalling system as a whole. Informally, the information theory approach is implicitly adopted whenever researchers classify complexity based on the number of different components or elements making up a given signal (e.g. number of notes in a given call [39] or amplitude variation in display movements [40]). Greater information suggests a greater number of distinct messages and meanings possible in the signalling system, but the question of messages and meanings is one that must be addressed with sensitive measures of signal production by senders in different contexts and by experiments on signal perception/recognition by receivers. For example, researchers might be able statistically to distinguish multiple elements within a signal, but this may not match up with what animals actually pay attention to or even discriminate in the signal [41].

For both of these typical methods of measuring communicative complexity, researchers testing the potential role of social complexity rarely address directly the actual ways in which variation in signalling by senders affects the behaviour of receivers. Being critical of one of our own studies, for example, Freeberg [42] experimentally demonstrated that group size could cause changes in vocal signalling complexity in Carolina chickadees (Poecile carolinensis; study described in more detail in §3a), but did not document how those changes in signalling affected the ways in which senders and receivers interacted with one another. The problem is particularly acute for large comparative studies that evaluate the causes of signal variation among closely-related taxa by using reports from the primary literature that summarize signal information or signal repertoire size for a given species [40,43]. In most cases, how variation in signalling by individuals in those species brings about changes in receiver behaviour is unknown. This is not to say that large comparative studies are without value; on the contrary, they can offer some of the strongest evidence in support of a hypothesis of how communication might have evolved. This is similarly true for experimental manipulations of social structure like the one mentioned above [42]. But there is clearly a need for both comparative and experimental studies to be balanced with some form of validation that variation in signal complexity is functional (elicits different responses from receivers). Thus, future research on links between social complexity on the one hand, and communicative complexity on the other hand, must emphasize and uncover the ways in which signal variation is used by signallers in their assessment and management of the behaviour of other individuals.

(c). Predictions of the social complexity hypothesis

Building from these earlier arguments and ideas, the social complexity hypothesis for communicative complexity makes a number of clear, testable predictions about the complexity of communicative systems in species or groups that vary in different aspects of social complexity (box 2). These predictions stand for both comparative and experimental approaches, for ultimate (phylogenetic) and proximate (developmental) questions, researchers might ask about communicative diversity in the species/populations/groups they study.

Box 2. Predictions of the social complexity hypothesis for communicative complexity.

In our discussion of these predictions below, we will use ‘units’ to refer to the primary level of analysis for the social groups or networks of individuals for a particular species or population. These units might be groups in the traditional sense, territory neighbours, a pool of potential mates, or any network in which animals frequently interact with one another. We additionally note that the following predictions are not exhaustive. Finally, these predictions assume that units are relatively stable (e.g. groups with the same individual members in space and for some time). Towards the end of this section, we consider units that are unstable—for example, groups in which individual membership is fluid and changing over time or space.

(i). Unit size

At a most basic level, taxa in which animals interact with a large pool of conspecifics will typically have greater social complexity than those taxa in which animals interact with fewer conspecifics, and so we would predict greater communicative complexity in taxa with larger units (groups, pods, flocks, territorial networks, etc.). This is because as the number of individuals in a network increases to n, the number of different possible non-directional individual–individual relations within the network increases to [n(n−1)/2]. A network of two individuals has one possible relation (A–B), a network of four individuals has six possible relations (A–B, A–C, A–D, B–C, B–D, C–D), and a network of eight individuals has 28 possible relations. For directional relations, in which a relation between individual X and individual Y is treated as distinct from a relation between individual Y and individual X, the same argument holds, with the number of possible relations being n(n−1).

At some point, group size must become too large for individuals to be able to recognize or remember every other individual in the group, let alone to keep a running update of previous interactions with every other individual in the group. Given the costs of social cognition [44,45], there must be an upper limit to the number of individuals that can be remembered for each species or population. For example, geladas, Theropithecus gelada, occur in enormous groups that are often several times larger than groups of closely-related chacma baboons, Papio ursinus. A playback study to test for vocal recognition of individuals in geladas found that male geladas could recognize far fewer individuals than the network of individuals with whom they regularly came into contact [46]. This supports the notion of costs of social cognition imposing a limitation on communicative complexity in very large groups. Bergman [46] suggested that the core ‘group’ in geladas is likely orders of magnitude smaller than the aggregations in which they are found—a ‘one male unit’ of a leader male, small number of follower males, and females and their offspring, compared with the several hundred individuals that might comprise an aggregation. Simple unit size, therefore, may have limitations as a measure of social complexity if the units being compared are quite large or vary in how stable or fluid they are in member structure. Simple unit size would seem to provide a stronger measure of social complexity when the units being compared are stable and smaller (e.g. species X average group size of eight individuals compared with species Y average group size of 24 individuals). In part, this reflects the difficulty of equating types of groups across species: for species like baboons that live in discrete, stable social groups, the ‘group’ is usually obvious; but for species like the gelada that live in multi-level social systems with fission–fusion dynamics, it is not always obvious what the equivalent grouping level actually is [47].

(ii). Unit density

Social units that occur at high densities result in individuals that interact with one another at higher rates, simply owing to closer spatial proximities. Given greater rates of interaction at high densities, we would expect to see a greater diversity of interactions among individuals, and so would predict greater communicative complexity for such individuals, compared with individuals in low-density units. This unit density argument is one of the ways in which two units with an equal number of individuals might nonetheless have different social and communicative complexity.

(iii). Unit member roles

Units in which a greater number of distinct social roles exist represent greater social complexity than units with very few distinct social roles. For example, one group might contain post-reproductive females and males, reproductive females and males, reproductively mature but non-breeding young adults, sets of offspring from a previous breeding season, and sets of current offspring, whereas another group might contain reproductive females and males and their sets of current offspring. The former group would represent more of an ‘information centre’ and more of a unit of collective, adaptive behaviour because of its diversity of social roles, compared with the latter group. As such, we would predict greater complexity of communicative signals in units with more distinct social roles compared with those with fewer distinct social roles. This unit member roles argument is another of the ways in which two units with an equal number of individuals might nonetheless have different social and communicative complexity.

(iv). Unit egalitarian structure

In many species with relatively stable multi-individual units, there exist relatively stable dominance hierarchies among members. Dominance hierarchies that are strongly linear are hierarchies in which the most dominant individual supplants (as one behavioural example) all other members and is supplanted by none, the second-most dominant individual supplants all other members except the most dominant individual and is supplanted by none except the most dominant individual, and so on to the bottom of the hierarchy and the least dominant, most subordinate individual. In such steep, linear hierarchies, the extent of possible relationships within the unit—especially in the form of directional relationships—is often severely limited by the hierarchy. In more egalitarian societies that lack highly linear hierarchies (‘mild despotism’ in Schjelderup-Ebbe [13]), there are by definition more reversals of agonistic interactions, such that the most dominant individual supplants (continuing the behavioural example) all the other individuals more than they supplant it, but many of them on occasion supplant it, and so on down the hierarchy to the least dominant, most subordinate individual. In addition to measures of agonistic behaviour and dominance hierarchies, measures of affiliative or approaching and accommodating interactions would prove useful to assessing how egalitarian or despotic a group might be. Network theory offers a battery of different indices of social structures in groups, which should provide sensitive metrics of social complexity and the diversity of directional relations within groups [48–52]. Because of the greater diversity of directional relations, more egalitarian networks should therefore possess greater communicative complexity than less egalitarian, more ‘despotic’ groups. This egalitarian structure argument is yet another of the ways in which two units with an equal number of individuals might nonetheless have different social and communicative complexity.

(v). Unit home-range size

When groups or individuals have relatively small home ranges, because of the greater density of individuals in the surrounding area, they will likely interact with other (extra-unit) individuals at higher rates [53]. If home ranges are stable not just in space but also for long periods of time, animals will likely interact with the same individuals at home-range boundaries for extended periods of time. In more widely dispersed taxa with larger home ranges, rates of extra-unit social interactions will be lower. As such, we would expect a wider and more complex social network for taxa with smaller home ranges, and so would predict greater communicative complexity compared with taxa with larger home ranges, assuming similar general movement rates.

(vi). Stable and unstable units

It is difficult to generate a straightforward prediction regarding units that differ in stability of members. As described above with regard to unit sizes that might be superficially enormous (e.g. hundreds of geladas in an aggregation, or ‘herd’) but whose core units might be considerably smaller (e.g. tens of geladas in one-male units), units that are highly fluid in terms of membership are difficult to characterize in terms of simpler metrics such as unit size or unit density. Individuals of some species might always be found in groups, but for any given individual, the network of interacting individuals might change completely over time. On the other hand, in certain species with fluid social structure, the social network of an individual can be measurable and finite. For example, great tits, Parus major, are a species of parid songbird that exhibit ‘basic flock’ structure—individuals occur in flocks that change membership over time and that intermingle with individuals of other flocks in space. Flock sizes in great tits have been reported to be as large at 20–50 individuals [54,55] or to be made up of rarely more than two individuals [54,56]. Perhaps, the differing methodologies, definitions of flocks and time-scales of these studies, are the reason for the different estimates of group size. In any event, we believe that in species with unstable or even semi-stable (such as great tits) group structures, researchers should place much greater emphasis on social network measurements such as estimates of network size, diversity and egalitarian structure discussed above. These measures will likely provide stronger metrics for comparisons of social complexity of different species or populations than would simpler metrics like group size.

3. A brief review of the evidence that social complexity affects communicative complexity

Here, we briefly review the work to date on potential links between social complexity and communicative complexity in animal species. Much of this work has taken comparative approaches involving two or more species, with some adopting phylogenetic comparative methods [40] to test the role of social complexity in communicative complexity within an explicit evolutionary framework. More recent studies are increasingly exploiting emerging behavioural datasets covering multiple species to perform phylogenetic comparative analyses that test the extent to which social complexity underlies the evolution of communicative complexity across species. The comparative method is a powerful tool for testing the social complexity hypothesis, and we hope that more researchers will exploit such approaches and the increasing availability of large behavioural datasets in the future. Another avenue that may prove fruitful is a developmental approach that will be especially informative for determining the plasticity of communicative complexity within species as a function of social environment. Do young individuals born or hatched into more socially complex groups end up developing greater complexity in their communicative systems compared with individuals born or hatched into less complex groups? This proximate approach may offer a powerful way to test the social complexity hypothesis, as the approach may offer experimental testing in some species. If researchers can manipulate the social complexity of groups experimentally, they can directly test whether social complexity is a causal factor in the communicative complexity of individuals.

We begin with a discussion of the most heavily studied signalling channel for the social complexity hypothesis—vocal signalling. We then discuss some of the work that has been carried out on another channel—visual signalling. Studies on other signal modalities are rarer, and are left for a later section (see §5). We do note the exciting developments on the question of social complexity and diversity in chemical signals documented recently in non-human primate species [57,58].

(a). The acoustic channel

Most of the current evidence for the social complexity hypothesis comes from the study of vocal communication. This should not be taken as evidence that social complexity is more central to the evolution of complexity in the auditory channel relative to other signal modalities. Rather, it more likely reflects the emphasis on vocalizations in the study of animal communication more generally. This is understandable considering how familiar animal sounds are to humans (most people appreciate the beauty of bird song, while few people—even biologists!—are even aware of the elaborate visual displays performed by some reptiles and other animals) and because of the early development of methods for quantifying and experimenting with animal sounds, especially in field settings [59].

In one of the earliest datasets to suggest a role of social complexity in vocal signalling complexity, Kroodsma [60] described song repertoire size in nine wren species (family Troglodytidae). The major result emerging from the study was the relationship between mating system and song repertoire size. Males in highly polygynous species had much larger song repertoires than males in more monogamous species, suggesting a role of sexual selection in song repertoire size. An additional result that emerged from the study related to intraspecific encounter rates—those species in which individuals occurred at the highest conspecific densities and with relatively low diversity of other avian species tended to have larger and more complex song repertoires than those species in which individuals occurred at lower conspecific densities.

Blumstein & Armitage [61] carried out the first explicit comparative test of the social complexity hypothesis using data on 22 species of ground-dwelling sciurid (genera = Cynomus, four species; Marmota, seven species; Spermophilus, 11 species). The authors assessed social complexity with calculations from information theory that addressed the diversity of distinct social roles within groups of each species. Communicative complexity was described in terms of the number of distinct alarm calls each species possessed. The authors found an overall positive relationship between their social complexity index and the number of alarm calls in species' repertoires, although the effect was sensitive to the particular phylogenetic tree used in the analysis. However, Pollard & Blumstein [62] have updated this analysis using the most recent sciurid phylogeny and confirm this previous result. Evidence from a different mammalian group—the family Herpestidae—further corroborates Blumstein & Armitage's findings. For example, social mongoose species such as the obligately social suricates, Suricata suricatta, [63,64] and the facultatively social yellow mongoose, Cynictis penicillata, [65] have more diverse systems of calls—particularly in terms of calls related to affiliation—compared with the more solitary mongoose species such as the slender mongoose, Herpestes sanguineus [66].

Wilkinson [67] compared the complexity of the infant isolation calls of eight different bat species and found a strong positive relationship between colony size of the species and the species' amount of information in their isolation call systems. The interpretation here is that in species with enormous colony sizes, such as the Mexican/Brazilian free-tailed bat, Tadarida brasiliensis, there is a need for maximal individual distinctiveness in isolation calls, to increase chances of parent–infant recognition. In species with much smaller colony size such as Schlieffen's bats, Nycticeinops schleffenii, which often roosts solitarily, the isolation call system contains much less information—there is much less variation in acoustic parameters of calls across infants in these species. On the other hand, although group size seems to be a strong predictor of call complexity for this one call system, there is little evidence in other aspects of bat vocal repertoires to suggest a relationship between social complexity (such as stability of social groups and social bonds among individuals) and vocal complexity [67].

Finally, McComb & Semple [68] compared data on group sizes, the time individuals spent grooming (as a measure of social bonds) and the vocal repertoire size for 42 non-human primate species. Using a phylogenetic comparative analysis, they found that both group size and time spent grooming were strongly positively associated with vocal repertoire size across these species. These findings are consistent with the notion argued here that increases in social complexity (group size and time spent grooming) may drive increases in vocal complexity. However, the authors also pointed out that the direction of causality was unclear because sexual or natural selection might have initially led to the evolution of vocal complexity that then allowed larger group sizes or more complex social structures to be possible in these species.

Freeberg [42] conducted an experimental study with Carolina chickadees, Poecile carolinensis, which permitted a more explicit causal analysis. The dependent measure in the study was the amount of information in chick-a-dee calls of individuals. This call is the most common vocal signal used by individuals throughout the year and functions in social cohesion [69–72]. There seems to be a relationship between chick-a-dee call production and social affiliation in this species. Captive males that were found frequently in close proximity with their female conspecific companion produced more chick-a-dee calls compared with males that were rarely found in close proximity with the conspecific female with which they were housed [73]. In the experimental study [42], captive flocks of chickadees that varied in group size of two, four or six birds (which are in the range of normal flock sizes of this population) were formed in aviary settings. After roughly one month of such captive housing, the birds in the larger flock sizes were producing chick-a-dee calls with more information than birds in smaller flock sizes. More information in these calls stemmed from a greater diversity of note types and note type combinations in the calls produced—the birds in larger groups were producing calls with more variation in note composition compared with birds in smaller groups. Variation in note composition of chick-a-dee calls of this species has been associated with predator detection [74–76], food detection [77,78] and signaller flight behaviour [79,80]. The Freeberg [42] study represented the first experimental evidence that variation in social complexity (group size) could drive plastic changes in vocal complexity (information in vocal signalling systems). These differences in amount of information in the chick-a-dee calls of individuals in experimentally manipulated group sizes were paralleled by differences in calls obtained via naturalistic observation of wild chickadees that varied in group size at time of recording.

(b). The visual channel

One of the first major comparative studies to assess links between social complexity and complexity of visual signals was Rohwer's [81] study of plumage variation in 29 avian species (orders Piciformes and Passeriformes) in relation to social grouping structure in overwintering months. Species that ‘flocked’ had greater plumage variability than did species that were ‘resident/spaced’. Rohwer's interpretation was that greater individual distinctiveness in plumage is necessary in large social groups for individual recognition or for assessment of relative dominance status, and such plumage variability is not necessary in solitary or very small groups (e.g. female–male pairs) that defend territories [81]. A much-needed follow-up to this work would be comparative analyses that incorporate the phylogenetic relationships among the species being compared.

Maestripieri [82] compared three macaque species (genus Macaca) in terms of their levels of social cohesion and the diversity of the use of gestures in their visual display repertoires. The three species were fairly similar to one another in the use of dominance and submissive gestures, but they differed in the use of affiliation and bonding gestures. Rhesus macaques (Macaca mulatta), the most despotic of the three species and the one with the greatest social avoidance, used significantly fewer gestures in common social interactions and contexts compared with the two species with greater social complexity. While it is difficult to make any conclusive statements with an effective N of 3, these data are nonetheless consistent with the social complexity hypothesis. More recently, Dobson [83] conducted a study of 12 non-human primate species and compared group size with the diversity of facial expressions used by individuals of those species. Using a phylogenetic approach, Dobson [83] found that non-human primate group size was a strong predictor of facial expression diversity. Taking these two non-human primate studies together, then, we have support for the early notion raised by Darwin [5] that in socially complex species gestures and expressions should be as important to signalling as vocal signals.

In a study of non-avian reptiles, Ord et al. [84] assessed the repertoires of distinct postures that elaborate the core ‘head nod’ and ‘push up’ visual displays of 122 territorial lizard species (families Iguanidae and Agamidae). The authors used phylogenetic regressions and ancestor state reconstructions to show that having a small home range was strongly predictive of species' possessing large display modifier repertoires, as were being arboreal and hunting moving prey. One possibility is that small home ranges result in greater male–male competition at closer or overlapping territorial boundaries, resulting in increased selection pressure for larger repertoires. This interpretation follows from a previous study by the same authors showing a positive relationship between an index for the intensity of territorial competition and large repertoires [43]. While variation in sexual selection need not always reflect variation in social complexity per se, selection that results from frequent and often intense social interactions (e.g. territorial encounters, courtship) do fall under our definition of social complexity. Another possibility raised by the authors is that species with primarily herbivorous diets are less able to defend territories owing to the nature of the food resource, and so may face less selection pressure for territoriality and therefore for aggressive visual signals, resulting in smaller display modifier repertoires in herbivorous species that often have larger home ranges [85]. This is also consistent with the social complexity hypotheses: as the intensity or frequency of interactions becomes diminished in herbivorous species (sociality becomes reduced), we would predict the evolution of simple signals or even the evolutionary loss of this type of signalling system altogether.

4. Social complexity as a driver of communication

While group size and role structure are important aspects of social complexity, the evolution of bonded social systems [86] within some lineages of mammals and birds adds a further layer of complexity. Although pairbonded monogamy might seem to be one of the simplest forms of group-living, in reality, the cognitive demands for maintaining these kinds of pairbond relationships through time seem to be much greater than those required to maintain very much larger social groups where relationships are less intense, less structured and more casual [87,88]. One reason for this is that members of a pairbond need to have a much more subtle appreciation of each other's needs and intentions than is the case when relationships are more casual and ‘of the moment’. Thus, in a large promiscuously mating social group, individuals' relationships can be based on simple memory of the last few encounters (who won, who lost). Predicting how another individual might behave could even be done on the basis of generic rules (is he bigger than me?). In contrast, in bonded relationships, individuals need to have much greater familiarity with the other member of the dyad and be able to respond in a more subtle and discriminating way to signals in order not to destabilize the bond. Primates have carried this trend to its limits by generalizing the pairbond relationship to other individuals so to create ‘friendships’ [89].

One example of this social complexity is the fact that, in primates, alliances (generally mediated through social grooming) are established well ahead of their being needed, whereas in other species, alliances are more typically ‘of the moment’ [90]. Since the future reliability of an alliance is easily damaged by, for example, one member threatening the other or failing to come to its support when attacked by a third party, relationships that have been damaged in this way are repaired by a near-unique process known as ‘reconciliation’, which typically involves one party grooming or touching the other, or giving contact calls or other vocal signals to it [91–93]. Another example of signalling complexity is the use of ‘meta-signals’ such as relaxed open mouth (‘ROM’) play faces and play vocalizations to comment on the significance of an action (in effect, ‘I am about to bite you, but don't take it seriously’ [94,95]).

These behavioural complexities associated with bonded relationships (and the greater cognitive processing demands that presumably underpin them) are reflected in the larger brains, and especially neocortex, of pairbonded species compared to those having polygamous mating systems in mammals and birds in general [87,96], and in anthropoid primates in particular [97–99]. More generally, this phase transition in relationship and signalling complexity may be related to the kinds of mentalizing skills (i.e. the ability to read the mind states of another individual in order to understand their intentions and appreciate their particular perspective) that are an essential feature of human social life, and for which the precursors may already be found in primate social cognition, and perhaps that associated more widely among the birds and mammals with pairbonded sociality [99]. That these kinds of mentalizing skills are cognitively demanding (and hence require more neural material to be recruited for their processing) has been shown by neuroimaging studies of humans demonstrating that individual differences in mentalizing competences correlate with volumetric differences in those regions of the brain known to be essential for basic theory of mind competences [44,45]. More importantly, there appears to be a parametric effect on recruitment of neurons in these regions when subjects are working at progressively higher levels of intentionality [100].

Such considerations might lead us to suppose that signal repertoire complexity should be more complex in such species, independently of group size. Although this prediction has never been tested, it is certainly the case that primates have unusually mobile facial musculature [83], and hence may well have a wider range of facial signals compared with most other mammals. Given the bonded nature of all primate groups, we might also predict that species that live in large polygamous groups will have more complex communication systems than those that live in monogamous pairs. However, there is no evidence that facial signal repertoire size or vocal repertoire size is larger in polygamously mating species than in pairbonded species once group size is partialled out (data from Dobson [83]: t = 1.748, d.f. = 10, p = 0.105; using genus mean values, data from McComb & Semple [68]: t = 0.003, d.f. = 20, p = 0.997).

A second aspect of social complexity is the structuring of the group into a number of layers, as is commonly found in anthropoid primates (including humans), but also in a number of other socially complex mammals such as elephants and orcas [101,102]. What makes these social systems complex is that the network structure does not necessarily parallel the size of the social group, but rather is additionally affected by the fact that relationships in different layers have different qualities [103]. One might, therefore, expect signal complexity to be related to network structural complexity as well as to the size of the group in these species.

The limiting case in terms of social complexity is, inevitably, our own species. Even by primate standards, humans live in unusually large and complexly organized societies [101,104]. This invites the question as to whether human signal repertoires are simply proportionately larger than those of other primates (relative to the differences in group size) or whether they represent a phase shift in qualitative complexity to accommodate the larger size and greater complexity of human societies. It is certainly the case that some conventional primate-like non-verbal signals have a more complex form in humans than they do in other closely-related primates such as the apes (for example, laughter [105,106]). However, this hypothesis has not as yet been formally tested.

Although human facial expression has been studied in some detail, beginning, as we have already noted, with Darwin himself [5], most of this research has focused on demonstrating either the universality or the cultural specificity of human facial expressions, having been somewhat trapped in the insidious nature/nurture debate. Unfortunately, few of these studies allow us to undertake comparative cross-cultural analyses. Most suffer from design problems (not least the fact that they invariably involve forced choice naming of emotions from still photographs) that make it difficult to draw any firm conclusions either way [107]. At best, the most compelling conclusion we can draw from these studies is that familiarity with Western culture is the best predictor of ability to correctly name facial expressions of emotion [107]. Worse still, almost all these studies focus on facial cues for a handful of emotional states (happiness, sadness, anger, etc.,), and ignore the many other facial signals that humans commonly use (nods, winks, grins, laughs, pouts, head shakes, etc.). In the only detailed naturalistic cross-cultural study, Eibl-Eibesfeldt [108] reported many cultural universals, but also a number of marked cultural differences (notably in the way affirmation and negation are signalled). Unfortunately, working in the classical ethological tradition as he did, Eibl-Eibesfeldt [108] does not provide the kind of data that would allow us to undertake quantitative cross-cultural analyses. Nonetheless, taken together, these findings do seem to suggest a role for learned flexibility in signalling and cue recognition that leaves open the possibility for a relationship between signal complexity and social complexity or community size. But this is, at best, speculation and the hypothesis awaits detailed testing.

While it is not clear whether human facial or non-verbal signal repertoires are larger than those of other primates, it is obvious that language at least adds a level of signal complexity that is unmatched by any other species. This looks suspiciously like a phase transition whereby a new channel of communication has been opened up because more conventional channels (i.e. non-vocal signals) have reached an asymptotic limit and cannot be stretched any further [29]. If these patterns hold, then, in terms of language itself, one might predict that there will be a relationship between language complexity and social community size. In this context, the appropriate social group is the language community (i.e. all those who speak the same language), which may be numbered in the millions, rather than the social community in the cognitive sense (who do you know?). However, contrary to prediction, the phonological and morphological complexity of human languages is negatively related to community size [109]. It seems that the larger the number of people who need to speak a language, the simpler its morphological structure and grammar has to be. Most likely, this reflects transmission constraints, but perhaps also drift. However, with much more detailed data available on language structure in very large numbers of languages (greater than 2000), this clearly provides rich opportunities for testing the social complexity hypothesis.

5. Limitations to current knowledge and alternative routes to communicative complexity

Whereas the social complexity hypothesis is an intuitive explanation for why communicative complexity evolves, and has support from a number of studies at both the proximate and ultimate level, there are critical gaps in our understanding of how communicative complexity arises. The generality of the social complexity hypothesis rests on the range of taxonomic groups and signal modalities that exhibit a link between social complexity and communicative complexity. Much of the existing evidence for the social complexity hypothesis is limited to birds and mammals, and to vocal communication as the primary channel of study. Yet, there are a diverse array of organisms in the natural world that rely heavily on communication (fish, reptiles, amphibians, crustaceans, insects and other invertebrates) and do so in a variety of different modalities (visual, olfactory, tactile/vibratory and electrical). To a lesser or greater degree, communicative complexity within a particular signalling system varies among all closely-related species, as does the relative social complexity of those taxa. We argue that the social complexity hypothesis has much broader relevance than current research suggests, but the extent of this relevance is unclear because of the shortage of research on species outside the typical model systems used in communication and on non-vocal forms of communication.

Furthermore, we know very little about the relative roles of social and non-social factors that drive or limit the evolution of communicative complexity in the taxa that have been studied to date. There are other ways in which communicative complexity might evolve, either in conjunction with, or independently of, increases in social complexity, and these alternatives need to be evaluated if we wish to gain a full understanding of how communicative complexity evolves. While we inevitably view social complexity as an important, indeed fundamental, driver of communication complexity, it is essential to be able to exclude alternative explanations that might lead to differences in communication complexity between taxa. These essentially provide the null hypothesis against which the predictions of the social complexity hypothesis need to be tested. In other cases, they may provide selection pressures that act against the pressures generated by social complexity, so constraining the evolution of communicative complexity in cases where we might predict it to occur. We briefly review some of the more important of these possibilities here.

(a). Habitat

The physical environment plays an important role in shaping the design of animal signals for most taxa, irrespective of the function or modality of communication used [110,111]. This is because properties of the environment constrain the type of signals that will be readily detected and evaluated by receivers, leading to the selection of some signal designs over others. The environment is often invoked as an explanation for the origin of signal diversity among closely-related species when those species communicate in habitats that differ in the conditions that affect signal reception [112]. Conditions that can result in signal degradation include background noise that masks calls [113] or displays [114], or poor lighting that reduces the visibility of ornaments [115] or displays [116]. These environmental variables are normally thought of as limitations on signal design rather than factors that promote signal elaboration or diversity. Noise might lead to changes in call frequency [117,118] or the speed of display movements [119], but it is rarely considered influential in promoting an increase in call or display complexity. There are, however, important precedents that demonstrate how the environment can result in the evolution of signal complexity. Examples of habitat-dependent signal elaboration include the evolution of some multi-modal signals in which an animal uses signals that incorporate components in more than one modality (e.g. a foot thumping display—vibration signal—and a conspicuous leg ornament—visual signal [120]) or the evolution of alert components at the start of a signal sequence [116]. The question is not whether signal elaboration occurs as a function of the environment (it does; see Hebets & Papaj [121] for other examples in addition to those presented above), but how frequently it occurs.

(b). Predation

Another variable that is more frequently assumed to act as a constraint on signal elaboration rather than a driving force for complexity is predation [122]. Complex signals are often those that are conspicuous, and conspicuous signals can draw the attention of unwanted receivers as well as the intended targets. A classic example is the túngara frog in which a male can produce a mating call that is either basic in structure—a ‘whine’—or one that is more complex and more attractive to females—a ‘whine’ followed by one or more ‘chucks’ [123,124]. Predatory bats, however, are able to use the conspicuous chuck component of the mating call to home in on calling males more effectively than if the call were simply left at a whine [125,126]. When the probability of predation is high, males compromise on signal complexity and produce only the most basic call, even though calls containing chuck components give males an advantage in attracting females [125].

Yet, predation can promote the evolution of communicative complexity in a number of different ways. For example, predation pressure in Campbell's monkeys may be driving vocal signalling complexity through variation in how signals are combined in sequences [127]. Communicative complexity can also emerge through the evolution of entirely new signals that increase signal repertoire size with no change in the frequency or nature of social interactions among conspecifics. The evolution of pursuit-deterrent signals is an obvious example. The ‘stotting’ of Thomson's gazelles is believed to signal to a predator that it has lost the element of surprise and honestly advertises physical condition, and consequently that the likelihood of capture is low [128]. The recent discovery of a previously unknown signal modality in ground squirrels provides another example. Female California ground squirrels use conspicuous tail flagging to deter snakes from entering burrows that shelter pups. In the presence of infrared-sensitive pit, vipers, such as rattlesnakes, females elaborate the tail-flagging signal (a visual signal) with radiant heat by increasing the flow of blood to their tails (an infrared signal, [129]). This is a fascinating case in which a signal has evolved from one modality to two. Other examples of predation leading to the elaboration of signal design or repertoire size are found in the electric signals of weakly electric fish [130] and the alarm calls of Diana monkeys [131]; in both cases, there was no change in social complexity, although in the case of the electric fishes, increased signal complexity became important in mate choice once it evolved [130].

(c). Species recognition

When animals frequently encounter congeners in the environment, there are clear selective advantages for animals to discriminate conspecifics from heterospecifics in both mating and territorial contexts (reviewed in Ord et al. [132]). This should in turn result in considerable selective pressure for social signals to convey reliable cues on species identity. Many studies have confirmed that species-specific components exist in mating and territorial signals [132]. It seems reasonable to expect the frequency of encounters with sympatric species, especially those of similar appearance and ecology (e.g. those heterospecifics belonging to the same or closely-related genera), to promote complexity in signal designs or repertoires. Character displacement—the phenomenon of ecological and phenotypic divergence when previously allopatric species come into contact [133]—occurs in signalling systems. However, when encounters generally occur between few congeners (e.g. two species), character displacement is more likely to prompt divergence in signal characteristics rather than the evolution of communicative complexity per se (e.g. a shift in song or call frequency that minimizes overlap between the songs or calls of a sympatric heterospecific [134,135]). In most instances, the path of least resistance is to modify an existing signal characteristic (e.g. song frequency) rather than add complexity to a communication system (e.g. an entirely new note or syllable in a song).

The situation changes when species begin to encounter many different congeners. In a crowded community of communicating animals, the only viable solution left might be to increase the overall complexity of a signal because the range of possible modifications to existing signal characteristics that minimize overlap with all other heterospecific signals is severely limited. This should result in the evolution of communicative complexity with increases in the number of sympatric species encountered, and there is evidence for this in nature. Caribbean Anolis lizards that overlap with many sympatric congeners perform territorial headbob displays that are more elaborate than species that rarely encounter other Anolis lizards [40]. In ducks, geese and swans, the evolution of brightly coloured plumage is more likely in species occurring in sympatry than those in allopatry [136]. In much, the same way that the social complexity hypothesis predicts that the frequency and nature of interactions with conspecifics will drive signal complexity, the frequency and the range of heterospecifics that a species encounters will drive complexity for enhanced species recognition.

(d). ‘Neutral’ evolutionary processes

Perhaps the most radical suggestion is that communicative complexity can arise in the absence of direct selection and purely through the stochastic nature of evolution. That is, complexity incrementally accumulates in some signals over evolutionary time through processes such as genetic drift. This will result in species that differ in signal design because of differences in the number, order or type of mutations that have arisen in each lineage. Evolutionary biologists are well aware that ‘neutral’ phenotypic change (change that serves no particular adaptive function) is an inherent outcome of evolution. Indeed, the original motivation for developing phylogenetic comparative methods was to account statistically for the fact that species accumulate phenotypic changes over time and tend to share or differ in their phenotypes as a function of their phylogenetic relationships [137]. Communication researchers may be less familiar with the notion that stochastic processes feature prominently in evolution and that the behaviour of animals may not always be the product of current or even past selection. It is not surprising then that there are few studies examining the question of whether factors such as genetic drift underlie variation in signal complexity among closely-related species. Yet, we should not expect the evolution of communication to be any different from other phenotypic traits.

One telltale signature of neutral evolution is the presence of ‘phylogenetic signal’ in comparative datasets. Phylogenetic signal reflects the tendency for closely-related species to share phenotypic characteristics. In the context of signal complexity, we should expect neutral evolution to increase communicative complexity over evolutionary time (e.g. greater signal complexity in lineages that have had more time to evolve) and similarities in the level of communicative complexity (high or low) among close relatives compared with more distantly related species. There is evidence that signal complexity tends to increase with lineage age and vary among taxa as a function of phylogenetic relationships in the chemical signals of ants [138] and Drosophila [139], in the vocal signals of birds [140–142] and frogs [143], and the visual signals of darter fishes [144] and Anolis lizards [40]. We should add that interpretations of phylogenetic signal should be made cautiously because it can reflect a number of different biological factors (e.g. niche conservatism or stabilizing selection more generally) or statistical artefacts (e.g. biases in taxon sampling, measurement error in phenotypic traits [145]).

An intriguing example of the potential for neutral evolution to produce communicative complexity is presented by Wischmann et al. [146] who studied the trajectory of signal evolution in replicated populations of cooperatively foraging robots. Robots could signal to one another by flashing blue and green lights. A remarkable outcome of this study was the evolution of two different forms of signal behaviour: a simple signal that relied on flashing the light of just one colour and another that used a more complex combination of both colours. In every other respect, the populations of robots were identical. This study shows that, at least in principle, communicative complexity can evolve as an arbitrary outcome of mutation and recombination independently of any social or ecological variable. Caribbean Anolis lizards offer a potential example of something similar in nature. Two separate clades of lizards occur on the islands of Jamaica and Puerto Rico. Lizards on both islands use a combination of headbob displays and the extension of a large dewlap to advertise territory ownership, lizards on both islands defend territories in visually difficult habitats (environments in which there is a lot of distracting movement from windblown vegetation or poor visibility from low light [147]), and lizards on both islands have similar levels of competition for territories (e.g. the range of conspecific densities is comparable on both islands [148]). However, the signal strategy used by lizards on Puerto Rico is arguably more complex than the strategy used by lizards on Jamaica. Puerto Rican lizards enhance communication by tailoring the speed, duration and timing of their display sequences depending on the prevailing level of visual noise or ambient light at the time of display [119,147,148]. Jamaican lizards, on the other hand, do not change the production of displays in any way, but seem instead to rely on the use of a highly conspicuous dewlap display [147,148]. Two alternative signal strategies that appear to be equally effective in advertising territory ownership in difficult signal environments [116] that differ in complexity have evolved independently of social and ecological factors. Although signal plasticity may not fit the traditional definition of signal complexity (box 1), we use this example to illustrate that differences in behavioural complexity can evolve among taxa as an apparent consequence of the vagaries of evolution.

6. Conclusions: aiming for causal inference

We hope we have made the case (i) that social complexity may play an important role in driving communicative complexity in animal species and (ii) that further tests of this possible role will be of great interest—and are much needed—to advance our understanding of communicative evolution. We are still in large part in an inductive phase of study on the question of how the complexity of social groups might influence complexity of signalling systems. There are substantial comparative, correlational and even experimental data to support the argument, which we have reviewed here. On the other hand, it is necessary to move more solidly into the deductive phase of study on the question. This was one of the underlying motivations for us in writing this piece, and for the theme issue in general. In short, what is needed are datasets that provide strong tests of the social complexity hypothesis against alternatives such as predation or species recognition pressures.

Social grouping provides substantial benefits to group members [149]. As such, socially complex groups may provide ‘insulation’ from the physical environment that is not experienced by individuals in very simple social groups or by solitary individuals. We have argued that socially complex groups demand greater communicative complexity of group members compared with less complex groups, following from arguments about the need for increased social cognition (the social intelligence hypothesis raised above [16–19]). Social complexity may therefore emerge from increases in individual-level complexity stemming from the needs of larger groups or groups with a greater diversity of roles or relations. On the other hand, social complexity can also emerge from lack of change in, or even decreases in, individual-level complexity combined with greater divergence of individual roles within the group. This has been theorized and documented in social insects, for example [150]. As an example, a sentinel individual in a group may not need to produce food detection signals if it is never in that role in its group, but can simply rely upon other group members to obtain food. As such, we believe that our understanding of communicative complexity will grow with increased understanding of physical environmental pressures faced by individuals, in addition to that of social environmental pressures we have stressed here.

It becomes evident when considering alternative causes of communicative complexity that the direction of causality becomes especially important in tests of the social complexity hypothesis. Complex communicative systems may evolve from pressures related to social complexity, but if complex communicative systems evolve for other reasons, this could in turn facilitate the evolution of greater social complexity among animals (e.g. the formation of larger social groups [68]). While studies might demonstrate a link between social complexity and communicative complexity, this in itself does not confirm the direction of causality that is explicit in the social complexity hypothesis: social complexity drives communicative complexity, and not vice versa. Experimental manipulations [42] will help, but must still be conducted with careful consideration of other factors that might influence the design of signals. Phylogenetic comparative analyses in which social, ecological and evolutionary factors are weighed against each other are another valuable approach [151]. Although problems can exist with phylogenetic methods that reconstruct ancestor states [152,153], if done carefully, these methods can provide a way to determine the likelihood that social complexity preceded changes in signal complexity [154]. Path analysis may be especially helpful, because it allows us to test between alternative models that separate out evolutionary causes (i.e. selection), and constraints that must be resolved to allow evolutionary change and its consequences, such as windows of opportunity that an evolutionary change opens up (for an example of such an analysis in the context of primate brain evolution, see [98]). Perhaps, the greatest challenge faced by future studies wishing to test the social complexity hypothesis is finding a way to disentangle the alternative selection pressures that inevitably act on complex animal signals and how these factors are related to changes in signal design.