Evolutionary transitions in learning and cognition

Abstract

We define a cognitive system as a system that can learn, and adopt an evolutionary-transition-oriented framework for analysing different types of neural cognition. This enables us to classify types of cognition and point to the continuities and discontinuities among them. The framework we use for studying evolutionary transitions in learning capacities focuses on qualitative changes in the integration, storage and use of neurally processed information. Although there are always grey areas around evolutionary transitions, we recognize five major neural transitions, the first two of which involve animals at the base of the phylogenetic tree: (i) the evolutionary transition from learning in non-neural animals to learning in the first neural animals; (ii) the transition to animals showing limited, elemental associative learning, entailing neural centralization and primary brain differentiation; (iii) the transition to animals capable of unlimited associative learning, which, on our account, constitutes sentience and entails hierarchical brain organization and dedicated memory and value networks; (iv) the transition to imaginative animals that can plan and learn through selection among virtual events; and (v) the transition to human symbol-based cognition and cultural learning. The focus on learning provides a unifying framework for experimental and theoretical studies of cognition in the living world.

This article is part of the theme issue ‘Basal cognition: multicellularity, neurons and the cognitive lens’.

1. Introduction

There are many characterizations of cognition (sometimes called intelligence) in living organisms, ranging from a very broad one that assigns cognition to all autopoietically organized living systems to a very narrow one that portrays cognition as the ability to form mental representations that can guide behaviour (for reviews, see [1,2]).

We opt for a characterization of cognition that encompasses non-neural and neural systems and allows the study of different types of biological cognition from an evolutionary-comparative perspective. To be useful, this characterization has to provide a common ground for comparative studies. Memory and learning capacities are common to most characterizations of cognition. For example, Lyon 7 [1, p. 2] suggests that memory and learning are part of the basic cognitive tool kit that is present even in bacteria, with cognition being characterized by ‘…sensory ST [signal transduction], valence, communication, sensorimotor coordination, memory, learning, anticipation, and decision-making in complex and changing circumstances’. Similarly, Shettleworth [3, p. 5] defines cognition as ‘the mechanisms by which animals acquire, process, store, and act on information from the environment. These include perception, learning, memory, and decision making’. Memory is also fundamental to Baluška and Levin's view of non-neural and neural cognition: ‘Here, “cognition” refers to the total set of mechanisms and processes that underlie information acquisition, storage, processing, and use, at any level of organization [1]. Memory is an essential component of these processes, at all levels' [4, pp. 1–2]; their notion of memory includes both storage and recall.

(a). Learning and cognition

Learning encompasses or enables all the features that have been listed as capacities or mechanisms of cognition, so any system with the capacity to learn can be described as a cognitive system. Learning is defined as a process leading to an experience-dependent behavioural response of a system. It requires that:

• (i) A sensory stimulus that originates either from the activities of the system or from the external biotic or abiotic world leads to a change in the internal state of the system (the stimulus is encoded).
• (ii) A memory trace of this change is retained (storage); retention requires active stabilization and involves valence mechanisms of positive or negative reinforcement.
• (iii) Future interactions with the stimulus or associated stimuli lead to a change in the threshold of the behavioural response (recall).

(For a more detailed characterization emphasizing the relationships among these processes, see [5, pp. 228–230].)

Learning thus includes (by this definition) sensor–effector coupling, attribution of valence (intrinsic reinforcement), storage and recall involving a change in response threshold (so the system can be said to ‘anticipate’ the effects of learned stimuli). It enables complex (experience-dependent) decision-making within the lifetime of the individual, and can lead to changes in response thresholds when the signal is produced by another biological entity (social learning; communication). A learning system is, therefore, a cognitive system (for some explicit identifications of learning with cognition, intelligence or ‘mind’, see [2,6, pp. 20–21, 7]). Cognition, which includes both learning and the processes and actions enabled by learning, is the outcome of the evolution of learning.¹

The advantage of focusing on learning is threefold: first, learning is a single capacity that is clearly defined; second, it points to functional and temporal links among sensory changes, storage and behaviour, that is, to a cognitive architecture; third, learning theory is well-developed both conceptually and methodologically, so it is possible to experimentally test for learning in any living system, and to distinguish among different types of learning and hence of cognition.

Our focus in this paper is on the evolution of neural learning, and the first evolutionary transition we consider is the transition from learning in non-neural organisms to learning in neural animals. The first question we need to ask is whether learning can be attributed to non-neural organisms. The answer is a categorical YES. If one accepts the definition of learning just presented, learning can be found in single-celled organisms, single cells within a multicellular organism (e.g. neurons), non-neural multicellular organisms such as plants and fungi, and subsystems like the immune system (see [8] and references therein). Learning by habituation and sensitization has been found in bacteria [1], plants [4], slime moulds and fungi (for reviews, see [2,4] and papers in this issue), as well as in dividing and non-dividing cells (like neurons) in multicellular organisms. Habituation, the decrease in a reflexive behavioural response to a repetitive stimulus, enabled organisms to ignore irrelevant stimuli, thus, saving energy. Sensitization, in which an increase in a reflexive behavioural response follows a strong stimulus leading to decreased threshold of response, is the mirror image of habituation, enabling a more effective (e.g. more rapid) response to a salient stimulus. Sensitization also occurs when a specific reflexive response is affected by the general excitatory state of the organism (and, in animals, by the state of neighbouring linked neural reflex pathways, which can modify the response threshold). We suggested that learning by habituation and sensitization in non-neural organisms is based on epigenetic molecular mechanisms such as chromatin-marking (e.g. DNA methylation and histone modifications), self-sustaining metabolic loops, RNA-mediated memory and memory based on three-dimensional molecular templates, all of which enable an induced phenotypic cell state to persist even when the inducing stimulus is no longer present. We presented toy models illustrating how habituation, sensitization and more complex forms of learning can occur in single cells on the basis of these epigenetic mechanisms [8].

The involvement of epigenetic mechanisms in neurons during behavioural learning is well known and has been intensely researched for over a decade [5,9]. Bronfman et al. [10,11] suggested that epigenetic mechanisms are involved in the establishment and maintenance of long-term neuroplasticity, with specific epigenetic learning dynamics corresponding to specific behavioural learning histories involving synaptic modulations. Old epigenetic memory mechanisms were recruited when neural learning and synaptic memory first evolved [8], as were bioelectric fields that maintain and regenerate morphological patterns in all organisms [12].

(b). Evolutionary transitions

We approach the evolution of learning from an evolutionary-transition perspective that highlights qualitative differences in biological organization among organisms and makes sense of broad evolutionary patterns of diversity, novelty and adaptability. Four major types of evolutionary transitions have been recognized:

• (i) Ecological transitions (e.g. from aquatic life to terrestrial life), which involve new integrated suites of physiological and morphological adaptations.
• (ii) Transitions that involve additions to the type of hierarchically nested variations that are selected—selections among genes; among genes and behaviours; among genes, behaviours and virtual non-symbolic representations (e.g. action plans); and among genes, behaviours, virtual representations and symbolic-cultural representations. Following Dennett [13], we call these transitions in intentionality.
• (iii) Hierarchically nested teleological transitions—from non-living to living systems, from non-sentient organisms to sentient ones and from non-reflective animals to reflective-rational ones [5].
• (iv) Informational transitions, which, as suggested by Maynard Smith & Szathmáry [14], involve changes in the acquisition, encoding, storage and transmission of information that lead to higher-levels entities with greater division of labour and new levels of hierarchical control. Such changes include either increase in nested hierarchy (such as the transition from single cells to multicellular organisms made up of cells) or the addition of a new way of storing and using information (such as the transition from RNA as hereditary material and enzyme to DNA as hereditary material and proteins as enzymes). Both types of transition entail the addition of new and higher levels of information integration and top-down control within the individual.

2. Neural transitions

Maynard Smith & Szathmáry [14] focused on the transmission of information between generations that is determined by the genetic inheritance system, so the transition to neural organisms, which epitomizes a new way of transmitting information within an individual animal, was overlooked. With the exception of the transition to humans with symbolic language, evolutionary transitions that altered the way neural information was integrated and transmitted and which led to new types of cognition (including the type of cognition that constitutes sentience) were also omitted by Maynard Smith & Szathmáry.

In this paper, we focus on informational transitions in cognition, which involve new mechanisms that integrate neural information, evaluate and store it, and coordinate the actions of the organism. We recognize five major neural transitions, with the first two, on which we expand, occurring in phylogenetically early animals. The five transitions are: (i) the transition from non-neural to neural organisms that learn by neural habituation and sensitization; (ii) the transition to animals with a central nervous system (CNS) and flexible but limited associative learning (LAL); (iii) the transition to animals with open-ended (unlimited) associative learning, with hierarchically organized brains enabling mental representations (subjectively experienced mappings of world, body and prospective actions); (iv) the transition to imaginative animals with additional hierarchical levels of neural and mental representation; and (v) the transition to symbolizing, culturally learning humans [5]. Sections 2a and 2b examine the first two neural transitions, and §2c briefly outlines the principal features of the three subsequent cognitive transitions. We end by discussing the implications of the learning-transition framework.

(a). The transition from non-neural to neural animals and the first neural learning

Although based on factors and mechanisms already present in non-neural organisms, the nervous system employs a novel way of information encoding, processing, storage and transmission. To appreciate the functional revolution that the nervous system brought about, we first briefly discuss its evolutionary origin.

It is widely agreed that the nervous system evolved in multicellular animals, and that one of its major functions was rapid and flexible sensory–motor coupling through coordination of the animal's locomotory movements, and also the movement of its internal organs (e.g. the gut, reviewed in [5]). The first neural transition was, in fact, muscular-neuronal.

The nervous system's coordinating functions are enabled mainly by the plasticity conferred by the evolution of novel types of developmental exploration-stabilization processes [5]. Exploration-stabilization processes are manifest at all levels of biological organization and are based on a common principle—the generation of variations from which only a subset is eventually stabilized (selected). Examples are the selection of genetic mutations in populations; selective stabilization of biochemical networks within a cell; developmental selection processes that occur during ontogeny in plants and animals and lead to homeorhesis [15]; stabilization of exploratory motor behaviours. In all cases, variations that confer benefits, or, more generally, that lead to a set-point (an attractor state) are stabilized/selected. As the nervous system evolved, new levels of developmental selection were introduced: in addition to selection among neurons during embryogenesis, differential stabilization of synaptic connections takes place. Synaptic connections are overproduced, and most are pruned: connections that have the highest functional efficacy persist through the activity of internal reinforcing (valence) mechanisms. The net result is that only certain synaptic connections among the many initially produced are stabilized and retained. An additional level of developmental neural selection is selection of patterns of synaptic links among neuronal maps, a process that occurs at a higher hierarchical level, during later stages of neural evolution [16,17]. In moving neural animals, there are thus (minimally) two additional levels at which developmental selection through differential reinforcement is manifest: synaptic–neural and behavioural. These additional levels of developmental selection confer great adaptability on animals with a nervous system.

When and how the first neurons and nervous systems emerged are frequently debated questions. There is a controversy over whether neurons and nervous systems evolved only once or several times [18–21] and there are several hypotheses regarding the evolutionary origin of the neuron (reviewed in [5]; see also [18,21]). There is, however, general agreement that the simplest nervous systems arose over 600 MYA in ctenophores and/or cnidarians, as relatively non-differentiated nerve nets, i.e. neural meshes spread throughout the entire body. Nerve nets enable animals to perform a plethora of activities that require precise sensory–motor coupling, and to learn by habituation and sensitization.

Four significant interrelated innovations distinguish the neural communication system from other communication systems (such as the hormonal or the circulatory systems). These are: (i) highly targeted short- and long-range interactions based on contact wiring; (ii) a new common ‘currency’ of communication; (iii) high-speed signal transmission, and (iv) a new (synaptic) memory system.

• (i) Neurons are morphologically differentiated cells, with three major specialized elements: the cell body (the soma) and the projections leading to and from it (dendrites and axons). The projections can link together many different neurons, forming endless numbers of small- and medium-sized neural circuits embedded within huge nerve networks. The number of connections between a given neuron and other neurons may be vast, and the distances between two connected neurons may span dozens of centimetres and even reach several metres (in giraffes, see [22, p. 212]). Crucially, connections among neurons and between the sensors and effectors they link involve contact wiring, enabling targeted signal transduction, which is far more efficient and directed than signal transduction in non-neural organisms.
• (ii) Although electrical signalling is ubiquitous, in neural animals, the discrete, regenerative, neural electrical signal, the action potential, which transmits information along neurons, is a common communication currency into which all sensory inputs can be transformed. Information can be encoded digitally as a sequence of parallel and/or sequential action potentials. The advantages of such digital transmission (over analogue information transmission) are that it produces fewer errors and makes it easier to integrate data. In the neuron, the arrangement, density and properties of ion channels embedded within the membrane make the nerve cell excitable in response to local electrical changes. When the changes reach a certain threshold, the action potential is triggered. This electrical signal is discrete, in the sense that once the threshold is crossed and the action potential is elicited, it propagates, in one direction only, along the axon, as a stereotyped regenerative signal of constant amplitude, until it reaches the end of the axon. Crucially, the action potential is the same whether carried by sensory or motor nerves, and all modes of sensory stimuli—photons, chemicals, heat, sound waves and other mechanical types of energy—are translated into this common communication currency, the electrical impulse. This allows information from various sources to be integrated and is the basis of mapping within the nervous system of patterns of stimuli emanating from the world and the body.
• (iii) Electrical transmission occurs at high speed and with great specificity. If information is to be transmitted efficiently between remote body parts of a moving organism, it cannot rely on molecular diffusion alone, which is far too slow and non-localized. Days would be needed for a molecule to diffuse along an axon of 1 cm length [22], and some axons may be 100 times longer. The discrete electrical signal that evolved in neurons propagates through a 1 cm long axon in milliseconds, at a speed of 0.5–10 m s⁻¹ (and even faster in axons that are electrically insulated by a myelin sheath).
• (iv) The synapse is the basis of a new type of information storage. The synapse comprises the ends of two neurons and the tiny physical gap in between; it is the ‘point’ of contact between neurons (or one neuron and a muscle cell or a gland cell). Signal transduction (from electrical to chemical) occurs at the synapse through the release of neurotransmitter molecules that diffuse across the gap and bind to receptors of the postsynaptic neuron. The postsynaptic cell may have many points of input from different neurons, and the chemical information it receives is translated into local electrical changes that are summated and transmitted passively to its soma. If the summated change crosses a threshold, an action potential is triggered in the postsynaptic neuron. The synapse thus operates as a filter and a point of control, allowing only strong enough signals to be transmitted further. The area near the postsynaptic cell membrane is inhabited by hundreds of proteins that play various roles in modifications of the synapse, endowing it with great plasticity. They may, for example, modify the neuron's excitatory parameters and hence its responses to future stimuli. The modifiable synapse is the basis of a new type of memory—synaptic neural memory—which enables a new (neural) type of learning based on a novel type of developmental selection: differential stabilization of synaptic connections [23]. Neural animals, therefore, have two-tiered learning: in addition to the epigenetic memory and learning within neurons, their learning is also based on inter-neuronic synaptic memory.

Neural learning in the first neural animals—those with non-differentiated nerve nets—involved the modification of reflexes and exploratory motor patterns by habituation and sensitization. Since, as we noted in §1, unicellular organisms as well as plants, slime moulds and fungi also exhibit habituation and sensitization, what is the added cognitive value of the neural type of learning?

In animals with nerve nets, the scope of learning by habituation and sensitization is far greater than that in non-neural organisms. First, the co-evolved new type of motor effector, the muscle cell, greatly extends the range of sensor–effector reflexes and patterns of exploratory motor activity that can be modulated by habitation and sensitization [5, ch. 6]. Second, since habituation and sensitization in neural animals involve the strengthening or weakening of synaptic connections, highly targeted, rapid and long-range habituated and sensitized responses can be established, and can be influenced by many combinations of neural inputs. In plants and unicellular organisms, in contrast, learning through habituation and sensitization is far more limited. It is based on interactions between chemical signals sent by source cells and received by specific receptors in effector cells, on direct cell–cell contacts between neighbouring cells, on intracellular epigenetic memory and on diffusible hormonal signals and bioelectric fields [2]. Neural organisms employ these same memory mechanisms, but each and every wired neural connection can be modified in a specific and targeted manner, so that the response threshold of every postsynaptic neuron and every effector, including muscle sheets dedicated to motor activity, can be altered.

Although no cnidarian or ctenophore has a unique centre of communication like the bilaterian brain, some have local neural aggregations, such as nerve rings, which control specific types of movement, and clusters of ganglion-like neural structures that are involved in sight. Studies of cnidarians suggest that even the simplest neural nets support complex coordinated activities. These animals have multiple sensor–effector relations that can be modified by habituation and sensitization, and the extent and type of their exploratory motor behaviour patterns can also be sensitized and habituated [24]. They also have synchronous collective neural firings—for example, pacemaker activities that control and coordinate alternative movements. Another crucial neural function, reafference, requires lateral inhibitions between motor and sensory neurons: every moving multicellular organism must distinguish between stimuli emanating from its own activity (e.g. vibrations that result from its movement in water) and stimuli that are independent of its own activity (e.g. identical vibrations generated by a predator). While the latter require generation of the flight response, the former require that the flight reflex is inhibited. The motor neurons send an inhibitory copy of the signal they receive to the down-stream parts of the reflex path, suppressing the reflex reaction [25]. These types of regulatory interactions and plastic learning in nerve nets paved the way for the evolution of conditional associative learning (AL), one of the most revolutionary adaptive strategies that have ever evolved.

(b). The transition to neural centralization and to limited associative learning

Associative learning is defined differently by artificial intelligence scientists and psychologists. For the former, any change in the connection between elements as a result of their past activity counts as associative learning. Hebb's law—neurons that fire together wire together—is their example, and the nature of the link (reflexive or non-reflexive) is irrelevant [26]. For psychologists, associative learning refers to learning that involves the formation of a conditional pairing between a non-reinforcing stimulus or action and a subsequent reinforcing stimulus [27], and they refer to learning by habituation and sensitization as ‘non-associative learning’. We use the term conditional learning or conditioning in this sense here. Restricted learning by association has been recorded in plants (for a debate, see [28,29]), and there are a few disputed cases of conditioning in the protist Paramecium and in a cnidarian species [5].

Conditional associations can be formed between stimuli that under ordinary conditions do not trigger a response (neutral stimuli); between biologically important stimuli like those typically linked to the maintenance of basic homeostatic and reproductive functions and unrelated reflexive responses (e.g. one may become conditioned to blink when smelling food); between stimuli and the contexts in which particular stimuli and actions occur; and between motor activities and their reinforcing effects [27]. Such flexible conditioning, which in biological organisms depended on the evolution of a CNS, led to an enormous jump in adaptability, enabling animals to flexibly adjust their behaviour and physiology during their lifetime. It was probably one of the factors driving the greatest ecological diversification in the history of animals, the Cambrian explosion [30].

Two types of conditioning are recognized: classical (Pavlovian) and operant/instrumental (Skinnerian/Thorndikian) conditioning. Classical conditioning entails the formation of an association between a conditioned stimulus (CS) and an unconditioned stimulus (US). A US is defined as a stimulus that elicits a reflex response (an unconditioned response, UR). For example, the smell of food (US) innately elicits salivation (UR). Conversely, the sound of a metronome does not elicit salivation prior to learning and is, therefore, considered a conditional neutral stimulus (CS) with respect to salivation. The CS–US association is typically formed when the US repeatedly follows the CS, usually in close temporal proximity. Owing to the formation of the CS–US association, a conditioned response (CR) is acquired: the next presentation of the CS will elicit the CR, even in the absence of the US. Thus, the organism has learned to respond to the CS (e.g. salivate) as if the US were about to arrive.

In Skinnerian or operant conditioning, the probability of eliciting a certain action changes as a function of its reinforcement history: actions that were followed by a positive (or negative) outcome will be more (or less) likely to occur in the future, under similar circumstances. For example, a rat can learn to press a lever when hungry if this action is followed by the delivery of food. Skinner suggested that complex behaviour is the result of a sequence of stimuli, such that a discriminative stimulus not only provides the conditions for learning a subsequent behaviour pattern, but can also reinforce a behaviour that precedes it.

The relation and co-dependence between the two types of conditioning were debated during the first half of the twentieth century. But, however the two processes were conceptualized, it was clear that under most ecological conditions, it is very difficult to tease apart classical and operant conditioning, because both usually occur during a single learning episode. Special measures, such as tying dogs to a stand, or gluing a fruit fly to a rigid rod hooked to a torque meter, are required to distinguish between the classical and operant aspects of learning. This difficulty led Colomb & Brembs to replace the traditional distinction, based on the processes involved in each type of learning, with a classification based on what is learned: sensory world stimuli (world-learning) or one's own behaviour (self-learning) [31, p. 142]. This distinction is useful, but the question about the mechanistic and evolutionary relation between world- and self-learning remains. We suggest that world-learning and self-learning are functionally entangled in most learning conditions (the animal learns both about the world and about its own reinforced actions), that they share the same neural and cognitive architecture, that they have co-evolved and that they require a brain.

Both types of conditioning entail: (i) the sensing of a stimulus (or stimuli) that initiates the process (CS in classical conditioning; the internal sensory state of the exploring animal and the stimuli triggered by the object upon which it acts in operant conditioning); (ii) behaviour (CR in classical conditioning; the reinforced behaviour in operant conditioning); (iii) reinforcement that determines the salience of the stimulus or the elicited response. Another important similarity is that in both types of learning, the extent to which the animal anticipates a reinforcement determines the extent of learning. The more surprising and unexpected the reinforcement, the more learning occurs. These similarities led us to suggest that the same basic cognitive architecture supports both types of conditioning [5].

Most conditioning studies in animals have tested for classical conditioning, so any survey of the learning literature is biased and may suggest that classical conditioning is more common than instrumental conditioning. However, in well-studied ‘model’ animals, both types of conditioning have been shown to occur. For example, the tiny nematode Caenorhabditis elegans shows both [32,33], Aplysia possesses both classical [34] and operant [35] conditioning, and there is evidence for both classical and operant conditioning in planarians [36]. This does not mean that all animals employ world- and self-learning to the same extent. It is expected that the relative importance of world- and self-learning will depend on the sensory capacities of the animal and the degrees of freedom of its movements. At the mechanistic level, both world- and self-learning employ epigenetic memory mechanisms in addition to synaptic memory mechanisms [11].

There are, however, important differences among animals in the scope and sophistication of associative learning [5]. In this section, our focus is on what we call LAL. With LAL, spontaneous and stochastic exploratory activities and preexisting simple reflex reactions can be flexibly combined, reinforced and recalled. Moreover, other non-compound (elemental) stimuli such as a flash of light or single actions like pushing a button, which are unrelated to a particular reward or punishment, can also become associated with the reinforcement and lead to a future anticipatory response. However, although the animal can learn about the value of stimuli and actions, it cannot discriminate between differently organized multimodal, compound, novel stimuli or complex action patterns; it can only learn if there is a temporal overlap between the CS and the US, or the action and the reinforcer (i.e. there is no trace conditioning, where there is a temporal gap between the CS and US); it has a very limited ability for cumulative learning, and cannot make decisions requiring a motivational trade-off among learned actions, or learned and reflexive actions. LAL is, therefore, distinguished from associative learning that enables compound multimodal discriminations, trace conditioning and cumulative learning (called unlimited associative learning, UAL; discussed in the next section).

A survey of the taxonomic distribution of conditional associative learning (both limited and unlimited) suggests that all animals in which conditioning has been conclusively shown have a brain, although not all animals with brains can learn by conditioning [5]. A brain seems to be necessary but not sufficient for conditional, neural, associative learning. This may seem surprising, because a simple type of conditional learning (when the CS is a weak, habituated, reflex-eliciting stimulus) can occur in neural ‘preparations’ or through ganglionic learning (in ganglia severed from the brain; [37]). Conditioning, however, did not evolve in neural preparations or severed ganglia. It evolved from sensitization and habituation in intact moving animals [5, ch. 7].

If the connection between the presence of a brain and the capacity to learn through conditioning is not merely an artefact owing to our current scant and patchy information about the distribution of learning in animals, it may be an important clue for understanding the transition to associative learning. An integrating communication centre, a brain, seems to have been a necessary condition for the conditional, usually interrelated, world-learning and self-learning in animals.

The evolution of brains was linked to the advent of bilateral symmetry. As Holló and Novák [38] argued, after bilaterality evolved (probably more than once) in tiny, slow-moving ciliated animals during the Ediacaran, it was maintained in their macroscopic descendants, and is now present in 99% of animals. Once a single forward direction of movement was defined, the front parts of the animal were the first to meet (or seek) environmental stimuli. As a result, sensory neurons were concentrated in the anterior part of the body, the head, with coordinating motor circuits concentrated both in the brain and along the rest of the body. Such differentiation between the sensory and motor regions required integration of sensory inputs and coordination of motor outputs within regions as well as coordinated interactions between regions. Moreover, bilateral symmetry maximized the ability to swiftly change direction, because changing directions requires the generation of instantaneous ‘pushing’ surfaces from which the animal can obtain the necessary force to depart in the new direction. This resulted in hugely improved manoeuvrability, conferring great benefits on animals living in a world full of cnidarian and ctenophore predators.

The evolution of a CNS enabled the integration of incoming information (from the whole body and from the external environment) and the control of motor responses based on the evaluation of these stimuli. But the CNS controls more than behaviour. As Cabej [39] stressed, the CNS also controls morphogenesis and differentiation, and its regulation of development is the key to the understanding of the morphological evolution of animals.

As already noted, bilateral symmetry and a simple centralized brain were necessary but not sufficient for the evolution of conditional associative learning. In all animals capable of conditional associative learning, there is also some differentiation within the brain into sensory and motor integrating centres and recurrent interactions between them. This enables complex sensory–motor coordination. We suggested that this brain organization was related to the increase in oxygen concentrations during the era that preceded the Cambrian period, which supported more effective metabolism and enabled an increase in body size [30]. Large size, which included larger sensory organs (concentrated in the head) and muscle sheets along the body, led to improved sensory discrimination and speed of movement, which required new mechanisms of integrating sensory information, motor coordination and the central control of their relations. Increased size was correlated with an increased lifespan, which made learning by association worthwhile because events are likely to re-occur when one lives longer. Phyla with neural centralization evolved during the early Cambrian era, and the capacity for conditional associative learning in a subset of extant brain-possessing animal phyla suggests that it was at the early Cambrian that the first limited manifestations of conditional associative learning originated. Since even simple forms of conditional AL dramatically expand the capacity for learning during ontogeny, it enabled animals to exploit and construct new niches, promoted new types of interactions and arms races, led to adaptive responses that became fixed through genetic accommodation processes and drove the evolution of sense organs and of more complex motility. It gradually evolved into a more sophisticated form of associative learning that constitutes, as we argue in the next section, another major transition in cognition.

(c). The transitions to unlimited associative learning, to selection of imagined events and to symbol-based learning

The evolution of increasingly complex associative learning culminated in what we call UAL. The learning capacities that distinguish UAL from LAL at the behavioural levels are (i) the ability to discriminate among novel (non-reflexive and not previously learned) compound stimuli that differ in the configuration of the elements of which they are composed (within the same modality and from different modalities) and among different motor action patterns; (ii) the ability to learn cumulatively, through second-order conditioning, pointing to a flexible value system; (iii) escape from immediacy—the ability to learn about a stimulus even when there is a temporal gap between the CS and the US or the action pattern and the reinforcer, pointing to working memory. The generativity and the ability for cumulative and recursive learning led to a further leap in cognitive adaptability.

A survey of the learning literature suggests that these learning capacities are present in three phyla: in almost all vertebrates, some arthropods (including honeybees and cockroaches) and some cephalopod molluscs (the coleoid cephalopods: octopods, squid and cuttlefish) [5, ch. 8]. The underlying embodied cognitive architecture of these animals includes: appendages with many degrees of freedom of movement; the addition of association areas in the brain that integrate information within and between sensory modalities and motor controlling systems, enabling the mapping of the world and the body through integration of precepts and action programmes; dedicated memory circuits for the storage of compound precepts; dedicated and flexible value systems; regions dedicated to the integration of learned world and body maps enabling stable distinction between world and self. These brain circuits have been identified in fossilized vertebrates and arthropods that lived during the Cambrian, and UAL seems to have evolved in these phyla during this era. We have suggested that the dramatic learning ability of animals with UAL contributed to the Cambrian explosion [5,30]. Cephalopods with UAL capacity and the corresponding brain regions appeared 250 MY later.

As we argued in detail elsewhere [5], UAL can be considered as an evolutionary transition marker of sentience or minimal consciousness and the driver of its evolution. A transition marker is a single capacity that requires that an enabling system that manifests this capacity has a set of properties that jointly suffice for the characterization of this system as one that has completed the transition in question. The transition to a sentient mode of being is a major teleological transition, characterized by the possession of new value systems and goals [40]. The transition to UAL is, therefore, informational, intentional and teleological, and, we argue, contributed to the Cambrian ecological explosion.

UAL was the basis for the evolution of more complex types of cognition. It culminated in the evolution of what Dennett called Popperian organisms, animals that can select among imagined, alternative actions without having to try them out [13]. The ability to engage in selection among simulated representations requires that an animal remember what, where and when things happened (has event memory), and on the basis of the recalled event imagine alternatives (i.e. can represent permutations of experience offline) and plan actions that require the ability to assess how the planned action may affect itself (self-monitoring). This imaginative planning and self-monitoring, which Tomasello [41, p. 9] identifies with thinking, is a qualitatively new type of cognition—an elementary form of meta-cognition.

The transition to Popperian organisms was gradual and the ability to remember and transform mental representations that include self-monitoring is manifest, to different extents, in mammals and birds; it has more restricted manifestations in some fish, some arthropods and in the coleoid cephalopods. This type of cognition requires differentiated dedicated memory systems, such as a differentiated hippocampus in mammals, and additional levels of integration (of associative and executive areas) between different aspects of experience. It is well developed in higher apes and was a necessary condition for the evolution of the next cognitive transition, the transition to human, symbol-based cognition.

In its completed form, the transition to human cognition includes the ability to use a new system of representation and communication—symbolic language, which, like neural communication, uses a novel, general-purpose, communication currency. Language, as Dor suggested, is a collectively constructed technology for the instruction of imagination, enabling humans not only to imagine and select among representations, but to communicate about them, collectively construct them and culturally learn them [42]. Language enabled the construction of symbolic values (e.g. the good, the just, the beautiful), made possible human cumulative and open-ended cultural evolution through cultural learning [43] and altered the emotional profile of humans [44]. Like the transition to UAL, the symbol-based cognitive transition is informational, intentional and teleological, and has led to an ecological transition, to the era of the Anthropocene. The great growth in the human brain during the past 3 Myr and the sophistication of both domain-general and domain-specific brain areas underlie this transition. Current human cognition is beginning to be extended by using artificial learning systems. In the future, the interface between biological and artificial cognition may lead to another evolutionary transition in cognition.

3. Discussion

An evolutionary approach to cognition from a learning perspective provides a unifying view of cognition and is a platform for comparing very different cognitive systems, both neural and non-neural. During the twentieth century, rigorous molecular technologies that identified the building blocks of cognitive processes and that pointed to their ancient origins and remarkable conservation enabled the relations between sensory, motor, value and memory faculties to be studied and compared at the behavioural and molecular levels. At the conceptual level, new general frameworks for studying living systems from a cognitive perspective were developed [2,45]. Neural learning was conceptualized as an evolutionary process involving cumulative selection processes within the brain [16,17], and the evolutionary process itself was conceived as a learning process—learning theory can explain how past experiences can lead to an incremental evolutionary adaptation, which is used to direct future adaptive behaviours [46]. We can, therefore, look at the relation between learning and evolution from three perspectives: evolution as learning, learning as evolution and the evolution of learning.

In this paper, we focused on the evolution of neural learning. We classified major evolutionary learning transitions using informational criteria. However, as we noted, some informational transitions in cognition overlap with evolutionary transitions classified according to other criteria, so the transition-oriented approach provides a new way of understanding the relations between findings in cognitive science, neuroscience, ecology and the philosophy of mind.

Although a transition-oriented approach highlights the novelties that characterize a qualitatively new type of cognition, evolutionary analysis points to the origins of these novelties in earlier systems—to the continuities between the pre- and post-transition systems and to the novel recruitment of preexisting processes and factors. The evolutionary-transition approach can also address questions concerning the evolutionary significance and the primacy of different types of activities and sensory–motor couplings.

The relative evolutionary significance of exploratory-spontaneous activity versus reflexive responses to external stimuli (see [47]), and the relative importance of responses to external stimuli versus responses to internal and proprioceptive stimuli (see [48]) are important and debated questions. The perspective we present here suggests that these responses co-evolved: spontaneous activity is a basic property of all living organisms, as is the ability to respond to a changing world. There is little benefit in increasing the sophistication of motor movement if this does not lead to an increased ability to cope with the incessantly changing external conditions that a rapidly moving animal encounters. The ability to distinguish between self-generated and world-generated stimuli, which is necessary for movement and is the basis of the distinction between self and world, depends on close coupling between interoception, proprioception and exteroception. This coupling suggests that the different aspects of sensations coexisted and co-evolved in the first moving neural animals.

We suggested that the answer to a related question—the relation between conditional world-learning and self-learning in animals—is similar. Both world- and self-learning were present in the first associatively learning animals and evolved from habituation and sensitization mechanisms that modulated reflexive action and exploratory behaviour. The two types of learning are based on the same functional architecture, and in natural conditions are entwined because animals learn simultaneously about the world and about their own actions.

The continuity between the cognition of non-neural and neural organisms is the topic of several papers in this issue of Phil. Trans. B. In addition to the formal similarity among different types of learning, all the cognitive mechanisms and many of the molecular factors involved in perception, motor activity, memory and valence signals that have been identified in non-neural organisms are present in somewhat modified forms in neural animals. Gradations between different learning systems are inevitable, and it is not surprising that many variations in learning have been recognized (over 100 were distinguished by Moore [49]). Nevertheless, not every sophistication of learning can be described as an evolutionary transition. A transition, whether informational, teleological, intentional or ecological, must satisfy clear criteria.

As the papers in this issue show, a broad view of cognition has many ramifications. We believe that recognizing basal (epigenetic) learning [8] has additional research implications for the study of cognition. First, because epigenetic memory is part of the cognitive-learning tool kit of all living organisms, epigenetic learning in non-neural organisms from different taxa—bacteria, protists, slime moulds, plants and fungi, sponges and placozoa—needs to be characterized and compared at the functional and molecular levels. Second, a comparison of the ways in which epigenetic learning mechanisms are employed in the non-neural placozoans and sponges (where learning has not yet been studied), in cnidarians and ctenophores with non-centralized nervous systems, and in basal bilaterians with a simple brain can shed light on the changes in the epigenetic memory systems during the first two learning transitions. Third, the implications of a two-tiered memory can be studied at the computational level: since all neural organisms have both epigenetic-intracellular memory and synaptic inter-cellular memory, and these two systems store information for different time spans, the learning capacity of a two-tiered system may be increased [50]. Fourth, the possibility that transitions to sophisticated forms of cognition, such as the transitions to UAL, to Popperian animals and to symbol-based human cognition, involve new neurodynamics needs to be explored. Szathmáry and his co-workers [51] suggested that insight learning, which is within the capacities of imaginative Popperian organisms, requires ‘Darwinian neurodynamics'—the replication, multiplication and selection of newly formed neural patterns in the brain, a neural-evolutionary process that goes beyond the selective stabilization of synaptic variations suggested by Changeux & Danchin [16] and Edelman [17]. Another suggestion [5] is that long-term storage of multiple features of a stimulus or an event may involve the transmission of molecular memory-traces (proteins and small RNAs) by exosomes from neural circuits at low levels of organization to higher levels where they can be encoded as persistent chromatin modifications.

The evolutionary-transition approach to learning highlights both the continuities and discontinuities between different types of cognition. It provides a solid framework for integrating and relating all the capacities identified as characterizing cognition in living organisms, and can inform the study of future types of cognition that may emerge at the interface between living and artificial systems.

Data accessibility

This article has no additional data.

Authors' contributions

Both authors wrote the paper.

Competing interests

We declare we have no competing interests.

Funding

We received no funding for this study.