Valuing what happens: a biogenic approach to valence and (potentially) affect

Abstract

Valence is half of the pair of properties that constitute core affect, the foundation of emotion. But what is valence, and where is it found in the natural world? Currently, this question cannot be answered. The idea that emotion is the body's way of driving the organism to secure its survival, thriving and reproduction runs like a leitmotif from the pathfinding work of Antonio Damasio through four book-length neuroscientific accounts of emotion recently published by the field's leading practitioners. Yet while Damasio concluded 20 years ago that the homeostasis–affect linkage is rooted in unicellular life, no agreement exists about whether even non-human animals with brains experience emotions. Simple neural animals—those less brainy than bees, fruit flies and other charismatic invertebrates—are not even on the radar of contemporary affective research, to say nothing of aneural organisms. This near-sightedness has effectively denied the most productive method available for getting a grip on highly complex biological processes to a scientific domain whose importance for understanding biological decision-making cannot be underestimated. Valence arguably is the fulcrum around which the dance of life revolves. Without the ability to discriminate advantage from harm, life very quickly comes to an end. In this paper, we review the concept of valence, where it came from, the work it does in current leading theories of emotion, and some of the odd features revealed via experiment. We present a biologically grounded framework for investigating valence in any organism and sketch a preliminary pathway to a computational model.

This article is part of the theme issue ‘Basal cognition: conceptual tools and the view from the single cell’.

1. Introduction: the problem with affect

How a set of circumstances comes to be valued by human and non-human animals as advantageous, harmful or not worth bothering about remains poorly understood [1–4], particularly under natural conditions. Why is this? The desire to understand evaluative systems as a vital part of the human experience and to relieve suffering arising from their dysfunction has meant that modern affective research has concentrated largely on animals with complex, highly sophisticated brains, notably humans and their proxies. Some emotion researchers now agree (not always for the same reasons) that the human focus has enabled continued reliance on long-standing theoretical assumptions predominantly derived from first-person intuitions about human experience, to the detriment of the field [1,2,4,5]. Nevertheless, widespread consensus on three fundamental points suggests that it may be time for the sciences of emotion to expand the range of model systems if the aim is to better understand where value comes from and how valence is assessed in the living world. The first point relates to the state of the field itself. As per a recent dialogue in Current Biology:

After more than a century of scientific inquiry … emotions remain essentially contested concepts: scientists disagree on how they should be defined, on where to draw the boundaries for what counts as an emotion and what does not, on whether conscious experiences are central or epiphenomenal, and so on. Such disputes have sown great discord among scientists, leaving the field in perpetual upheaval, and without a unified framework for guiding scientific inquiry and accumulating knowledge.

[6, p. R1060]

The second point is that basic evaluative capacities (this is advantageous, this is harmful) are products of evolutionary selection [1,2,4,7] acting upon variation at multiple levels: genetic, epigenetic, behavioural and, in humans, symbolic behaviour [8]. The third point of agreement is that valence—defined here as the biological impetus of attraction or repulsion to a state of affairs based on an assessment of value relative to an individual's goal structure—necessarily informs an organism's decisions about what to do next [5,9–12]. This impetus need not be conscious [7,13,14].

The consensus of the past two decades holds that ‘core affect’, considered necessary but insufficient for emotion, is composed of valence plus arousal (physiological activation, intensity) [12,15], roughly conceived as two, independent bipolar dimensions (valence = positive/negative; arousal = high/low). While research into valence during that period increasingly admitted non-mammalian models, including insects [16], worms [17] and other invertebrates [18], to date aneural systems (e.g. plants, microbes and some animals) have not figured in any aspect of affective research. This is despite long-standing evidence that even bacteria exhibit valenced behaviour [19,20], and were known to do so long before the term valence was introduced into psychology [21–23].

The aim of this article is not to convince the reader that the building blocks of affect are present in prokaryotes or any particular phyletic lineage on the more basal branches of the tree of life for the simple reason that emotion researchers currently cannot agree about whether non-human animals—of any sort¹—have genuine affect [4]. We believe that the ubiquity and potency of stress and growth responses in prokaryotes [24], single-celled eukaryotes [25], plants [26], and aneural and simple neural animals [27] are proof of concept that valence coupled with high activation plays a major role in making a living at all levels of biological complexity whether these processes are called by their usual names or by caveated terms. Some of the behaviours induced by such responses are among the most complex found in aneural organisms, involving global coordination of changes in expression of (up to) hundreds of genes and resulting in profound changes in form, function and behaviour, all based on an assessment that environmental conditions have changed in ways that threaten or advantage survival.

Our aim, rather, is to provide a theoretical–conceptual framework for investigating valence in all biological systems. This is needed for two reasons. First, nothing currently exists. Research in several aneural organisms—notably social bacteria, acellular slime moulds, social amoeba and plants, to say nothing of simple multicellular animals without and with nervous systems—has advanced to a point where investigating valence is a logical next step. Second, emotion science arguably requires it. By concentrating solely on animals with brains, even simple brains, the affective sciences are denied the most successful strategy biology has to offer: start small and simple, understand principles and mechanisms, then scale up.

Bacterial ion channels provided insights into the operation of ion channels in neurons, and were the basis for a Nobel Prize,² long before their function in microbes was known. Only relatively recently a team of researchers in Gürol Süel's laboratory at the University of California (San Diego) discovered the function of ion channel-based electrical signalling in structured communities of Bacillus subtilis is similar to that in neurons: propagating information over long distances and large numbers of cells [28], in this case for the purposes of regulating metabolism between the centre and the periphery of the growing colony [29,30]. These surprising discoveries have led to more results. Transient exposure of biofilms to a complex pattern of light (in this case, a University of California at San Diego logo) was found to induce ‘persistent and robust’ changes in the membrane electrical potential of individual cells such that ‘complex memory patterns' (where memory is the retention of information about previous exposure) were encoded in the collective oscillation of the exposed cells in opposite polarity to unexposed cells [31]. According to Süel and coworkers, 'The ability to encode robust and persistent membrane-potential-based memory patterns … suggests a parallel between neurons and bacteria' [31, p. 417]. In short, behavioural phenomena (in this case, memory) in aneural organisms, and the mechanisms that implement them, potentially can tell us much about how these processes originated and differentiated in the course of evolution. There is every reason to suspect that studies of valence may yield similar insights. If this suspicion is correct, we believe it could revolutionize our understanding of affect and its evolution.

The paper is organized as follows. Section 2 sets out the historical context in which valence was introduced as a theoretical construct into psychology, whence it migrated into the sciences of animal behaviour and biology. Section 3 describes how a scientific consensus formed around the concept of core affect, while section 4 shows how the valence concept figures in four emotion theories, three proposed relatively recently. Section 5 introduces evidence that valence is probably not (as commonly conceived) a single bipolar dimension of value, which led to the finding that consciousness is unnecessary for its induction. In section 6, we move into alternative theoretical territory with the biological basis for valence. Section 7 provides a case study of valence via pH sensing in B. subtilis and entertains some objections to ascribing valence to aneural organisms. Finally, given the critical role of valence in learning, section 8 proposes a computational approach to valence via a preliminary model that draws on homeostatic reinforcement learning (HRL) and the free-energy principle (FEP) but aims to fill gaps in these approaches.

2. Valence: birth of a concept

The value to an organism of a state of affairs and the type of response it sets in train have been conceived since the late nineteenth century in terms of opposites, usually along a single dimension [32]: positive/negative, pleasant/unpleasant, pleasure/discomfort, approach/avoidance, reward/punishment, appetitive/aversive, activation/inhibition [11,33,34] and (our preference) advantage/harm. The stimulus may be an object, an object in a particular context or a context featuring a variety of salient stimuli.

Valence was formally introduced into Anglophone psychology in 1935 by the German-American social psychologist Kurt Lewin [35]. Bonding in valence theory, then transforming atomic physics and chemistry,³ required an electron pair, each member of which had an opposite direction of ‘angular momentum’ (spin), attractive/positive or repulsive/negative. As its Latin root (valentia) implies, valence has power [36]. Lewin believed that borrowing theoretical constructs from the physical sciences captured the intrinsic dynamism of psychological phenomena. Other loan terms included ‘field’, ‘tension’, ‘system’, ‘energy’ and ‘free energy’ (energy available for work).⁴ From the beginning, then, valence—unlike ‘mind’, ‘memory’, ‘learning’, ‘emotion’ or ‘motivation’—was a scientific concept, not one borrowed from (and carrying the freight of) ordinary usage.

Lewin saw psychological valence as a kind of force exerted on an individual that drives (steers) an expenditure of energy in the form of behaviour. Undefined, valence was illustrated by example: the ‘attractive’ motivating force chocolate exerts on a child, attainment of which is experienced as a pleasurable reward; the ‘repulsive’ force of a mathematical task to which a child has an aversion is experienced as a kind of punishment [35]. Despite using bipolar adjectives, Lewin emphasized that valence is neither static nor general: the same stimulus may have a different valence for an individual at different times. The ‘psychical field’ may also include many variables with different valences, some of which may conflict. Moreover, each encounter changes elements of both ‘inner and outer environment[s]’, which have the potential to change subsequent responses.

3. Core affect: the ground floor

Widespread acceptance and use of valence as a psychological concept took several decades. When it began to appear regularly in affective theorizing, in the 1980s⁵ and thereafter, valence had been more or less concretized. Where Lewin saw valence in terms of changing, context-dependent processes resulting from an individual's continual dynamic interaction with an environment, by the late twentieth century consensus had formed around the construct of valence as a single, reactive, bipolar dimension with attraction/pleasure at one end and repulsion/unpleasantness at the other. With arousal, valence formed one half of core affect. As proposed by James Russell in 2003, core affect comprises ‘the simplest raw (nonreflective) feelings evident in moods and emotions' accessible to consciousness [12]. These could be plotted along horizontal and vertical axes: valence (pleasant to unpleasant; hedonic response) and arousal (sleep to frenetic excitement; energetic activation). According to Russell, core affect is: (i) ‘primitive, universal, and simple (irreducible …)’; (ii) like temperature, can exist ‘without being labelled, interpreted, or attributed to any cause’; (iii) also like temperature, may seem simple but its biological realization may be quite complex, and (iv) always present [12, p. 148]. Now part of the furniture of emotion research,⁶ core affect has been advanced as a foundation for investigating affect in non-human animals [37].

In a series of eight experiments performed in the mid-2000s, neuroscientist Lisa Feldman Barrett set out to test the valence concept for its capacity to support a scientific study of emotion [11]. The findings convinced her that valence is basic, is invariably present, is capable of being observed or measured in multiple modalities, and results from ‘a process of valuation’ that identifies the perceiver's relationship to the environment (or elements within it) in terms of advantage or harm. Relationship ‘to the flow of changing events', according to Barrett, is registered in ‘the constant stream of transient alterations in an organism's neurophysiological state’, and the potential outcomes of action calculated relative to survival or other immediate goals. Valence, she concluded, ‘constitutes the most basic building block of emotional life’, the ‘core’ of core affect [11, p. 39]. The function of core affect is to affect the core, the homeostatic core of the individual organism, the basic physiological functioning that supports the individual's existence.

4. Valence in emotion theories

In this section, the role of valence in four recently articulated emotion theories is briefly discussed with special attention to remarks or proposals by different researchers that could be construed as potentially applicable to a wider diversity of phyla. Although prima facie these theories appear to be more widely applicable, believing that they are would be a mistake.

(a). The somatic marker hypothesis (Damasio)

At a time when disembodied reason was the heart of cognitive science and emotions figured nowhere, neuroscientist Antonio Damasio proposed that reason and emotion were, at least, equal partners in human decision-making. Moreover, he suggested that, as messengers of the body and existential imperatives (somatic markers), emotions might well be more influential than then could be imagined [9]. By the end of the twentieth century, Damasio's account linking homeostasis to emotion and the eventual evolution of consciousness had taken shape [10]. While Damasio was concerned predominantly with ‘this primordial story—the story of an object [the brain] causally changing the state of the body’ [10, p. 30], he nevertheless acknowledged the story began when vastly smaller objects were causally changing the state of their own bodies, long before nervous systems.

[S]ensing environmental conditions, holding know-how in dispositions, and acting on the basis of those dispositions were already present in single-cell creatures before they were part of any multicellular organisms, let alone multicellular organisms with brains.

[10, p. 139].

However, nowhere does Damasio attribute valence and/or core affect to very simple neural organisms, much less aneural organisms. Neither does any emotion theorist we cite here. In Damasio's case, the consensus on core affect did not form until after his first two books were written. Valence per se does not figure in these early articulations of the somatic marker hypothesis (if indices are indicators). For others, a major obstacle is likely the centrality to theories of emotion, past and present, of ‘feelings’ and subjective experience, traditionally grounded in human phenomenology and which present (to date, insurmountable) epistemological challenges even in organisms with nervous systems. A discussion of the nature of feeling and subjective experience is beyond the scope of this article. Suffice to say the long-standing philosophical ‘problem of other minds'—the inability to know for certain how another being thinks or feels in the absence of language (and even then)—applies in spades when it comes to aneural organisms.⁷

(b). Theory of constructed emotions (Barrett)

According to Barrett's dynamic, biologically based theory, the brain is continually engaged in computing four broad categories of information: (i) the status of the organism's current physiological state and immediate future needs, given internal (interoceptive) and external (exteroceptive) sensory inputs; (ii) making predictions or drawing inferences (including based on past experience) about what is causing these sensory inputs; (iii) how circumstances might be trending, and (iv) generating signals about what to do next (metabolically, behaviourally) in order to keep the organism functioning [38]. These signals—constructed on the fly relative to the state of the individual in a specific context and against a background of goals of varying importance and urgency—are experienced as emotions, at least (uncontroversially) in humans [38].

As per the Principles of neural design [39], which Barrett cites, the brain's main function is regulation of the internal milieu in all its facets, which includes managing the deleterious effects of continual change required by such regulation (allostasis) while generating the behaviour necessary for survival, growth and reproduction. As a valuation of potential advantage or harm to the organism, and a motivator of behaviour, valence is an intrinsic feature of these complex, ongoing, regulatory computations and their output. As Barrett puts it, the elements of core affect are ‘simple summaries’ of your ‘body budget’: ‘Are you flush? Are you overdrawn? Do you need a deposit, and if so, how desperately?’ [2, p. 73] These summaries depend upon an ‘internal model’ of body-plus-affective-niche that manages allostasis [38]. Having an internal model apparently is not exclusive to animals with brains, or even to animals, but, rather, is found generally throughout the living world, even as the content of each individual organism's internal model is species-specific. ‘Even single-celled organisms that lack a brain learn, remember, make predictions, and forage in service to allostasis’, she writes [38, p. 6].

(c). Functional approach to emotions (Adolphs & Anderson)

In contrast to Barrett, Adolphs & Anderson are explicitly interested in promoting affective research across animal species, including in insects and other invertebrates [1]. Rather than advancing a theory, they sketch a biological framework for making sense of empirical findings in affective research without recourse to widely differing ‘intuitive’ and ‘common-sense’ beliefs about emotion, which (they argue) have dominated the field and stunted knowledge production. The framework they advance in their recent ‘new synthesis’ of the neuroscience of emotion is based on the function affective states have in the existential (especially social) economy of humans and other animals. It comprises a set of properties they call ‘emotion primitives’ common to emotion states (see box 1).

Box 1. Emotion properties [1, p. 66]*.

(1) Valence, hedonic quality;

(2) Scalability, a version of arousal referring to gradations in intensity;

(3) Persistence of the affective state after initial detection of a stimulus;

(4) Generalization, the ability to confer value in a new context owing to memory and learning;

(5) Global coordination, the tendency of emotion states to induce widespread changes in the whole organism;

(6) Automaticity, the fact that emotion states exist somewhere between reflexes and deliberate actions, and their potency typically trumps deliberate acts; and

(7) Social communication, the fact that emotion states, or elements of their expression (e.g. facial and/or other bodily movements), seem to be ‘pre-adapted to serve as social communicative signals'.

* The content of this box is adapted entirely from this source.

Adolphs & Anderson note that ‘general agreement’ exists in the field that motivation and arousal can be studied in animals, but not emotion [1]. Another factor may be the belief that valence has important innate aspects, even in humans, which may be difficult to disambiguate in other species. Adolphs & Anderson observe:

Without some such innate basis, there would be nothing to ground valence, nothing upon which learned associations could build. Those innate representations of valence, in turn, would not have been selected in evolution if they did not afford the species a survival advantage.

[1, p. 230]

What Adolphs & Anderson mean by ‘innate’ in this context is not entirely clear—the ‘hard-wired’/acquired distinction seems likely—but it should be noted (with the greatest respect) that the innateness concept is problematic. Over the past 20 years, conceptual excavations of innateness—including repeatedly by behavioural ecologist Patrick Bateson—have unearthed at least six different meanings, which connect to the word's original common-sense meaning (from birth), if not in easily calibrated ways, are often confused, can be phrased more precisely in technical terms and have resulted in calls (so far unsuccessful) for its elimination from scientific discourse entirely [40–42]. More recently, scientists have been surveyed on what they think the term means, which (it turns out) is not all that different from what lay people think [43]. The concept of innateness is now considered by those who have grappled with it to be generally unsafe for drawing valid scientific inferences [44,45]. This is important for considering behaviour in aneural organisms because innateness is a concept easily reached for, despite being next to impossible to clearly delineate relative to novel behaviour in any organism (including humans), and rarely figures in the domain of the relevant behaviour, that is, molecular biology (see [46] for an excellent historical–philosophical summary of the innate/acquired distinction).

(d). Survival circuit concept (LeDoux)

An important underlying issue in emotion research appears to be how to distinguish what occurs in humans (conscious feeling) from the rest of the animal kingdom. Joseph LeDoux's ‘survival circuits’ concept aims to integrate ‘ideas about emotion, motivation, reinforcement, and arousal in the effort to understand how organisms survive and thrive by detecting and responding to challenges and opportunities in daily life’ [4, p. 653], without recourse to the emotion-related concepts quoted. Survival circuits are described as ‘devices’ that integrate sensorimotor input for ‘specific adaptive purposes’ and include ‘at a minimum, circuits involved in defence, maintenance of energy and nutritional supplies, fluid balance, thermoregulation, and reproduction’ [4, p. 655].

Despite the generality of the label and their likely origin in ‘primordial mechanisms' present in early life, survival circuits are not present in all living systems, according to LeDoux; they exist only in animals with nervous systems [4]. The rationale for this exclusivity is not explained or justified, although the need for complexity is stressed. So it is, not only with all of these theories of emotion, none of which questions core affect as fundamental to the constitution of emotion, but also in the psychological literature on valence in general. A nervous system is presumed necessary but is rarely (never?) argued for. However, if valence is, as Lewin suggested, a ‘force’ (impetus) within an individual that steers behaviour towards perceived advantage and away from perceived harm [35], then surely the generality of this property in the biological world is an open empirical question. Open empirical questions demand investigation.

5. Valence: asymmetrical and nonconscious

It was long thought that positive and negative valence existed as two poles of a single dimension of roughly equal potency. This is no longer the case [47]. Damasio flagged this two decades ago. Pain and pleasure, he wrote, ‘are different and asymmetric physiological states, which underlie different perceptual qualities destined to help with the solution of very different problems’ [10, p. 77]. Positive and negative valence are now known to be far from equal, and recent evidence suggests they are processed differently [48]. Negative valence almost always packs more punch than positive valence, a phenomenon with uncomfortable implications demonstrated comprehensively nearly 20 years ago [49], questioned repeatedly since, but which research continues to affirm [50–53].

There are robust asymmetries in the processing of positive and negative information at virtually all levels of human information processing … . Negative information draws more attention, leads to stronger neurological reactions, and is recognized more accurately. Most of these asymmetries can be summarized under the observation ‘bad is stronger than good’, meaning that negative information has a stronger psychological impact than positive information. [48, p. 69]

Given the default goal of survival, the perception of harm may have greater potency than the perception of advantage in the living world generally. A classic example in mice and rats (which is to say nothing about its experimental value, which is increasingly questioned [54]) is the open-field test, where the impetus to investigate a novel stimulus competes with the (evolutionarily canalized) anxiety of entering an area where predation from above is a predictable danger. In bacteria, stress responses far outnumber growth responses [24], and some of the most complex multicellular behaviour in prokaryotes is induced by negative changes in prevailing conditions [55]. Uncertainty is typically perceived as a stressor in humans and other animals [56], and unfamiliar stimuli initially tend to be rated more negatively [57].

Unpredictability, a variety of uncertainty, is also a stressor. Experiments with cichlid fish found that any change in early life in a conditioned feeding regime (positive or negative) affected memory and learning lifelong [58]. Compared with those exposed to an unchanging routine, young fish in the mildly unpredictable environment were more vigilant, had better memory and learned faster. One possible explanation is that what is unknown or unfamiliar may harm, even kill. It is also good to be reminded that strengthening cognitive traits, which is typically considered a good outcome, can arise from responding to mild stress.

We might expect, then, that what is familiar, because it is more certain, should generally be more positively valenced. Ample experimental evidence supports this conjecture. The mere exposure effect is a robust, well-established psychological phenomenon investigated in hundreds of experimental studies since its discovery in 1968 [59]. The effect has been demonstrated ‘across cultures [and] species' (in rats, chicks and prenatal chicks) as well as across ‘diverse stimulus domains’ [60, p. 244], including auditory stimuli and visual stimuli presented as ideographs, words (nonsense and meaningful), drawings, photographs, geometric figures, real objects and real persons [61]. The mere exposure effect is typically summarized as familiarity breeds liking. Subjects briefly exposed to a neutral stimulus subsequently rate the ‘familiar’ stimulus more positively than they rate other neutral stimuli to which they have not been exposed [61]. The mere exposure effect does not work with negative stimuli, and negative mood has been shown to inhibit the effect in human experiments [62].

Evidence suggests that the mere exposure effect also works in reverse: positivity signals familiarity. Results of experiments using three different paradigms converge on the idea that positively valenced stimuli tend to be regarded as more familiar, ‘perhaps because the experience of familiarity is typically positive’ [63, p. 585]. The familiar/positive linkage recently was found to generalize to similarity as well. Two positively valenced stimuli are judged to be ‘similar’, whereas two negatively valenced stimuli are not; they remain distinct [64].

One plausible reading of the asymmetricality of valence, as Damasio [10] suggested about pleasure and pain, is that positive valence and negative valence perform two very different functions in the existential economy of an organism. Positive valence, associated with pleasure and reward, may serve to reinforce behaviour induced to meet the needs and goals of the organism. Negative valence, by contrast, could be said to spur action, both towards what is needed—for example, the negative state of a deficit in some physiological variable, such as thirst or hunger—as well as away from harm.

An important feature of these valence experiments is the finding that the effects can manifest when the exposed stimulus is presented below the threshold of consciousness. Exposure experiments involving subliminal stimuli were among the first to provide evidence that affective and/or cognitive processing may take place ‘without conscious awareness' [61, p. 281] but still influence conscious processes, an idea increasingly contested in human subjects as a result of the reliability/reproducibility crisis in psychology [65]. Since at least 2007 experiments have raised and continue to raise doubts about whether the familiarity effect is possible without conscious recognition [66,67]. Nevertheless, in 2017, the first meta-analysis of experiments in the quarter-century since a seminal review [61] concluded that more recent findings where subliminal effects were tested were ‘consistent with the proposition that conscious awareness is unnecessary for operation of the mere exposure effect’ [67, p. 468]. Supporting this result are findings of experiments performed subsequently, showing that subliminally presented neutral stimuli affect implicit (but not explicit) attitudes [59]. Similarly, negatively valenced stimuli presented subliminally to healthy subjects recently have been shown to induce a variety of adverse physiological changes [68].

In summary, evidence in humans and other animals suggests that consciousness is unnecessary for detecting, processing and acting on valenced stimuli, or for an assessment of value to have subsequent effects on physiological state, cognitive processing and/or behaviour. In our view, it would be pointless to propose that valence be investigated in aneural organisms were this not the case.

6. A biogenic approach to valence

The study of emotion, like the study of cognition more generally, has tended to keep one eye on the human case, often relying on common-sense intuitions about the nature of the phenomenon [69]. Such an approach is often called anthropocentric (human-centred), which is an accurate description when the subject of study and/or the basis for comparison are explicitly human psychological capacities. We prefer a different term-of-art: anthropogenic (human-generated) [70], which specifies an approach grounded in human phenomenology, what we infer about psychological phenomena based on our first-person experience. In both anthropocentric and anthropogenic approaches to psychological functions and properties, the human case is not simply one model system; it is the paradigm. A biogenic approach, by contrast, starts from the principles of biology and develops an account of psychological phenomenon to include Homo sapiens as a (typically) highly sophisticated and complex example, but not an exemplar or paradigm [70]. This distinction has a tendency to seem intuitive, which we take to be a good sign, but has existed only since 2004 [71].

The distinction grew out of a cluster of principles determined for each of the two approaches derived from a wide variety of sources [72]. The biogenic principles (box 2) were derived from analysis of the leading accounts of biological organization in terms of thermodynamics [73–75], self-organization [76,77], autopoiesis [78,79], irreversible and dissipative structures [80], semiotics/information-dependence [81,82], autonomy [83,84], interactivism [85] and anticipatory complex systems [86–89]. As they do in the anthropogenic approach, the biogenic principles facilitate and constrain explanation in the domain of investigation—cognition in the first instance, valence in this case—by providing a well-accepted foundation for explanation and a set of boundary conditions that must be met, or at least not violated. The cognitive toolkit described in table 1 of the introductory essay to this theme issue [90], which includes valence, was derived in part from these premises.

Box 2. Biogenic approach: principles (adapted from [70]).

Evolution	Organisms are products of evolution. A degree of similarity in functions and mechanisms is to be expected, either through conservation within lineages or convergence in distant phyla by virtue of meeting similar ecological challenges.
Thermodynamic openness	Organisms maintain themselves far from thermodynamic equilibrium by importing ‘order’ from their surroundings in the form of matter and energy, chemically transforming it to do work and exporting ‘disorder’ in the form of waste products of various sorts.
Future orientation	In addition to being thermodynamically open, biological systems are structures that actively dissipate entropy through the production of order. Although past events affect their adaptive behaviour, organisms are intrinsically oriented towards what happens next.
Interaction	Organisms must establish causal relations with features of their surroundings that lead to exchanges of matter and energy, which are essential to the organism's persistence in the first instance.
Autopoiesis	Organisms are continually being produced by a network of components, which are themselves being continually produced by networks of components. Simultaneously, the organism as a whole (including its constituents) is interacting with a surrounding medium.
Homeostasis/allostasis	Organisms are constituted by a wide variety of control and regulatory mechanisms, including multiple kinds of feedback mechanism, which maintain the stability of the system and buffer it against the effects of more or less constant internal change.
Functional norms	Homeostatic and allostatic processes operate within a range of values outside of which the organism's persistence, wellbeing or ability to reproduce are threatened.
Functional linkage	Functions critical to an organism's persistence, wellbeing or reproduction are linked, directly or indirectly, strongly or weakly (e.g. affect and metabolism).
Adaptive behaviour	To persist, grow, thrive or reproduce, an organism must continually adapt to its surrounding medium by altering its internal structure and/or its interactive relation to features of that medium.
Information-dependence	Adaptive behaviour is dependent upon information. A state of affairs that stimulates an organism to adaptive behaviour (i.e. alteration of its internal structure and/or its interactive relation to environmental features) conveys information for that organism.
Selectivity	An organism is capable of interacting profitably with some, but not all, features of its environment as a result of its evolutionary and individual interactive history. Not every state of affairs is information for that organism.
Operational closure	Organisms are operationally closed as well as open to flows of matter and energy; the activities that produce and maintain an organism take place within a semi-permeable boundary, which is the basis of its autonomy.

Collectively and severally these principles yield valence fairly straightforwardly as a necessary facet of biological existence. To maintain autopoiesis and their far-from-equilibrium state as entropy-dissipating structures, biological systems must evolve means of interaction with elements of their surrounding milieu that will facilitate productive exchanges of matter and energy. Not all elements of the surrounding milieu are advantageous to the goal of continued autopoiesis, and some are harmful. Therefore, an autopoietic system must evolve means of evaluating both its current functioning and the nature of surrounding conditions in terms of advantage and harm relative (at a minimum) to the existential goals of survival, thriving and reproduction. Internal monitoring is necessary to determine what the system needs. External monitoring is necessary to determine whether the requisite resources are available and whether other threats to continued autopoiesis loom. The state of each relative to the other (internal milieu to external milieu) will determine the system's ‘best guess’ about what to do next, in terms of adaptive behaviour.

On this account, then, valence is the impetus of attraction or repulsion to a particular state of affairs based on an assessment of value as advantageous or harmful.

Valence thus arises out of a multi-faceted matrix of physiological demand and goal structure—generated by the conditions of existence specific to the organism's phenotype and developmental history (which could be called motivation, or drive)—that steers adaptive behaviour. This impetus may be manifested covertly (as metabolic change) or overtly (as behaviour), or both.

The prediction is that all organisms, with the possible exception of endosymbionts, will be capable of doing this owing to the demands of autonomous biological existence. Disambiguating specifically human behaviour in these terms can be messy, however, owing to the strong influence of culture and conditioning. This is why it is important to work out the biological basics of valence in simpler organisms. The elaborations of behaviour made possible with more complex brains, and the value assigned to that behaviour both by the agent and by the social group of which she/he is a part, will come into much sharper focus.

7. Valence case study: pH sensing in Bacillus subtilis

The potential for hydrogen bonding (pH) of an environment, internal or external, is vitally important to the continued viability of all cells, including single cells that constitute individuals [91]. pH homeostasis affects gene expression, ATP synthesis, cell motility, apoptosis and the structure, stability and interactions of biological macromolecules [92]. As an existentially critical parameter, pH sensing is clearly valenced in our biogenic approach. Advantage and harm are clear. At the base of the evolutionary tree of life, however, pH sensing has been verified in only four bacterial species and is poorly understood. However, in all of those species, pH sensing employs a chemotactic pathway, which links information from the external environment to cell movement, what in animals is called sensorimotor behaviour.

pH sensing in B. subtilis, long an important species for studying cell division, multicellular behaviour and more recently electrical signalling, has only just been established via an ingenious series of experiments designed and conducted by Christopher Rao and colleagues at the University of Illinois [93].⁸ The team not only demonstrated that pH sensing exists in this well-studied microbe and is genuinely chemotactic as commonly understood—chemical sites on the chemoreceptors can be modified to modulate their sensitivity to relevant stimuli—but also identified the four proteins involved in chemoreception, the genes expressing the proteins and the eight amino acid residues receptive to modification on the key protein in each two-protein pair.

Bacillus subtilis, a soil-dwelling microbe that can live inside the guts of ruminants and humans, generally prefers a neutral environment. Both acid and alkaline environments are ultimately detrimental to cellular functioning. Advantage, therefore, is somewhere in the middle between two harmful states. Although the pH spectrum is usually represented as a single dimension, the researchers found that the microbe uses two different if similar mechanisms to sense base conditions and acid conditions, either of which can be harmful. (figure 1). Both two-protein mechanisms include canonical membrane-spanning chemoreceptors (TlpA and TlpB) [94], which the Rao team's subsequent bioinformatic analysis showed are present in many different types of bacteria [93]. In B. subtilis, neutral pH (advantage) is accessed and harm avoided by movement induced by competitive responses between the two mechanisms, each of which detects potentially deteriorating conditions in terms of hydrogen ion concentrations.

Figure 1.

(a) pH range of ordinary household items. (This range does not reflect bacterial sensing range, which is unknown.) Source: Wikimedia Commons: Openstax College. (b) pH sensing in Bacillus subtilis, with information from Tohidifar et al. [93].

There is no ‘attractant’ for neutral pH. Rather, if pH is low (acidic), base-sensing increases and acid-sensing decreases. If pH is high (alkaline), acid-sensing increases and base-sensing decreases. Both acid- and base-sensing are triggered by interference with hydrogen bonding on the leading partner in each two-protein pair. In Escherichia coli, which uses two different chemoreceptors to achieve the same kind of bidirectional movement to an intermediate optimum, this has been called a ‘push–pull mechanism’ [95]. Although Rao's team were baffled by the complexity of the B. subtilis pH sensing system (why two mechanisms? why so many modifiable amino acid residues?), the finding is consistent with the organism's role as a tractable model of bacterial complexity.

At this point the question may arise: Why call this valence? Is this not simply an example of homeostatic processes?

On one hand, maintaining acid–base balance is a textbook example of homeostasis, an organism's regulation of its chemical composition via adjustments in response to changes in its external and internal environments ‘so that physiological processes can proceed at optimum rates' [96, pp. 295–296]. On the other, bacteria do not have the ‘luxury’ of homeostasis to buffer their internal workings against changing environmental conditions, as do constitutively multicellular organisms, according to microbiologist Richard Losick [97]. If this sounds not quite right, consider the difference in scale: the width of a human red blood cell is five times the length of a single rod-shaped B. subtilis cell, and a single red blood cell by itself does not and cannot figure in the homeostatic regulation of the organism of which it is a part. Even if Losick is wrong, the line between adaptive behaviour and homeostasis is very thin indeed in bacteria.

Instead of homeostasis, Losick writes, bacteria ‘have evolved mechanisms for adapting to change by being versatile’ [97, p. 1146]. These adaptations may be relative subtle, involving a handful of genes, or large-scale, involving hundreds. They may be direct responses to environmental cues or stochastic mechanisms for hedging behavioural bets in the face of future uncertainty. While B. subtilis shares behavioural responses with many other bacteria, the firmicute has an ‘unusually rich repertoire of alternative states …, enabling it to cope with a wide range of environmental challenges' [97, p. 1146].

What does Losick mean by ‘an unusually rich repertoire of alternative states'?⁹ First, B. subtilis has multiple forms of motility: (i) swimming, via multiple flagella distributed evenly over the cell, for autonomous foraging; (ii) chaining, where cells suddenly switch phenotype, stop moving, stop separating from one another after cell division and exude a sticky exopolysaccharide matrix that enables them to adhere to surfaces to begin biofilm formation if (and only if) environmental conditions are conducive; (iii) swarming, a ‘team effort’ in which clusters of cells form ‘rafts’, hyperflagellate to increase motive force and secrete a surfactant (surfactin) to reduce surface tension, which facilitates their spread into nutrient-rich territory; and (iv) sliding, flagella-independent movement arising from cell growth and biofilm formation [97].

Of these different types of motility, swimming is said to have memory,¹⁰ because, on average, cells remain in the motile state for up to 80 generations. Entering into chaining is stochastic and ‘memoryless’—any individual cell has a constant probability of switching into this state—but, once chained, a cell and its progeny remain in that state for up to eight generations. The chained state thus is said to have memory, even if entry into that state does not. The chained state is controlled by a feedback loop characterized by molecular products that repress a pair of genes but is not self-sustaining. The concentration of repressor present in each cell halves with each cell division, ultimately falling below the threshold for maintaining the feedback loop. How long the chained state endures is relatively insensitive to initial amounts of repressor, and so is tightly timed [97]. If environmental conditions are not appropriate (that is, nutrient-limiting), biofilm formation does not begin.

Bacillus subtilis has two quorum-sensing (QS) systems for coordinating collective behaviour at high population density and depending on environmental conditions [97]. QS is involved in a wide variety of collective B. subtilis behaviours that involve developmental changes in cell form and function under different environmental conditions. These include: (i) the induction of genetic ‘competence’, the ability to take up DNA from the environment (including for DNA repair); (ii) conjugation, the ability to engage in direct exchange of genetic material with conspecifics (sex); (iii) the ability to secrete toxins for killing cells that do not carry genes for the anti-toxin (sometimes, but not always, non-conspecifics); (iv) swarming (in nutrient-rich conditions); (v) biofilm formation (in nutrient-limiting conditions); (vi) electrical signalling within and between biofilms (under metabolic stress); and (vii) sporulation (approaching starvation conditions) [98], the most dramatic of all B. subtilis behaviours.

Recent experiments found that some B. subtilis cells secrete QS molecules at low cell densities, in a manner that does not appear to be connected to the direct or indirect regulation of (imminent) collective behaviour. Rather, this form of secretion was found to accentuate the individual cell's own responsiveness to QS molecules on subsequent exposure, which increased antibiotic resistance and persistence [99]. QS molecule secretion under these conditions was characterized (perhaps misleadingly) as ‘self-sensing’, because its sole effect is to alter the secreting cell's own sensitivity [99]; it does not appear to be part of the regulation of collective endeavour, as currently understood.

An important point to make about B. subtilis motility, biofilm formation and sporulation is that it involves phenotypic heterogeneity, a division of labour. A population of cells, almost genetically identical, differentiates into distinct types with different functions based on the same genomic resources under the same environmental conditions [99]. For example, B. subtilis biofilms consist of three cell types in roughly three layers: (i) sliding cells (bottom), which depend on surfactin-producing cells to lubricate the interstices between cells and between cells and surfaces; (ii) matrix-producing cells (middle), which keep the colony together and enable changes in cell alignment and configuration; and (iii) endospore-forming cells (top), capable of a special type of compartmentalized cell division [97]. The interaction of surfactin-producing cells and matrix-producing cells enables an entire B. subtilis colony to migrate to new territory [100]. As Losick notes, ‘B. subtilis has evolved to exhibit strikingly different behaviours … largely by exploiting the same set of genes’ [97, p. 1149].

Where does that leave us concerning valence-as-homeostasis?

Homeostasis involves internal physiological regulation. Homeostatic processes may spur sensorimotor and social behaviour, but in the cognitive sciences they are not generally equated with those things. A substantial proportion of the adaptive mechanisms in the B. subtilis repertoire are intrinsically involved in sensorimotor activity and coordinating collective behaviour, so they cannot be considered wholly homeostatic, even were that term appropriate at this scale. Additionally, homeostasis per se is not ordinarily equated with developmental plasticity, the capacity to generate different phenotypes from the same set of genetic resources under identical environmental conditions, yet this is precisely what B. subtilis does. All of the behaviour in B. subtilis described above depends on environmentally influenced epigenetic control of genetic resources for the production of proteins and nucleic acids that perform various functions via systems of mechanisms dependent upon feedback, positive and negative. However, the same is true for human beings, no matter what we like to think. What about learning? Long-term memory, a non-associative form of learning, is rarely studied in bacteria, but has been demonstrated in B. subtilis using an existentially critical stress response. Compared with naive cells, cells in which the response had been induced responded faster when exposed to similar conditions again, a classic indicator of memory in human beings [101].

Above all, the existence of such an impressive array of adaptive mechanisms in a basal organism raises a question: Where did they come from? Evolution, certainly, but evolution includes epigenetic as well as genetic inheritance, and epigenetic inheritance is influenced by behaviour and life history [8]. ‘Holding know-how in dispositions’ has to come from somewhere. Philosopher of science Karl Popper was convinced, at a time when epigenetic inheritance was still anathema, that it came from ‘problem solving’, countless generations of organisms in countless lineages probing their environments by trial-and-error, exploiting the good, evading the bad and finding alternatives when conditions are sub-optimal (as they so often are)—with countless dead along the way [102].

Problem-solving, on evolutionary time-scales, requires learning what works and what does not, which is how we propose to compute valence. One of the great advantages of contemporary approaches to machine learning, which will inform our approach, is that homeostasis is built into the models and is not antagonistic to the incorporation of valence as a driver of learning.

8. Computing valence

Research in recent years has shown that forms of learning apply to a variety of adaptive behaviours at the lower levels of biological complexity, not only in bacteria but also in tissue-integrated mammalian cells. Wherever it occurs, learning influences the organism's capacity to anticipate predictable changes in its environment and expand its behavioural armamentarium for responding to novel opportunities and stressors [103–111]. These modes include non-associative learning (e.g. habituation and sensitization) and associative learning (e.g. classical conditioning), and are implemented on various time-scales and levels of biological organization, from fast responses of aneural bioelectric and metabolic levels [29] to slower responses on the level of gene regulation [109]. Learning in these contexts involves not only changes in adaptation but also prediction of environmental stimuli, which goes well beyond traditional formulations of homeostasis. Such learning can be seen to evolve from and integrate homeostatic processes, but this is as much the case in animals with nervous systems as it is for aneural organisms.

The reason why learning and homeostasis proceed in lockstep, even as the two are differentially describable, is not hard to discern. Any type of learning crucially involves the formulation of goal states against which the outcomes of action can be evaluated. In the absence of an overriding goal (say, reproduction), homeostatic processes and their evolution over time default to the over-arching biological goals of system persistence and growth. In other words, an organism's adaptive response to environmental conditions consists of actions marshalled, in the first instance, on the basis of an assignment of value, and whose outcomes are assigned value relative to the achievement of certain goal states. Learning in living systems, therefore, is virtually impossible without valence.

In this section, we employ concepts of learning and homeostasis (as biological maintenance) on multiple scales to sketch a model that incorporates valence explicitly. In contrast to current models, this move enables the direct incorporation of stressors, which as we have seen are potent guides to biological action.

One of the most widely used computational approaches to learning, from simple organisms to human beings, is reinforcement learning (RL) [112,113]. Unfortunately for our purposes, RL is critically focused on the centrality of reward and does not easily accommodate stressors. In this paradigm, the effect of aversive perturbation on learning is mostly indirect through secondary effects on rewards and subsequent learning [114]. Stressors impinge not on the actors in the RL model, but on the meta-parameters of the model [115]. This is a critical weakness for computing valence.

Homeostatic reinforcement learning (HRL), a recent variant of the RL paradigm advanced by Keramati & Gutkin [116] and grounded explicitly in physiology, makes incorporation of stressors and aversive responses to negatively valenced stimuli easier. HRL recognizes that any computational approach to an agent adapting to and learning from an environment must take account of the fundamental need of living things to maintain physiological stability within an acceptable range along a wide variety of parameters. In this framework, reward-seeking is equated with maximizing physiological homeostasis by minimizing deviations in environmental feedback from an internally defined setpoint [116]. The reward value to an agent of external sensations (outcomes of actions taken) is defined in terms of drive reduction, the current difference between a homeostatic setpoint and where the organism aims to be. Stress is when the distance from the setpoint is large.

As Keramati & Gutkin point out, the drive function in HRL corresponds to the information-theoretic notion of ‘surprise’ in the free-energy principle (FEP), which is given by the negative log-probability of an organism being in a certain state. In his attempt to unify brain theories under the FEP, Friston describes how the physiology of biological systems can largely be reduced to homeostasis, where the organism's drive is to minimize the dispersion (or entropy) of its interoceptive and exteroceptive sensory states [117]. In the FEP framework, stressors are integrated into the learning paradigm via the well-established neuroscientific finding that an unpredictable stimulus is stressful, or negatively valenced [118–120]. This can be extrapolated to any organism on the basis that all biological processes and behaviour aim to reduce uncertainty (‘surprise’ in FEP terminology) in the pursuit of existential goals. In the active-inference framework Friston developed to apply the FEP to human behaviour, the quantity being minimized in the dynamical system of interest (e.g. functional brain states over time) is called variational free energy,¹¹ which puts an upper bound on uncertainty [121–123]. While Friston's theory was initially developed to understand human behaviour, it has recently been applied, in principle, to all forms of life [124–127].

Peters et al. [56] provide an information-theoretic definition of stress that stems from uncertainty in the FEP sense and includes the observation from earlier empirical work that uncertainty increases energy demand [128], an important, well-supported finding that gestures towards the metabolic cost of computing valence and mounting an appropriate response [129]. Unfortunately, neither HRL nor FEP, as currently articulated, can cope with this dimension. Computing valence, in our view, will require a syncretic, modified learning paradigm based on HRL and the FEP with the following four key modifications:

• (1) Addition of an integrative field, which corresponds to an organism's capacity to compute valence (the integration of rewards and stressors) in the context of the potential energetic cost to the system of a response option. The cost of action always involves trade-offs [39]. In B. subtilis, for example, the process of sporulation is highly energetically costly and is undertaken only in manifestly deteriorating conditions, but there are opportunities within the developmental sequence to delay commitment by all cells in a population (a form of bet-hedging) through the introduction of noise-to-signal transduction cascades [130].
• (2) Instead of value (and thence policy selection/action) being assigned strictly as a consequence of how rewarding an outcome is with respect to an expected reward—which we term advantage—the prediction error between expectation and outcome will be fed into an integrative field that tracks how much uncertainty is reduced over time in relation to an energetic value function representing metabolic cost.
• (3) In addition to advantage, the integrative field will receive not only stress inputs indirectly through uncertainty but also negatively valenced sensory inputs (perceptions of potential harm) directly from the environment.
• (4) Consistent with Barrett's notion of interoceptive and exteroceptive sensory inputs giving rise to predictions about their source which carry prescriptions for action that humans experience as emotions [38], interoceptive and exteroceptive signalling fields will be integrated with the energetic value function, with the valence product influencing action directly as a result of policy selection and implementation by genetic and epigenetic regulatory networks.

De Berker et al. [118] have implemented a simple but effective strategy of incorporating stress arising from uncertainty into a hierarchical Bayesian learning model. Each of the model's three layers has its own set of beliefs and prediction errors through which learning rates are affected by uncertainties arising from the agent's beliefs, as well as environmental variability. The variances in the Gaussian distributions of predictions at each level represent the uncertainties corresponding to irreducible, estimation and volatility uncertainty, respectively. Building on the De Berker model, we can use a set of linear equations to fit these different uncertainties into stress responses. The coefficients in these equations will regulate the influence of each uncertainty level relative to stress and are determined by the energetic constraints of an organism associated with its actions and homeostatic state. Biologically, the interoceptive and exteroceptive signalling fields mentioned in the fourth point above represent the actual implementation of these energetic stressors and rewards through molecular signalling pathways, while computationally they allow for dynamic control of this type of learning regulation through selected actions.

Using the information-theoretic definition of surprise as the negative log-probability of finding an organism in a certain state, we are equipped to directly integrate the aforementioned uncertainties with their associated stressors and rewards into the learning outcomes and decision-making process (policy selection) as advantages and harm (see figure 2). To stay entirely in a hierarchical Bayesian learning model based on hidden Markov models, the homeostatic setpoint can be replaced with multivariate autonomic setpoints to implement homeostatic control [131].

Figure 2.

An extended HRL model integrating valence. Sensory inputs from the environment (exteroceptive) are evaluated against predictions about interoceptive and exteroceptive outcomes in an integrative field, which determines valence (advantage/harm) of incoming information. Internal state regulation further integrates these inputs by calculating allostatic load relative to meeting homeostatic setpoints and the metabolic cost of current and potential action. Based on the prediction errors resulting from this HRL-like learning scheme, together with valence and the reality of metabolic constraints, a policy for action is selected. Policy selection and resulting action are implemented by genetic and epigenetic regulatory networks. Action modifies the next round of exteroceptive sensory inputs the organism receives. The rounded rectangles represent higher-order functions (sensing, information integration, decision making, implementation, behaviour), while the ovals denote processes or products that feed into or arise from the higher-order functions.

In this way we can, in principle, construct a learning model based on HRL and FEP that introduces valence in terms of both environmentally sensed advantage and harm, based on external conditions, as well as internally sensed conditions in terms of metabolic cost and allostatic load. The next step in future work will be to develop this extended learning paradigm into a fully worked out computational model and apply it to selected organismal behaviour in a simple organism, for example, B. subtilis.

In summary, a valenced approach supplies us with the tools to formulate an agent's goal structure with respect to measurable physiological, as well as information-theoretic environmental inputs—stressors in particular. This set-up can be used to analyse and predict an adaptive response that would otherwise be obscured from a purely bottom-up mechanistic point of view, while not negating that agency itself is a manifestation of biological mechanisms to which we have access experimentally. This capacity to incorporate goal structure—which relies on valence to signal what works and what does not—is crucial to developing what Damasio called ‘holding know-how in dispositions' [10, p. 139]. We believe the addition of an assessment of value to the physiological and information-theoretic concepts of leading models of machine learning will provide a superior grip on understanding how this happens.

9. Conclusion

Valence arguably is the fulcrum around which the dance of life revolves. Without the ability to discriminate advantage from harm, and to induce behaviour to meet the opportunities and challenges presented by changing circumstances (external and internal), the complex, entropy-dissipating, autopoietic form of organization known as life very quickly would come to an end. In classical cognitive science, such abilities were often described as cognitive, a form of appraisal. The central role of core affect in such processes—which ineluctably produces a ‘feeling’ that can never be fully described but only experienced—places them squarely in the affective domain, like it or not. Even if creatures like us cannot seem to divorce the idea of such states from consciousness as we know it, experimental evidence tells us that (even in human beings) valence can be processed and affect subsequent behaviour without conscious awareness. This removes the highest hurdle to studying valence in aneural and simple neural organisms, and that potentially opens a new door. If the history of biology tells us nothing else, it is that we cannot begin to imagine where we might end up once we start looking closely at something that has not been looked at before.

Our principal aim here was to show that valence emerges from the conditions of life as a necessary feature of biological existence, to provide a simple bacterial illustration of a valenced behaviour in the rather more complex context in which it arises, and suggest a path to a computational model. That does not mean that aneural organisms (for example) display affect—that is for the relevant scientific community to decide based on empirical evidence. And we do need evidence from new and simpler model systems, which represent the only tried and true method for getting a firm grip on what matters to us most: Homo sapiens—what we share with the rest of the living world and how we differ. We cannot truly know the latter until we know the former. In biology, there are no archetypes of process and function, only examples from different model systems of greater or lesser clarity and explanatory force. We have no idea where the best examples eventually will come from in the study of valence. This paper, we hope, will be a useful step in the direction of finding them.

Data accessibility

This article does not contain any additional data.

Authors' contributions

P.L. conceived, researched and wrote the sections relating to valence. F.K. conceived, researched and wrote the sections relating to computational modelling. The manuscript as a whole is the result of our collaborative discussions.

Competing interests

We declare we have no competing interests.

Funding

We received no funding for this study.