Development weaves brains, bodies and environments into cognition

Abstract

Understanding how and why human cognition has the properties it does is one of science’s fundamental questions. Current thinking in Cognitive Science has delineated two candidate approaches that differ in how they address the question of the relationship between sensory-motor and cognitive processes. In this paper, we add to this discussion by arguing that this question is properly phrased as a developmental question and that ultimately to understand the properties of human cognition we must ask how does human cognition come to have these properties. We conclude that because development weaves brains, bodies and environments into cognition, cognition is inexorably linked to processes of perceiving and acting and inseparable from them.

Understanding how and why human cognition has the properties it does is one of the most compelling questions in all the sciences. The phenomena to be explained are vast and varied, including categorisation, language and communication, imitation, learning, tool use, the invention of advanced symbol systems such as mathematics, as well as art and architecture. What are the origins and the processes that give rise to a cognitive system that is principled, innovative and capable of abstract ideas? One relevant debate in answering this question concerns how cognitive and sensory-motor components relate to each other. One possibility, sometimes known as the cognitivist solution is that cognition receives information from the sensors and passes information to effectors but is fundamentally distinct and separate from sensori-motor processes (for discussion, see Barsalou, 1999, 2003b). It is this symbolic character that accounts for the varied phenomena of cognition and that explains the structure of the cognitive system (Fodor, 1975). The alternative possibility, sometimes known as embodied cognition (Wilson, 2002) is that there are no distinct computational principles for cognition versus perception and action and that, instead, cognitive processes emerge out of and are dynamically coupled to sensorimotor systems (O’Regan & Noe, 2001). In this view, cognitive products, although sometimes partially decoupled from the here-and-now (when we think in the future or about possible worlds), are realised in and through the same sensory-motor systems involved in action and perception (c.f. Barsalou, 1999, 2003a).

Despite many attempts in the experimental literature on adult cognition (for review see Leshinskaya & Caramazza, 2016; Wilson & Golonka, 2013; Zwaan, 2014) this debate has not been resolved. Although there are many experiments reporting momentary sensory-motor effects on higher cognition, there are also failed attempts, many criticisms of this work, and no agreement on what data would resolve the debate (see Zwaan, 2014, for recent review). The lack of resolution may be due to the question itself: An understanding of the origins of human cognition – and the role of sensory-motor processes in abstract thought – is a developmental question, and one that is not addressable solely through experiments with adults. Human cognition in all its forms emerges from complex patterns of activity that, in fundamentally important ways, depend on an individual’s developmental and experiential history. These experiences, in turn, depend on the developing individual’s behaviours and on how these behaviours are realised through the body (Byrge, Sporns, & Smith, 2014; Sheya & Smith, 2010b). We propose that it is how brains, bodies and behaviours are connected through development that gives rise to the properties of the adult cognitive system that have traditionally been ascribed to the symbolic character of cognition. That is, the vast and varied phenomena of cognition seem to be related to how end (or goal) states are promoted by current states, the capacity to sustain, store and retrieve previous experience, the ability to integrate experience to uncover abstract relational structure and the systematicity of knowledge. These properties are what the symbolic description of cognition captures. As opposed to assuming these properties or denying them, we ask how these properties might come to be realised in brains, bodies and behaviours in a physical, social world. This shift in perspective from a what property (or function) question to an origin question (comes to be) has also occurred in contemporary approaches to understanding brains. One core process essential to this understanding is how behaviour organises internal brain states and their dynamics. As a first step in under-standing how the properties of cognition might arise through development we review the current under-standing of brains as complex dynamic systems within a body-behaviour-environment.

Brain networks

The adult human brain is composed of many distinct regions, with different internal structure, different dynamics, and different responsivity to inputs. These brain regions are strongly associated with specific cognitive competencies (see Sporns, 2011). However, research over the last 20 years has shown that these different brain regions also cooperate with one another to yield systematic patterns of co-activation in different cognitive tasks (Sporns, 2011). These patterns of cooperation reveal two kinds of brain networks. Structural networks refer to the set of anatomical connections linking distinct cortical and subcortical brain regions. Functional networks refer to statistical dependencies among temporal patterns of neural activity that emerge in tasks but are also evident in task-free contexts (also called resting-state connectivity). For example, during reading, when left inferior occipitotemporal regions (linked with visual letter recognition) are active, temporally correlated evoked activity is also observed in left posterior superior temporal cortex (linked with comprehension) and in left inferior frontal gyrus (linked with pronunciation, see Dehaene et al., 2010). These regions thus form part of a “reading functional network” and jointly coordinate their activity during reading. Parts of this reading network are also involved in other functional networks, including spoken language production and on-line sentence processing (Dehaene et al., 2010). Detailed analyses of the statistical dependencies in neural activity across brain regions during task-free “resting-state” activity have revealed patterns of functional connectivity (for discussion, see Biswal et al., 2010; Smith et al., 2009). Thus, functional networks have enduring connectivity patterns even when not specifically engaged. The patterns of functional connectivity in these networks have been linked to memory, cognition, learning and to individual differences in performance (Fox, Snyder, Vincent, & Raichle, 2007; Honey et al., 2009; Kanai & Rees, 2011; Sadaghiani & Kleinschmidt, 2013; Zatorre, Fields, & Johansen-Berg, 2012), as well as, to patterns of structural connectivity (Smith et al., 2009).

What we have learned about these networks is directly relevant to how we conceptualise questions about abstract thought, embodiment, and development (Byrge et al., 2014). We propose 5 key understandings that connect properties of these networks to properties of cognition. In the sections that follow we elaborate on this connection. First, the role of connectivity goes beyond channelling specific information between functionally specialised brain regions. Instead, connectivity generates complex system-wide dynamics that enable local regions to participate across a broad range of tasks. Second, connectivity weaves a complex set of inter-relations among brain regions and seeming distinct competencies, connecting for example, hand-eye-coordination to visual recognition (James & Engelhardt, 2012; also see Raichle, 2010; Sporns, 2011, for reviews). Different tasks recruit different but overlapping assemblies of neural components, so that many different components are involved in any one behavior and one brain region may be involved in many kinds of behavior. This property of brain connectivity has been characterised in terms of degeneracy and pluripotentiality (for discussion, see Sporns, Tononi, & Edelman, 2000). Degeneracy refers to many-to-one structure-to-function relations. Pluripotentiality refers to a one-to-many structure to function relation, in which a single structure – a single gene or single brain region – can contribute to many different functions. Third, the role of external inputs goes beyond the triggering or activating of specific sub-routines of neural processing that are encapsulated in local regions (Byrge et al., 2014). Instead, inputs act as perturbations of ongoing activity whose widespread effects depend on how these inputs become integrated with the system’s current dynamic state (Destexhe, 2011; Fontanini & Katz, 2008). Fourth, connectivity interacts with experience so that the effects of experiences go beyond specific tasks and responses to include alterations in the spontaneous (resting state) activity across these networks that can potentially influence the response of the system in novel tasks (Byrge et al., 2014). Fifth, the cumulative history of perturbations as recorded in changing patterns of connectivity – in-the-moment and over progressively longer timescales (i.e. over developmental time) – defines the system’s changing capacity to both respond to input and to generate increasingly rich internal dynamics. These five components are the modern foundation for understanding embodiment and the role of sensory-motor processes in abstract thought (Sheya & Smith, 2010b; Smith & Sheya, 2010).

Brain-body-environment networks

Brain networks cannot be fully understood by studying the brain isolated from its outputs and its inputs, and their history (Sporns, 2011). The output is real time behaviour, but behaving is never just an output. All behaviour evokes neural activity that can change patterns of connectivity. For instance, when we hold a cup or read a book, different and potentially overlapping sets of neural regions – including motor, motor planning regions, vision, haptic processes – become functionally connected. This pattern of activity endures beyond the moment of co-activation as one moment biases the activity of the next promoting behaviours that evoke similar patterns of connectivity, and thus, become the source of enduring changes in patterns of functional and structural connectivity (see Byrge et al., 2014). A recent clear example of this change is provided by studies of young children learning to recognise letters (James, 2010). Localisation of specific functions (the components of larger networks) is a basic feature of the human brain and in literate adult letter recognition is localised in the left occipito-temporal sulcus. In a series of elegant studies with preschool children, James and Engelhardt (2012) showed that writing letters – and motor involvement – is central to the development of this specialised visual area. In these studies, patterns of brain activation in response to visually presented letters were examined prior to training in 4 and 5-year-old children. Letters did not evoke localised activation in visual cortex as is seen in adult letter processing. Children were then trained to recognise letters (to the same degree of accuracy) through a various assortment of tasks – purely visual, typing, tracing, writing – and then brain activation to visually presented letters was re-examined. Children who learned to recognise letters through writing letters show activation in motor and motor planning regions even when just looking at a letter. More critically, they showed the mature pattern of localised activation in the left fusiform gyrus. Children who learn to visually recognise letters through writing also show more generalised recognition of letters, including novel letters (James & Engelhardt, 2012; see also Freyd, 1983; Lake, Salakhutdinov, & Tenenbaum, 2015). Apparently visual specialisation for letter recognition in the brain and the formation of processes to recognise the abstract forms in letters, depends not just on visual experience, but on visual experience created through the developing child’s own behaviour (see generally Held & Hein, 1963).

These findings illustrate two important points for answering the developmental question about the role of sensory-motor processes in “abstract” thought. First, the evidence shows that activation of a larger functional network – beyond vision drives change in the visual system. What are the processes through which this happens? One is the evoked activity from behaviour itself. Evoked neural activity from performing even relatively brief tasks such as looking at images causes perturbations to intrinsic activity that last from minutes to hours (Betti et al., 2013; Harmelech, Preminger, Wertman, & Malach, 2013; Tambini, Ketz, & Davachi, 2010) these reverberations of an experience have been shown to be functionally relevant, predicting later memory for the seen images (Betti et al., 2013). Longer tasks produce longer perturbations (Tambini et al., 2010) and may also modulate structural topology via longer-lasting synaptic plasticity (Harmelech et al., 2013). The second point is that the visual information created by the activity itself, visual information that unfolds in time in a correlated way with motor planning and execution of those plans, input that is dynamically correlated to the brain activations that produce it, are potent forces that change both functional connectivity and, over the long term, structural connectivity in the brain (Gross & Blasius, 2008; Luo et al., 2012; see Byrge et al., 2014, for review).

These findings on letter recognition are about the origins of abstract ideas. Recent experiments with adults – training the recognition of novel symbols – as well as a computational model of the effects of this training (Freyd, 1983; Lake et al., 2015; Reeke & Edelman, 1984) have demonstrated how learning by generating the visual form may yield internal representations, not of the form itself but rather of the function that generates that form, that lead to a broad generalisation and to the ability to invent new forms that fit the principles. In sum, the origins of the visual processes through which we recognise the letter A or H is made through sensory-motor processes. But this does not mean that mature letter recognition changes when we hold a pen or imagine we are writing (or other simple effects of postural manipulations that have been taken by some as the marker of embodiment). Instead, the sensory-motor processes through which we learn letters are evident in the properties of letter recognition itself, in the abstractions that we have formed of what forms can possibly count as an A or H (see especially Freyd, 1983). In other words, functional and structural connectivity established through the coordinating of sensory-motor systems to write a letter in the presence of brain, body and contextual variability in a sense represent the general form of a letter and can be used to recognise it.

Thus, our approach contrasts with traditional approaches to cognition which suggest that the ability to respond to a letter appropriately despite variation in context and immediate appearance can be ascribed to the symbolic nature of an internal representation containing the essential characteristic of the letter. Instead, it appears that what is essential to recognising letters is sensory-motor activity adjusting to contextual and immediate variability, not a lack of sensitivity to this variability. Next, we consider this same developmental process but in a broader context that provides insight into how generally cognitive processes thought to be symbolic in character, like letter recognition, might arise developmentally from sensory-motor coordination.

The body, development and abstract ideas

Piaget (1952) described a pattern of activity – what he called a secondary circular reaction – that serves as a useful starting point for thinking about what all this means for cognitive development more generally. Piaget placed a rattle in a four-month-old infant’s hands. As the infant moved the rattle, it would both come into sight and also make a noise, arousing and agitating the infant and causing more body motions, and thus causing the rattle to move into and out of sight and to make more noise. Infants at this age have very little organised control over hand and eye movement. They cannot yet reach for a rattle and if given one, they do not necessarily shake it. But if the infant accidentally moves it, and sees and hears the consequences, the infant will become captured by the activity – moving and shaking, looking and listening – and incrementally through this repeated action gaining intentional control over the shaking of the rattle. Piaget thought this pattern of activity – an accidental action that leads to an interesting and arousing outcome and thus more activity and the re-experience of the outcome – to be foundational to development itself.

Circular reactions are perception-action loops that create opportunities for learning. In the case of the rattle, the repeated activity teaches how to control one’s body, which actions bring held objects into view, and how sights, sounds and actions correspond. Piaget believed this pattern of activity, involving multimodal perception–actions loops, to hold the key to understanding the origins of human mind. The core idea of a circular reaction and its driving force on development is now understandable in the dynamics of brain-body (behaviour)-environment networks. Holding and shaking the rattle couples different brain regions, creating a network, both in the generation of that behaviour as well as in the dynamically-linked sensory inputs created by its effects upon the world. The important point is this: Our behaviours have real time physical effects on the world and thus behaviour provides information about the world and about the processes that guide the behaviour.

Some of these physical effects of our behaviour, like the rattle shaking and consequent sights and sounds, are quite transient, lasting only as long as the arm shakes. But some of our effects on the world are longer lasting, like the form of the letter once it is written. These more stable products of our actions may be particularly important in the development of abstract ideas and innovative thoughts. Consider, as an example, a child who through a series of actions, stacks one thing onto the other, and then another, and then another. This stacking behaviour not only creates reverberations of activation in overlapping brain networks as the repetitive actions build the stack but in the end the child can also sit back and see and reflect on a tower of blocks that did not exist before. Stacking blocks creates a durable perceivable consequence of our own actions. In a way, this stable thing in the world – still there when we look away and look back – is like a symbol that by being linked to the processes that created it, compresses, or represents, the complex processes that gave rise to it.

The development of spatial classification provides an interesting phenomenon with which to consider these ideas. Between their first and third birthdays, children begin to use space to represent similarity, putting like things close together (Sugarman, 1983). Although this metaphor pervades our theories and mathematics about similarity in which we represent similar things as being close in some space, having represented the similarity between objects does not direct a child to place objects that are similar next to each other. The idea of placing objects that are similar next to each other is an abstract idea in that it wouldn’t just apply to a particular similarity or to particular objects but rather be an expression of a more contextually independent goal. The interesting fact is that two-to three-year-olds often become almost compulsive spatial sorters. Confronted with an array of 4 identical cars and 4 identical dolls, they physically group them – moving all the cars spatially close to each other and spatially apart from the groups of dolls even though there is no explicit task to do so (Mandler, Bauer, & McDonough, 1991; Nelson, 1973; Rakison & Butterworth, 1998). The developmental evidence with respect to this phenomenon suggests a progressive discovery. Nine-to 10-month-old infants when given sets of objects of like kinds do not systematically form spatial groups organised by similarity. However, they do – more often than expected by chance – pick up like objects (two spoons) but not unlike objects (a spoon and block), one in each hand, and bang them together (Forman, 1982). By 12 months of age these manipulations – like manipulations of like kinds – become more systematic (Sugarman, 1983). For example, given 4 cars and 4 dolls, the child may systematically push each of the four cars. Around 18 months of age, children will not only manipulate objects from one category in sequence but also systematically manipulate in different ways objects from two different categories, for example, first pushing each of four cars, one after another and then touching each of four dolls in turn. Sometime after 24 months, the sorting seems more purposeful with all of one kind gathered to form one group and the other kind left unorganised.

Analyses of the dynamics of infant and toddler exploratory play (Sheya & Smith, 2010a, 2010b) have shown how one action (a reach to an object) biases (with respect to both the visual properties of the next target and its spatial location) the next action and how this bias can lead to stable spatial constructions of like things spatially grouped together. But because the environmental effects of the infants own actions – including stable ones – are perceivable by the infant and coupled to the internal dynamics that created them, they may – like the writing experiences that create the letter A – create functional networks that instantiate goals in the form of planned sequences, as well as the goal to construct, that is, the goal to make something. In brief, what we do – how we do it, and the physical effects of these on physical and perceptible world – drives cognitive development. Therefore, developmental changes in the body – and its effects on the physical world – are central to a theory of embodiment. In addition, how the body affects the world that is how the world is structured by behaviour changes over development.

Developing environments

There are dramatic changes in the motor abilities of humans over the first 18 months of life. A large literature documents dependency between these specific motor achievements and changes in perceptual and other developments in typically (see Adolph & Robinson, 2015; Bertenthal & Campos, 1990; Smith, 2013) and atypically developing children (Bhat, Landa, & Galloway, 2011). For example, pre-crawlers, crawlers, and walkers have different experiences with objects, different visual spatial experiences, different social experiences, and different language experiences that are tied to posture and can be influenced by experimentally changing the infant’s posture (Adolph, Tamis-LeMonda, Ishak, Karasik, & Lobo, 2008; Smith, Yu, Yoshida, & Fausey, 2015). Input statistics change profoundly with every change in motor development.

Recent findings in egocentric vision provide a good example case. Ego-centric vision is the first-person view. The personal view is depends on the individual’s momentary location, orientation in space and posture (see Smith et al., 2015 for review). And critically, because infant-perspective depends on the perceiver’s body morphology and behaviour, the properties of these scenes change systematically with development (Fausey, Jayaraman, & Smith, 2016; Gilmore, Raudies, & Jayaraman, 2015; Jayaraman, Fausey, & Smith, 2015; Kretch, Franchak, & Adolph, 2014). In this way, sensory-motor development bundles visual experiences into separate datasets for infant learners. For example, people are persistently in the near vicinity of infants during their first two years (and people have both faces and hands connected to the same body. But analyses of a large corpus (Fausey et al., 2016) of infant egocentric scenes captured in infant homes during everyday activities shows faces to be highly prevalent for infants younger than 3 months and much rarer for infants older than 18 months. In contrast, for younger infants, hands are rarely in view but for older infants, hands acting on objects (own or others) are nearly continuously in view. Young infants – through the rewarding dynamic cycles of face-to-face play – generate regularities in behaviour and sensory inputs that are prior to and fundamentally different from the regularities generated by toddlers acting and observing the actions of others on objects, and thus, the information available to a child is determined by the development of their sensory-motor systems.

Brain networks change, bodies and what they do change, and the environment and its regularities change in deeply connected ways, with causes and con-sequences inseparable within the multi-scale dynamics of the brain-behaviour-environment network. Theories of how evolution works through developmental process have noted how evolutionarily important out-comes are often restricted by the density and ordering of different classes of sensory experiences (e.g. Gottlieb, 1991). This idea has been conceptualised in terms of “developmental niches” that provide different environments with different regularities (e.g. Gottlieb, 1991; West & King, 1987) at different points in time. These ordered niches – like a developmental period dense in face inputs or dense in hand inputs – play out in the development of individuals in real time and have their causes and consequences in the dynamic interplay of structural and functional brain networks through the body and in the world across shorter and longer times scales.

Conclusion

Over multiple time scales brains, bodies and environments are structuring each other. It is through the moment to moment influences that cognition emerges and it is with particular brains, bodies and environments that cognition is weaved. To understand our abilities to categorise, communicate, use tools, imitate and use abstract formal systems like mathematics, we must focus on their development. Traditional approaches to cognition have sought to explain the products of cognition individually, assuming the fundamental structure of cognition is static operating on a consistent environment. Here we have argued that to understand the structure of adult cognition, we must understand the process by which cognition emerges. That is the dynamic, online processes that coordinate sensory-motor systems with a changing, variable environment that is itself structured by and not independent of bodies, brains and behaviours. This is consistent with embodied cognition, which holds that bodies (including brains) and the environments in which those bodies are embedded are essential, fundamental structuring influences on cognition. Because development is driven by behaviour and its consequences development is embodied. Cognition is thus, embodied through development. That is, cognition is fundamentally a contextual-historical process that links brains, bodies and environments in reciprocal structuring interactions. By simply examining the woven, the product of development, we have discerned fascinating structure that has only led us to a theoretical morass in that there is no readily apparent way to resolve the conflict between embodied and traditional characterisations of cognition solely through experiments with adults. We have argued that the way forward is a focus on the phenomena of cognition at its most fundamental: how current states influence future states, the capacity to sustain, store and retrieve previous experience, the ability to integrate experience to uncover abstract relational structure and the systematicity of knowledge. To move forward we must embrace the historical-contextual (developmental) process from which the properties of cognition emerge. In studying this process, the weaving of brains, bodies and environments, we can determine what patterns might be woven and thus explain the properties of cognition.