Beyond the Isolated Brain: The Promise and Challenge of Interacting Minds

Abstract

As scientists, we brainstorm and develop experimental designs with our colleagues and students. Paradoxically, this teamwork has produced a field focused nearly exclusively on mapping the brain as if it evolved in isolation. Here, we discuss promises and challenges in advancing our understanding of how human minds connect during social interaction.

The scientific study of the brain has increased dramatically over the last century, spawning dozens of subfields from cellular to cognitive neuroscience. Modern neuroscience can be credited with incalculably important discoveries across scales from electrophysiological mechanisms to large-scale anatomical and functional architecture. Yet, despite this immense acquisition of knowledge, we still know very little about how human brains work in their most ubiquitous and biologically meaningful context: social interaction.

Social interaction is woven into the fabric of daily life. From birth, we learn from caregivers and, later, from teachers and peers. Long after developmental milestones have been reached, interaction continues to be the medium through which we generate and share ideas, align attitudes and beliefs, tune emotions, and experience the world. As scientists, we brainstorm and develop experimental designs with our colleagues and students. Paradoxically, this teamwork has produced a field focused near exclusively on mapping the brain as if it evolved in isolation.

If the brain had evolved to operate in isolation, we would be better equipped for living alone. However, from survivor spouses to solitary confinement, unwanted social isolation is one of the strongest predictors of mental instability, poor physical health, and suicide. Social interaction appears to afford the inter- and intra-brain processes that keep us sane.

The field’s historical focus on the individual brain is understandable. Serious methodological constraints have limited multi-brain, interactive paradigms. Scalp electrodes sensitive to weak electrical signals are swamped by movements of facial musculature, fMRI requires individual subjects lie supine in a noisy tube, and fNIRS, MEG, and other technologies each have trade-offs that make a robust interpersonal neuroscience challenging. But recent technological and analytical innovations are increasingly optimizing these tools for social contexts, and new tools are coming. Making headway on how brains interact is becoming increasingly tractable.

Moving Beyond the Brain in a Jar

One of the most influential discoveries to open up interpersonal neuroscience is that similar mental processing across brains is revealed in synchronous inter-brain activity. From shared sensory responses to shared social interpretations, greater inter-brain synchrony indexes similar mental states (see Hasson and Frith, 2016, for a review) and the degree of this synchronicity has real-world implications including friendship, learning, and mental health (see Redcay and Schilbach, 2019, for a review). Synchrony, including more complex higher-order correlations, will continue to be a fruitful index of mental alignment.

Separate from brain-to-brain synchrony are approaches that embed brains in live communicative settings, in which human participants are able to directly interact with each other. The rationale behind multi-person interactive settings is that people adapt their communication to their beliefs about a conversational partner (Kuhlen et al., 2017). Through the experimental manipulation of those beliefs, these types of settings have potential to shed light on elusive social conduct disorders as when prefrontal lesion patients do not tailor their interactions to their communication partners (Stolk et al., 2015).

Despite these advances, a large gap remains in our understanding of how human minds connect during social interaction. Although studies of similar mental processing and partner-adapted processing are informative in their own ways, we are still lacking neurocognitive theories of how humans co-create and share information. Beyond the above methodological constraints, the field has struggled to gain experimental access to the idiosyncratic and fleeting shared mental constructs that emerge from everyday dialog. For instance, a great deal of effort has been spent in understanding how human brains process social stimuli, implicitly assuming those stimuli contain stationary meanings. Yet, few studies have investigated how two or more people manage to converge on a shared meaning. As becomes clear below, this deceptively subtle difference is of fundamental importance if we want to understand how human minds meet during social interaction and create mutual understanding of intrinsically ambiguous social stimuli in the face of their ever-changing thoughts.

Toward a Neuroscience of Mutual Understanding

Human communication is at the heart of our social world and provides an exemplary testing ground for understanding the principles and mechanism of social interaction. Historically, cognitive neuroscience has approached human communication by focusing on how linguistic material and body gestures are encoded and decoded by individuals according to the features and structural dependencies of those communicative signals. Yet, human communication is not a sequence of monologues built on context-invariant signals. Daily conversation is built on turn-taking over ambiguous words and behaviors that can only be resolved by individuals who infer their context of use in an ongoing interaction. Using controlled experimental settings that allow pairs of individuals to coordinate and align their mental states on a trial-by-trial basis, a new line of research is investigating how people generate, negotiate, and converge on the meaning of their communicative behaviors. This research is focused on characterizing core interpersonal processes supporting the development and coordination of novel shared representations.

One notable discovery using this approach is that brains become synchronized due to the accumulation of shared representations at a scale independent from the communicative behaviors themselves (Stolk et al., 2014). This interpersonal coupling was driven by periods in which communicators needed to mutually adjust their understanding based on signals from their partners. Conversely, this coupling was reduced when communicators used stereotyped signals that both parties already understood. Observations like these suggest that the meaning of a communicative behavior is not a property of the signal, nor of individual minds interpreting that signal. Rather, communicators engage in a practice of continuous mutual adjustment to keep their thoughts aligned with one another, forming a shared cognitive space that provides the context for selecting and interpreting communicative behaviors that can be mutually and rapidly understood, even at their first occurrence (Stolk et al., 2016).

A recent study causally tested the notion of a shared cognitive space underlying human communication in pairs of individuals on the autism spectrum (Wadge et al., 2019). Despite otherwise indistinguishable performance from neurotypical pairs, pairs of individuals with autism had lower communicative success. This communicative impairment was not simply a consequence of reduced cognitive flexibility or social motivation, as individuals with autism showed a similar propensity to change their communicative behaviors following a misunderstanding with their communication partner. They even spontaneously placed emphasis on communicatively relevant portions of their behavior for the benefit of their partner, as neurotypical partners would do. Crucially, however, individuals with autism struggled to rapidly converge on a shared conceptualization of their communicative behaviors with their communication partners. This communicative misalignment predicted communicative impairment across all pairs and was greatest when the conceptualizations depended on the unique communicative context established through past interaction with their partners.

These findings illustrate how the meaning of a communicative signal is best understood within the conceptual frame of reference that people in dialog develop together. As epitomized by the largely segregated interaction spaces between individuals with autism (Figure 1, cf. proportion of single-color clusters in the pairwise interaction trajectories), neurotypical human minds navigate and constrain the vast space of meaning by continuously considering and aligning to recent signals from their partner. As discussed below, moment-to-moment neural measures of shared representations might provide a window into how shared cognitive spaces are mechanistically instantiated and updated, and how they determine the meaning of a signal.

Figure 1. Interacting Minds.

Network visualizations of pairwise interaction trajectories by a pair of individuals with autism (left) and a neurotypical pair (right). The nodes represent communicative signals constructed by two individuals during an interaction and are clustered to show signals that were used repeatedly. The colored edges connect each individual’s consecutively produced signals over the course of the interaction (blue for one individual, orange for the other). It can be seen that individuals with autism showed more individual exploration of their interaction space, as indicated by relatively large clusters of individually used signals (clusters of nodes that were connected only by edges of a single color) and small clusters of jointly used signals. In contrast, the neurotypical pair navigated the interaction space by continuously considering and aligning to recent signals from their partner, embedding those signals in a strongly interconnected space of meaning and relationships between one another’s signals. ASD, Autism Spectrum Disorder. Adapted from Wadge et al., 2019.

Conceptual and Methodological Challenges

Any new methodology brought to bear on social interaction should not extinguish the phenomenon for the sake of experimental control. Good science needs to be as simple as possible, but not simpler. We need to study human interaction under simplifying conditions without ignoring that the natural context of the phenomenon is multimodal, interactive, and cognitively opaque. Characterizing the neurocognitive mechanisms supporting human interaction will require experimental approaches that respect that context.

Multimodality raises the largely ignored problem of how an individual decides to distribute a communicative goal across different behavioral channels (e.g., speech, hand gestures, facial movements, prosody, body posture), and how those multimodal behaviors are rapidly integrated by another individual given the timing constraints of turn-taking during human dialog.

Interactivity raises the problem of how an individual decides to engage in communicative repairs, like when to interrupt a signaling behavior to ask for clarification, how much clarification to ask for, and how to signal what has not been understood without adding further ambiguities.

Cognitive opacity means that human interaction is driven by mental states, but those mental states are not unambiguously defined by the behaviors we use as referents. There is no explicit feedback or physical pointer to the “correct” mental referent, and communicators need to rapidly solve a complex inferential problem (Frank and Goodman, 2012). This fact differentiates human communicative interactions from competitive interactions, where game-theoretic optima can be analytically derived from a space of possibilities, or local minima could be found across a solution landscape based on supervised learning, as currently implemented in deep learning approaches.

Given that human interaction depends on the context of interaction, and given that communicators continuously modify that context, it will be important to capture behavioral and neuronal responses on a trial-by-trial basis, in order to track the trial-contingent context in which those responses occur.

A first step toward this goal is to describe brain-to-brain interactions in terms of information flow, that is, to quantify the effects that interacting partners have on each other beyond synchrony. Information flow is routinely studied as directed functional connectivity within single brains and might be extended to capture nonlinear neural dynamics of mutual adaptation on the dyad or group level. Another possibility is to use tools from dynamical systems theory, representational similarity analysis, or topic modeling to isolate relevant configurations in neuronal and behavioral state-space. Abstraction techniques like these might be able to capture brain-to-brain and trial-to-trial dependencies in a way similar to the interaction spaces of Figure 1 and prove fruitful for connecting shared representational geometries in brain data to behavior.

Open Questions and Future Challenges

How minds interact is not one question but an umbrella of questions at the heart of what drives us to connect. Such open questions include: why is social interaction protective for mental health? Do coupled brains reduce free energy by creating and leveraging shared mental models? What role does dialogue, the ecological niche of language, play in creating these across-brain patterns? Why do people find it easier to “click” with some people more than others? And to what extent are across-brain patterns observed in other social species?

As with any emerging field, the interpretation of novel brain-to-brain patterns requires great care. Validating the role that the different lawful relationships between and across brains play in social interactions depends on our ability to link them with behavioral outcomes as well as on our ability to generate careful theoretical scaffolding for potential mechanisms. As with any multifaceted question, the question of how minds interact can be posed in multiple ways and at multiple scales.

Though superficially a question of social behavior, the resulting insights will touch all domains influenced by interactive contexts from attention to learning and memory, and perception to executive function. This question is too large to be partitioned to a subfield and will undoubtedly require a vertically integrated, interdisciplinary effort of psychologists, neuroscientists, physicists, engineers, applied mathematicians, computer scientists, and sociologists.

We have learned a great deal from careful, rigorous research on the single brain. But we do not live in isolation. Our thoughts and behaviors are communicated and negotiated in concert with other minds. And much as spiders use their webs to think (Japyassú and Laland, 2017), the social webs in which we are embedded dynamically shape and are shaped by our mental models and experiences. Our health demands this socially organized neural regulation (Smith and Christakis, 2008), and this need is likely true for all social species. Indeed, we already know that social animals housed in solitary cages have substantially worse health outcomes than those co-housed. By looking in only one head at a time, our knowledge about how the human brain works has been necessarily limited. By incorporating the kind of multimodal, dynamic, and collective contexts the brain evolved to solve, the field will be better positioned to achieve a deep understanding of the human mind.