The emerging science of interacting minds

Abstract

For over a century, psychology has focused on uncovering mental processes of a single individual. However, humans rarely navigate the world in isolation. The most important determinants of successful development, mental health, and our individual traits and preferences arise from interacting with other individuals. Social interaction underpins who we are, how we think, and how we behave. Here we discuss the key methodological challenges that have limited progress in establishing a robust science of how minds interact and the new tools that are beginning to overcome these challenges. A deep understanding of the human mind requires studying the context within which it originates and exists: social interaction.

Introduction

Since its inception in the late 19th century, experimental psychology has focused almost exclusively on the individual. In many ways, this atomistic focus is defensible. Mental activity presumably reflects activity in the central nervous system, which is largely confined within a single person’s brain. Building on this work, cognitive neuroscientists have attempted to ground these psychological processes in biology by systematically mapping an individual’s thoughts to patterns of neural activity (Figure 1A). This image of the mind has guided over a century of scientific progress in psychological science. However, it ignores the most important driver of human thought and behavior: interaction with other minds. Here we argue for the importance of studying individuals as interacting nodes within social networks, their most natural ecological context.

Figure 1. From individual minds to collective behavior.

A) Traditional research in psychology and neuroscience attempts to study psychological processes within a single individual using experimental paradigms devoid of social context. B) Social interaction research attempts to study the interaction between latent psychological processes and behaviors that occur within the context of an interaction. This work tends to focus on dyads and small groups and has primarily sought to understand how individuals communicate and represent others’ unobservable mental states. C) Collective behavior attempts to study emergent properties of the collective based on modeling behavioral processes with minimal consideration of the latent psychological processes of a single individual.

In this paper, we briefly discuss prior research that demonstrates the immense consequences of social interaction for individuals and collectives. We then address why the study of interaction itself—the meeting of minds that co-constitutes thought and behavior— has been empirically neglected despite its fundamental importance. Finally, we explain why we believe we are on the cusp of a conceptual and methodological renaissance in psychology that will refocus the field on the importance of interaction for understanding the human mind.

Interaction shapes the individual

From birth, we are predisposed to attend to others and to elicit attention from those around us (Goldberg, 1977). This system of coupled attention is thought to be facilitated by two things: (1) a set of innate detectors for social primitives that help infants locate a caregiver (e.g., face pattern—(Johnson et al., 1991), smell—(Browne, 2008), sound of the mother’s voice—(Fifer & Moon, 1994) and (2) a reward system that seeks contingent experience (P. Watson et al., 1967). Infants whose caregivers act contingently on them (e.g., by responding rapidly to their cries) are developmentally advanced relative to other infants whose mothers are less responsive (Ainsworth et al., 1974). Caregivers, in turn, are rewarded when their infant directly responds such as tracking them with their eyes, attending to changes in pitch, and becoming calmer with rocking and singing (Mehr et al., 2016). In this way, both infants and adults are rewarded when they capitalize on the skills and preferences of the other. This adult-infant coupling is honed and reinforced by mutual contingency to create a dynamic co-regulating system which is critical for healthy development.

Over time, the infant-caregiver dynamic gives way to other social relationships (Vygotsky, 1978). Adolescence, for example, is marked by a strong drive to mutually influence the perspectives of peers on everything from aesthetic preferences to social norms (Berns et al., 2010). Our individual preferences are influenced by others (Zaki et al., 2011) and the interaction context itself shapes our thoughts and behaviors in accordance with specific social roles such as leaders, followers, facilitators, or contrarians (Dowell et al., 2019). Problematic interaction dynamics are often the specific focus of individual or couples/family psychotherapy and may manifest in a therapy session via transference (Safran & Kraus, 2014) or an interaction dynamic within a couple such as a demand-withdraw pattern (Christensen & Heavey, 1990). By coupling our minds, we form bonds and alliances, learn information, establish norms and preferences, coordinate actions, and regulate our emotions. These bonds can be so strong that their disruption is destabilizing. In a phenomenon known as the widowhood effect, people whose spouses had just died had a 66% increased chance of dying within the next three months (Moon et al., 2014). Throughout the lifespan, individual minds are forged in and maintained by interaction. Through interaction, we stay adapted to the group; an essential skill for group living.

Interaction shapes the collective

Social interactions not only shape the individuals that comprise them but also the social networks they collectively form. Knowledge and ideas shared in one interaction, are transmitted and transformed through future interactions with new partners (Fowler & Christakis, 2010). Furthermore, by increasing the size and reach of our networks, social media has revolutionized the speed and distance by which information travels. Political dissension can turn into massive coordinated responses (e.g., the Arab Spring—(Eltantawy & Wiest, 2011); fall of communism—(Shirky, 2011)), memes go viral, and health behaviors spread (Centola, 2011; Fowler & Christakis, 2010). We can be influenced by people we know only distantly, due to the web of interactions between us and them.

Gossip, a common feature of social interaction, has important consequences for shaping the collective. Despite its negative connotations, recent research suggests that gossip actually promotes social connection and cooperation (Jolly & Chang, 2021). Exchanging reputational information facilitates vicarious learning to allow the collective to identify bad actors, and also establishes and reinforces collective norms that benefit the group.

How people are connected in a social network is not random. For example, most people tend to be homophilous—gravitating to individuals who think in ways that are similar to themselves (Parkinson et al., 2018). These interactions are rewarding but also lead to cliques or “echo chambers” where homogeneous beliefs, backgrounds and skills constrain collective thought. Other individuals seem to resist the pull of homophily, connecting broadly across a social network. These individuals, known as social brokers, interact in ways that cross pollinate ideas across cliques. Recent research suggests that more culturally diverse environments (Wood et al., 2023) and interdisciplinary environments (Smaldino & O’Connor, 2022) may facilitate this broader pattern of interaction.

Comparative studies with other species have demonstrated the efficacy of modeling how interactions shape collectives (Figure 1C). For example, bees have to maintain their hive within a particular temperature range. When the temperature gets so high as to threaten the life of the hive, bees begin to fan their wings to drive the hot air out. Critically, the threshold to start fanning varies across bees, which optimizes efficient temperature control for the hive as a whole (Peters et al., 2022). Ants, also, act like a distributed computer in which tasks get dynamically re-allocated across the population to match challenges facing the colony (Gordon, 2011). Fireflies alone blink randomly but, together, can create symphonies of coordinated displays, maximizing the reproductive success of the species (Sarfati et al., 2021). And schools of fish and flocks of birds can create intricate patterns of collective behavior that increase protection from predators based on each individual organism following simple rules (Couzin, 2009; Katz et al., 2011). All of this work demonstrates one very simple but profound idea: biological processes in social species cannot be wholly understood by focusing on a single organism. Instead, biological processes are systems of interactions. Interactions between individuals create emergent patterns that cannot simply be reduced to the sum of the individuals.

Of course, it is important to recognize that humans are different from ants, bees and fish—species that tend to exhibit stereotyped behavioral patterns governed by relatively simple rules. Although humans are similarly dependent on social support and cooperation, human sociality is marked by constant mutual adaptation with rules generated on the fly (Misyak et al., 2014). Human interaction involves moment-by-moment monitoring of subtle cues of attention and understanding, and constant readiness to repair misunderstandings the moment something goes awry (Cheong et al., 2020; King-Casas et al., 2008). In pursuit of relational and informational goals, we tweak our prosody, pause structure, and gestures, we vary the delivery of nods and affirmations, move closer or farther apart, and make and break eye-contact (Clark, 1996; Cooney & Wheatley, in press; Grieser & Kuhl, 1988; Wohltjen & Wheatley, 2021). All of these channels of information, and many more, are simultaneously adapting across diverse timescales in the coupled system of interacting minds in order to allow us to create shared beliefs, which in turn shape the processing of future information and guide subsequent interactions, allowing thoughts to travel mind-to-mind through vast social webs (Chen et al., 2019; Higgins et al., 2021; Jolly & Chang, 2021; Sievers et al., in press).

Rather than be deterred by this complexity, we must recognize its existence and work towards advancing methods to study it. As a feature inherent to interaction, this complexity necessarily also underpins both the individual mind and the behavior of the collective. Thus, a deep understanding of the human mind, at any level, depends on developing the tools and training programs that can capture and analyze the complexity of real human social interaction.

The troubled history of interaction science

The importance of social interaction was not lost on scientists in the early 20th century. Kurt Lewin and Lev Vygotsky were some of the first to highlight the dynamic interdependence between focused interactions with varying members of society and the cognitive development of an individual (Lewin, 1939; Vygotsky, 1978). Gregory Bateson, an anthropologist, similarly noted that interaction between young monkeys communicated multiple levels of information, allowing them to negotiate play fighting without getting hurt (Bateson, 1955).

Despite an early understanding of its importance, studying interaction itself proved difficult. Interaction is, by nature, information rich, unfolding across multiple timescales and communication channels. Early efforts to quantify social interaction were markedly impoverished. One of the earliest was the “interaction chronograph” (Chapple, 1939): an apparatus housing a roll of paper and two keys operated by an observer. The paper would pass through the apparatus at uniform speed. When person A spoke, the operator pressed the A key; when person B responded, the B key was struck, and so on. The resulting data comprised a set of intermittent lines with variable spaces between, denoting when each speaker had spoken. Although Chapple’s chronograph showed that people have reliable patterns of initiation and response, this information was “only the barest bones of interaction” and its application proved limited (Goodenough, 1941).

Developments in the late 1940s—such as Wiener’s cybernetics, Shannon and Weaver’s information theory (Shannon, 1948; Wiener, 1948), and game theory (Axelrod & Hamilton, 1981; Rapoport & Chammah, 1965; von Neumann & Morgenstern, 1944)—provided a richer analytical frame by conceiving interaction as a self-organizing network of nervous systems. Game theorists found that they could predict attractor dynamics of action spaces within simple social dilemmas such as a Nash Equilibrium where each player converges on a set of actions in which they can no longer improve their positions in the game (Nash, 1950). Inspired by this framework, researchers embarked on a search for “natural sequences of behavior” that could be treated as structural units of interaction, including word orderings, turn-taking patterns, and other regularities (Greenberg, 1963; Sacks et al., 1974). Work focused on describing the behavioral units of interaction eventually emerged as the field of conversation analysis.

While linguistics and information theory were busy describing the units of interaction, the field of social psychology became more invested in understanding its influence (Clark & Wilkes-Gibbs, 1986). In the wake of World War 2, there was a new and pressing desire to understand how human beings could commit atrocities. The lay intuition In the United States at the time—that the rise of the Nazi party was a peculiarly German phenomenon—did not resonate with social psychologist Stanley Milgram. Milgram’s background in conformity theory with Solomon Asch instead led him to test how the social context might enable one mind to overpower the moral compass of another (Milgram, 1963). The seminal work of Asch and Milgram elevated social influence as an important object of psychological study, providing the foundation for later research on topics such as persuasion, group-think and prejudice (see Ross, Letter & Ward for a review).

Social psychologists were more invested in the psychology of interaction than their economic or linguistic colleagues, but their methods were limited in several important ways. Dyadic and group paradigms tended to restrict analysis to observable outcomes rather than elements of the interaction itself. These interactions were also often contrived, involving one or more employed actors, leading to concerns of deception, ecological validity and reproducibility across laboratories (Kuhlen & Brennan, 2013). Whenever the actual interaction was the focus of a study, it had to be laboriously coded by human raters. This approach nonetheless had successes including spawning the field of couples research (Gottman & Levenson, 1992) and schools of family systems psychotherapy such as the Palo Alto model (Rohrbaugh & Shoham, 2001) and structural family therapy (Minuchin & Vetere, 2020). Ultimately, however, the challenges involved in data collection and analysis meant that psychologists drifted away from studying human interaction.

This move was hastened by the proliferation of personal computers in the 1980’s which offered new heights of experimental control. Computers could be programmed to present stimuli (e.g., photographs of facial expressions), repeatedly, affording averaging of responses across trials. Single-participant paradigms could be reproduced exactly from one participant to the next, lab to lab. Social psychological questions adopted the methods of psychophysics and cognitive science which were focused on the study of a single individual. Rather than the science of social interaction, social psychology became centered on the question of how an individual processes social stimuli. The advent of social neuroscience exacerbated this trend further by rotating the cubicle 90 degrees; sliding each participant into an fMRI scanner. Optimizing for the fidelity of the psychological constructs and experimental control effectively excised real social interaction from experimental paradigms designed to study human sociality. In many ways, the price of this perceived objectivity was the object itself.

The outlook: New reasons for optimism

Calls for the study of naturalistic social interaction are almost as old as the field of experimental psychology (see Cooney & Wheatley, in press, for a review). On various occasions, these calls have coalesced into movements which have gained some traction yielding interesting results and perspectives (e.g., Galantucci & Sebanz, 2009). However, each of these prior movements eventually lost steam, and fell short of their proponents’ ambitious goals for radically transforming psychology. The field of psychology is still dominated by the study of isolated individual minds, rather than interacting minds.

Given the fate of these previous efforts, it would be prudent for us to consider whether our current advocacy will meet a similar fate. Will the present generation of interaction science finally realize its promise as a generative research program? We believe that there are reasons for optimism this time around; a sentiment shared by an increasing number of researchers (e.g.,(Dingemanse et al., 2023; Higgins et al., 2021; Redcay & Schilbach, 2019; Shteynberg et al., 2023; Wheatley et al., 2019)); who believe that this research will reshape our understanding of cognition “not as the province of singular minds but as an interactional achievement of embodied agents” (Dingemanse et al., 2023).

At a surface level, previous movements seemed to fade for diverse reasons. However, beneath the surface, we suggest that there are common challenges that help explain their struggles. In this section, we describe four such challenges: lack of scalability, lack of tools for causal intervention, lack of suitable modeling techniques, and lack of diverse theoretical frameworks. We also describe recent innovations that may finally equip interaction researchers to meet these challenges which improves the outlook for studying naturalistic social interactions.

Scalability

To fully understand interactions, one must study what actually occurs in those interactions. Asking people later about what they thought, felt, or did during an interaction can be useful, for instance, when the outcome of interest is an individual’s explicit construal of the interaction, or when such assessments might predict an individual’s future behavior. And the ease of administering and analyzing self-reports makes these paradigms amenable to large scales of data collection. However, such measures are less useful for understanding what happens within an interaction itself, as they provide only a low-dimensional representation (e.g., ratings on a Likert scale) constrained by what can be remembered and verbalized (Nisbett & Wilson, 1977). As such, self-reports miss the more complex and nuanced dynamics of interaction that unfold moment-by-moment.

Until very recently, studying social interaction in its natural complexity has required extraordinary effort, limiting its scale. Whether running an interaction chronograph or coding video recordings, researchers had to directly observe and manually annotate the behaviors of interacting individuals (Gottman et al., 1990). Annotating even a single feature in this context is an extremely slow, tedious business. For example, coding facial expressions using the Facial Action Coding System (Ekman, 2002) requires extensive training and is time-consuming to perform (Freitas-Magalhães, 2021; Hamm et al., 2011). Annotating a few hundred images might not be too difficult, but annotating hundreds of hours of video sampled at thirty frames per second is incredibly time-consuming. This approach simply cannot scale up in an efficient way to large datasets or large numbers of features. As a result, the notion of completely quantifying all aspects of massive interaction datasets has seemed like a pie-in-the-sky notion: until now.

Fortunately, recent advances in machine learning now make it possible to extract socially-relevant features in an automated way from images, video, audio, and text. These tools allow for the annotation of objects (Wang, 2016), scenes (Xie et al., 2020), faces (Baltrusaitis et al., 2018; Cheong et al., 2021), body pose (Kocabas et al., 2021), the content of speech (Radford et al., 2022), tone of voice (Wagner et al., 2022), and the meaning of text (Devlin et al., 2018). Although they are presently imperfect in multiple respects, these tools already offer options that scale much more efficiently than manual annotation. Similar tools are already dramatically reshaping the study of behavior in nonhuman animals (Mathis & Mathis, 2020).

How will these automated quantification tools - and the scalability they offer - benefit social interaction research? The most direct way will be as a labor, cost, and time-saving measure. This is not a trivial benefit. Studying naturalistic interactions is a slow, challenging process. In the time it takes to manually annotate a single interaction dataset, a researcher could instead run a dozen simpler online surveys. The publish-or-perish nature of the modern academy may perversely incentivize the latter. While survey research certainly has a place in psychological research, it—like all methods—should be used based on its merits to solve a particular problem, not because it is simply more convenient. The time-saving property of automated quantification makes it possible to more effectively capture and analyze the complex data that interactions produce.

Another benefit of improved scalability is that it can help researchers collect larger samples. There are multiple motivations to do this. First, an increasing awareness has permeated psychology and allied fields that larger sample sizes are needed to provide adequate statistical power/estimation precision (Cremers et al., 2017; Fritz & MacKinnon, 2007; Marek et al., 2022; Rossi, 1990; Schönbrodt & Perugini, 2013; Stanley et al., 2022). Obtaining sufficient sample sizes—particularly when the unit of observation may be a dyad or group—can be challenging, but better scaling helps to mitigate this problem. Second, psychologists have also started to place increased emphasis on diversifying participant samples and drawing samples from different cultures (Apicella et al., 2020; Henrich et al., 2010; Moshontz et al., 2018). To capture the full range of human variability, and to draw comparisons between different groups, inevitably entails the use of larger samples. Third, naturalistic social interactions are incredibly high-dimensional processes. Between all of the nuances of natural language, facial expressions, vocal qualities, and body language, there are likely hundreds or thousands of quantifiable features. Disentangling this complexity requires large samples for both statistical and conceptual reasons. Statistically, large samples - of both participants and stimuli - are needed to help overcome the curse of dimensionality (Jolly & Chang, 2017). Conceptually, human social behavior plays out dynamically over time, and one can rarely ascertain all of the relevant variables from any single instant in isolation. For example, if one has ever observed a married couple arguing, it immediately becomes obvious that one cannot fully understand this interaction without also understanding the long history of past interactions which led to it. Indeed, this sort of social inscrutability can emerge surprisingly quickly in naturalistic interactions (Garrod et al., 2007; Schober & Clark, 1989).

One other important benefit to automated coding approaches is that they can happen in near real-time. A human annotator could never keep up with annotating the facial expressions of people as they observe them, but with suitable hardware, automated methods can. This ability to know how a person is behaving, as they are behaving, opens up a wide range of exciting new study design possibilities and real-world applications. We detail one of those causal interventions in the next section.

An important caveat to using these automated annotation methods is that—despite the popular stereotypes of machines as objective—they seem to be at least as susceptible to bias as human coders. Indeed, there has been considerable discourse of late surrounding issues of bias in, and unethical application of, machine learning and AI (Birhane & Prabhu, 2021; Buolamwini & Gebru, 2018; Caliskan et al., 2017; Stanovsky et al., 2019). AI trained on data produced by humans tends to manifest the same social biases as the humans, reproducing and perpetuating familiar patterns of racism and sexism. Moreover, the accumulation of training data for large image and language models can pose serious legal and ethical concerns, such as violations of privacy and copyright protections. Care must be taken, therefore, to benchmark automated tools not only in terms of their overall accuracy, but also in terms of their potential biases, and to establish the ethical provenance of their training data. This problem may also prove an opportunity for psychologists. Psychologists interested in bias, for example, can use these tools to measure bias in new ways (Charlesworth et al., 2021). Moreover, psychologists are well positioned to contribute to the growing literature on machine bias and the mitigation thereof.

Causal manipulation

Psychological science uses many different empirical methods. However, a focus on randomized tightly-controlled experiments has long been a defining feature of the field, which has historically distinguished it from other social sciences such as sociology and economics (Estes, 2019). As we argue elsewhere in this article, we believe that experiments—particularly paradigms that oversimplify psychological processes in the name of experimental control—have been overemphasized by the field. We cannot understand interactions or collectives solely by studying individual people in isolation, nor can we understand the full richness of naturalistic interactions like conversations by stripping them down to bare bones. More observational and descriptive work, and a greater tolerance of complexity, are needed.

That said, experiments certainly do have their place. One major advantage current interaction researchers have is their ability to experimentally manipulate naturalistic interactions via the medium of the interaction itself. That is, rather than staging an intervention before an interaction starts, or manipulating the entire context of a conversation, researchers increasingly have the option to manipulate specific aspects of the interaction in real-time, such as what people are saying to one another, or their nonverbal behaviors. Previously, the only means by which this could be effectively accomplished was via the use of fake participants, commonly known as “confederates,” who would deliver scripted lines when interacting with real participants. Although these scripts provide experimental control over an interaction, this control is also necessarily at odds with natural conversation which is typically constructed on the fly, by both partners (Kuhlen & Brennan, 2013).

Fortunately, the same sorts of tools which are allowing us to automatically quantify naturalistic social behavior are also making it possible to intervene on it. The increasing ease of web development makes it possible to stage large social interactions and manipulate who can interact with whom (Jolly & Chang, 2021). Already tools exist to manipulate naturalistic static stimuli. For example, artificial neural networks make it possible to generate photorealistic faces that are manipulated in appearance to elicit different trait impressions from participants (Peterson et al., 2022). Tools have also started to emerge to allow researchers to manipulate dynamic cues, such as facial expressions and vocal tone (Arias et al., 2018; Rachman et al., 2018). Large language models (Adiwardana et al., 2020; Floridi & Chiriatti, 2020) make it possible to synthesize ever more human-like text chat responses, and text-to-speech models are becoming ever more proficient at translating that text into realistic-sounding speech (Tan et al., 2022).

Together, such advances are paving the way towards a world in which psychologists can dynamically manipulate different streams of information flowing between participants in naturalistic interactions. Since participants’ actual behavior can also be monitored in real time, such interventions can be contingent and precise, manipulating a specific variable at just the right moment. Rather than breaking interactions down into their simplest components to facilitate experimentation—and risk distorting or derailing critical emergent properties in the process—this approach would make it possible to perform well-controlled interventions on naturalistic social interactions without losing any of their richness.

Fully closing the loop from naturalistic observation to naturalistic experimentation will lend the current generation of interaction researchers a critical tool that previous generations lacked. In a recent study using unstructured conversations, a robust association was observed between faster response times and feelings of social connection (Templeton et al., 2022): the faster the response times, the more connected conversation partners felt. This led to an experiment to test whether response time alone might be causal of connection. The researchers manipulated the response times in recorded natural conversations by shrinking or inflating the gaps between turns. They found that conversations with shorter gaps (faster response times) were perceived as more connected by third party listeners compared to the same conversations with longer gaps. Here, the naturalistic observation of a conversational feature associated with rapport (short gaps) led to an experiment to identify a possible causal direction.

It is easy to imagine how real-time, contingent feedback on one’s own and others’ behavior could become a platform for everything from clinical treatments for conditions such as social anxiety disorder to workplace interventions to mitigate social biases. Realistically, the technologies to perform these interventions are still far from being perfectly naturalistic or seamless. However, the state of the art in relevant domains of computer science is advancing rapidly. Psychologists should not only prepare themselves to capitalize on this approach as the technology reaches maturity, but take a proactive role in developing these tools in ways that center psychological understanding and ameliorate ethical concerns.

Complex modeling

Naturalistic social interactions are complex systems, replete with nonlinear dynamics, elaborate dependencies, and influential hidden states. Modeling such systems requires appropriately powerful tools to match. This statement applies to statistical modeling to facilitate aims such as parameter estimation, hypothesis testing, and out-of-sample prediction. It also applies to cognitive modeling, to allow researchers to capture the latent psychological representations and processes which govern human social behavior.

Unfortunately, the modeling tools available to previous generations of interaction researchers have not been up to this significant challenge. Psychology has traditionally relied on relatively simple statistical modeling techniques, based on the linear model (e.g., t-tests, ANOVAs, and regressions). Such models are foundational yet insufficient to grapple with the full complexity of social interactions. In comparison, computational cognitive modeling has been relatively rare in any form within the field. There have been efforts to model psychological processes such as mentalizing using psychological game theory (Chang & Smith, 2015; Dufwenberg & Kirchsteiger, 2004; Gao et al., 2021; Geanakoplos et al., 1989; González & Chang, 2021). There have also been developments emerging from the field of cognitive science modeling communication (Frank & Goodman, 2012; Hawkins et al., 2023) and inferring representations of others mental state using inverse planning frameworks and spatiotemporal reasoning (Baker et al., 2017; Fan et al., 2020; Ho et al., 2022; Jara-Ettinger, 2019; Jern et al., 2017; Malik & Isik, 2022).

Computational cognitive modeling is likely to prove particularly important for studying human social interactions and collectives. In some social situations, such as crowd flow, humans behave largely in accordance with relatively simple rules like those expressed by nonhuman animal collectives. Although the emergent behavior of such collectives can be complex (Couzin, 2009), modeling it does not require a complex representation of each individual mind. However, contexts such as crowd flow differ in important ways from many types of common human social interaction. The course of a dinner party cannot be fully described, nor its sequelae accurately predicted, using something like a swarming model. This isn’t merely because body movement is not the predominant means of interaction at a dinner party. It is because—in such contexts—human behavior is largely governed by complex internal states and processes that cannot be directly observed from their manifest behavior. Moreover, even the manifest behaviors cannot be understood from their observable physical properties alone, as they acquire hidden symbolic meanings by virtue of social conventions and on-the-fly social construction (Stolk et al., 2022). Modeling the individual completely requires modeling the interactions they are embedded within, but conversely, modeling the interactions requires modeling the minds of the individuals. As seen elsewhere in this issue, the relatively simple mental models built into contemporary agent-based models do effectively capture some aspects of human collective behavior in specific contexts. However, we fear that current instantiations may not prove adequate to capture the full gamut of human sociality. We look forward to future work that incorporates more sophisticated models of individuals’ internal states and mentalizing processes that can be used to simulate emergent behavior across a variety of complex interactional contexts.

Another important advantage of formal computational modeling is that it makes our theories more precise (Robinaugh et al., 2021), and thus easier to falsify. It also makes it possible to distinguish between theories which might sound similar verbally, but which make radically different quantitative predictions. The conceptual vagueness of verbal theories hinders psychology’s attempts to become a more cumulative field (Jolly & Chang, 2019; Yarkoni, 2022).

Fortunately, an increasingly powerful array of tools is becoming accessible to support psychologists’ cognitive and statistical modeling efforts. Here we highlight three types of modeling that we suggest may be particularly useful for advancing research on interactions and collectives: graph theory (e.g., social network analysis), artificial neural networks, and methods for dealing with complex statistical dependencies.

Social network analyses have already gained in popularity in social psychology and allied subfields in recent years (Baek et al., 2021; Coman et al., 2016; Morelli et al., 2017; Paluck et al., 2016). These methods allow researchers to quantify the social structure of groups, such as the presence of interconnected communities or the small-world-ishness of a collective. Social networks also make it possible to measure the emergent properties that individuals acquire by virtue of their relationships, such as various forms of centrality. For many psychologists, the most canonical form of social network is a friendship network. However, to study live social interactions, psychologists will need to adopt approaches pioneered by animal collective behavior researchers for analyzing rapidly time-varying graphs (Farine & Whitehead, 2015). One could imagine a modern version of the interaction chronograph which captures not only when each member of a group is speaking, but also the content of that speech, to whom it is directed, and how others react to it nonverbally - all encoded in multilayer social graphs (J. P. Chang et al., n.d.; Dowell et al., 2019; Zhang et al., 2018).

Artificial neural networks, or ANNs, are likely to play an important role in both the statistical and cognitive modeling of social interactions (Zhang et al., 2018). Deep learning has already revolutionized machine learning, and currently holds state of the art performance records across a variety of statistical prediction tasks (Floridi & Chiriatti, 2020; Tan et al., 2022; Wang, 2016). ANNs have also already met with considerable success as cognitive models in other subfields of psychology and neuroscience. For instance, in vision science, deep convolutional neural networks are providing new insights into the functional organization of visual cortex (Mehrer et al., 2021; O’Toole & Castillo, 2021). Indeed, the success of these models has been so dramatic that some have described them as the new paradigm for neuroscience (Richards et al., 2019). Beyond psychology, ANNs have also proven capable tools for modeling complex systems, such as three-body gravitational dynamics (Breen et al., 2020) and weather (Pathak et al., 2022). Given that social interactions share many features of these systems, such as sensitivity to initial conditions, this capacity seems likely to be a prerequisite for discovering regularities within the chaos of the collective. Simulated agents empowered by ANNs - such as deep reinforcement learners - could embody complex interaction policies. Inserting such cognitively sophisticated agents into agent-based models or human-computer interactions may enable these models to capture more complex types of collective behavior (van der Hoog, 2017).

Finally, as psychologists shift their focus from the individual to interactions, they will need to wean themselves off the assumption of statistical independence. The vast majority of hypothesis testing techniques used by psychologists assume independence of observations at some level - typically the individual participant. Even when dependencies do exist, they are typically simple in structure, such as trial observations nested within a participant, or participants nested within a group. Such dependencies can be captured by relatively straightforward mixed effects models (D. Kenny et al., 2020). However, as the nature of the dependencies between people in interacting groups grow more complex, so too will the application of this solution. In theory, one could preserve the independence of observations (at some level) by recruiting multiple groups. In practice, recruiting and coordinating the participation of large groups of participants will pose major logistical challenges, to the extent that collapsing all analyses to the group summary statistic level is unlikely to be feasible. Learning and/or developing techniques to appropriately adjust analyses for these dependencies will be critical to support valid inference moving forward. These methods could take on a wide variety of forms. For example, the social relations model leverages repeated observations of dyadic interactions to separate the influence of each person from the influence of the dyad as a whole (D. A. Kenny & La Voie, 1984). Other approaches include time series analyses such as autoregressive integrated moving average (ARIMA) models that accommodate different types of temporal dependency (autocorrelation, seasonality), Markov models and state space models can help capture dynamical systems (Chang et al., 2021; Thornton & Tamir, 2017), permutation methods such as the Mantel test (Mantel, 1967) or circle shift (Lancaster et al., 2018) to support statistical inference, and more sophisticated mixed effects models to capture more complex patterns of dependencies. As methodologies such as fMRI have shown, the heady early days of a new approach can foster statistical errors (Bennett et al., 2009; Eklund et al., 2016; Vul et al., 2009). Armed with the right statistical tools, interaction researchers may avoid such missteps. Regardless, the path forward is clear: a deep understanding of social interaction must harness its natural complexity. Covariation among features (e.g., prosody, language, gesture) is part of that natural complexity, which we should embrace rather than experimentally excise.

These new approaches may also help the field widen its portfolio of interaction metrics. Synchronous physiological and kinetic states such as those involved in shared attention, clearly manifest in social interaction (see (Hasson & Frith, 2016; Wheatley et al., 2012) for reviews). And, although the term may currently be too loosely defined (Ravignani & Madison, 2017), there is evidence that synchrony in one context can predict synchrony in another. For example, the tendency to synchronize to an audio beat predicts whether people synchronize their attention with another mind (Wohltjen et al., 2023). But adopting synchrony as the lens through which to study all human social interaction is inherently limiting (Holroyd, 2022; Stolk, 2014). Consider the most prototypic human social interaction: a conversation. What would a fully synchronous conversation look like? People would either constantly talk over each other, or sit in silence; they would make the same gestures when speaking as when listening; and indeed, they would have no need to communicate, because their thoughts would already be perfectly aligned. These features do not sound like conversation as we know it. Indeed, researchers have found that people desynchronize their attention often during conversation, creating a rhythm of coupling and decoupling. This rhythm is thought to balance the shared (synchronous) and independent (asynchronous) modes of thought inherent in engaging interactions (Mayo & Gordon, 2020; Wohltjen & Wheatley, 2021). Further, one can imagine finding synchrony between conversation partners at one timescale (e.g., at the level of topic changes or narrative arcs) but not at a finer grained timescale, or vice versa). The ease of conversation belies in multi-layered complexity (Garrod & Pickering, 2004; Levinson, 2016). As our computational and statistical approaches develop, so too will our understanding of how interaction is best mathematically described, including across multiple nested scales.

Theoretical frameworks

A final advantage for current interaction researchers is the array of different theoretical frameworks available to them. These frameworks have accumulated over many decades, from diverse corners of science. Rather than having to view every problem through a single lens, we can now triangulate interactions from multiple theoretical perspectives. This flexibility can help interaction scientists study a wider range of phenomena across a diverse range of interactions to help facilitate aggregation of insights.

For example, information theoretic approaches such as transfer entropy and directed functional connectivity have been used in neuroscience to trace information flow from one brain area to another (Bilek et al., 2015). The same approaches can be used to capture information flow across social networks, as well as mutual adaptation between interacting minds as information flows back and forth. Connectionist modeling posits that complex psychological processes such as collective behavior emerge from the interactions of simpler elements (McClelland, 2013; Rumelhart et al., 1986). For example, in their model, Goldenberg and colleagues use three layers of connection weights to propose that collective emotions unfold as a result of emotional interactions among individuals (Goldenberg et al., 2020).

Dynamical systems theory provides a framework for modeling interacting minds as complex dynamical systems. In this way, interaction can be understood as a trajectory through a dynamical phase space where attractor basins correspond to patterns of coordination between interacting partners (Richardson et al., 2014; Walton et al., 2018). Greater coordination is associated with higher interpersonal stability and a lower energetic cost (Felmlee & Greenberg, 1999; Fusaroli & Tylén, 2016; Koban et al., 2019)—a potential reason why good conversations are described as “effortless.” These approaches can also assess interaction partners’ joint (interpersonal) complexity by assessing convergence and divergence of their dynamic features over time. Greater dyadic complexity has been linked to negative affect and relationship decline (Nasir et al., 2016). The language of dynamical systems may be particularly important for helping us understand the social regulation function of interaction with implications for mental and physical health (Butler, 2011; Butler & Randall, 2013; Dumas et al., 2014).

Finally, the conceptual alignment framework offers a neurocognitive account of how individuals with unique perspectives can come to share a common conceptual understanding during a conversation (Stolk et al., 2016, 2022). Through their behaviors, conversational partners engage in a process of probing, aligning, and shaping each other’s conceptual structures, creating a shared conceptual space that allows them to focus on relevant details and coordinate their next steps in the dialogue. This framework is supported by empirical evidence, including neural observations that reveal communicators and addressees using the same computational procedures, implemented in the same neuronal substrates, and operating over temporal scales separate from behavioral dynamics (Stolk et al., 2013; Stolk, Noordzij, Verhagen, et al., 2014). Research has also shown that individuals with autism spectrum disorder may have difficulty meeting the alignment demands of interaction, leading to more individual exploration when constructing interactive behaviors (Wadge et al., 2019).

These approaches comprise a small subset of the diverse, and increasingly integrated, theoretical frameworks available to the interaction scientist. Each provides its own language for understanding and analyzing interaction, yet none are mutually exclusive. The language of dynamical systems is not in conflict with the language of information theory and so on, and because each framework’s utility transcends academic disciplines, they provide natural channels of communication across departments that might otherwise remain siloed.

Interaction science is already bearing fruit

These new advances are increasingly affording a quantitatively rigorous and psychologically rich study of social interaction (de Ruiter et al., 2010; Galantucci, 2005; J. Misyak & Chater, 2022; Scott-Phillips et al., 2009; Selten & Warglien, 2007; Tomasello, 2008). For example, we are already seeing these approaches provide new insights into how people interact. Natural language processing (NLP) techniques are beginning to reveal how interaction partners coordinate with each other (Dowell et al., 2019) and how the split-second timing of their responses telegraphs connection (Templeton et al., 2022). Chat paradigms that enable real interactions between participants playing a large online game have revealed how gossip naturally emerges to enable vicarious learning across the group and enhance social bonds (Jolly et al., 2021). Experiments using communicative games reveal how people forge a joint understanding even when they cannot use conventional language (e.g., by communicating through artificial words, cursor movements or scribbly drawings; for reviews, see (Noelle & Galantucci, 2022; Toni & Stolk, 2019). By minimizing participants’ access to linguistic and other social conventions, this work reveals the underlying non-linguistic adaptations that forge common ground (Hawkins et al., 2023). A key insight is that effective communication requires continually building a shared context, informed by what is presumed to be known and believed by the other (Stolk et al., 2016, 2022). This joint construction process is critical for achieving group consensus and its neural corollary of inter-brain alignment. Finding common ground not only aligns people as they interact but this alignment persists into the future: individuals process new information through the lens of their prior interactions (Sievers et al., 2022; Stolk, Noordzij, Volman, et al., 2014).

By orchestrating social networks in the lab, scientists have shown that we learn and remember information differently depending on how our social ties are constructed (Coman et al., 2016; Momennejad et al., 2019). By studying real-world social networks, scientists are also discovering how people forge these connections through interaction, and how these connections evolve over time (Falk & Bassett, 2017; Perkins et al., 2015). By investigating actual social interactions in a shared social network, we are beginning to trace how these interactions predict the centrality (and thus influence) of its members (Sievers et al., 2022) as well as large scale social effects such as political participation, polarization (Canen et al., 2022), and poverty (Chetty et al., 2022). Analysis of body camera footage has begun to uncover racial disparities in police officer’s behavior during everyday traffic stops, including their tone of voice and use of respectful language (Camp et al., 2021; Voigt et al., 2017). These examples are just the beginning of how research in psychology can be transformed by investigating phenomena within the context of social interactions.

Today, and increasingly, we do not have to sacrifice the study of social behavior on the altar of reductionism. Now, and increasingly, we have computational tools that can handle the inherent complexity of real social interaction. With these tools, comes a new optimism for re-orienting psychology toward situating the human mind in its natural social context. Natural language processing techniques extract patterns in conversation that can help elucidate how minds mutually influence each other, share emotional experiences, co-create ideas, and forge bonds. Large-scale social media and sensor data can help track thoughts and behaviors as they evolve in the wild. Information theoretic approaches are able to quantify mutual influence and metastability in social interaction. Social network analyses reveal evolving patterns of social connectivity. And because these methods are portable across disciplines, they provide a lingua franca to engage fruitfully across areas within psychology and across departments.

The future of interaction science

Generalization and translation

The relevance and utility of psychological science depends on its ability to generalize beyond the laboratory. Theoretical principles must be translated into practical applications in order for them to positively impact people’s lives. However, establishing the ecological validity of psychological findings has been a persistent challenge for the field. Nearly 70 years ago, Egon Brunswik identified lack of representative sampling - not just of participants, but of stimuli and tasks - as a clear threat to generalization (Brunswik, 1955). Although Brunswik suggested solutions, these never took hold in the field. As a result, the discussion continues, with contemporary commentators declaring a generalizability crisis in psychological science (Yarkoni, 2022). This crisis affects not only development of psychological theory, but also its employment, as translational researchers struggle to develop effective clinical interventions and industrial applications (Sheth et al., 2022).

Multiple paths lead to improvement in the generalizability of psychological science, but here we highlight the importance of studying social interaction in ways that retain ecological validity. As with any complex phenomenon, a purely reductionist approach is likely to run into the same problem as the king’s men: they may be able to break social interaction apart, but they won’t be able to put it back together again. Or rather, there will be so many component parts to reassemble that their possible combinations are nearly infinite, making it impossible to discern which combinations are characteristic of real life. Studying interactions holistically retains their natural complexity thus increasing the likelihood that scientific findings will generalize to real-world situations.

Studying interacting minds with the tools described above may help overcome the scientific challenge of generalization and also the applied problem of translation. For example, the same machine annotation programs that can support scalable interaction science could also be deployed in the clinic to quantify interactions between mental healthcare providers and their patients (Chen et al., 2019; Ramseyer & Tschacher, 2011). Reviewing audio recordings of therapy sessions is already a common training technique in many clinical training programs. With the aid of machine annotation, these reviews could come with the addition of hard metrics on the body language, facial expressions, tone of voice, and linguistic content of both the patient and the practitioner (e.g., (Sanders et al., 2023).

Another clear application for automated techniques is to detect patterns that may be otherwise undetectable. One example comes from research on implicit bias. Dupree and Fiske (Dupree & Fiske, 2019) found that White liberals (but not conservatives) use less competence-related language when speaking to Black audiences and interaction partners, a phenomenon known as “downshifting.” Dupree and colleagues suggest that this behavior may be due to liberal Whites unconsciously drawing on low-competence stereotypes in a well-intentioned, but ultimately patronizing, attempt to affiliate. Machine learning algorithms may be useful for detecting this and other behavioral or linguistic patterns associated with bias, affording valuable opportunities for education and training. Further, it is easy to imagine other domains in which people lack self-awareness for behavioral patterns that machines may more easily detect (e.g., speech patterns diagnostic of dementia —(Liu et al., 2021)).

The amount of interaction data being collected has exploded exponentially in recent years. Researchers are also exploring increasingly diverse datasets, including interracial interactions (Sanchez et al., 2022; Shelton et al., 2023), romantic partner dyads (Arican-Dinc & Gable, 2023; Brinberg & Ram, 2021), online political conversations (Shugars & Beauchamp, 2019), and conversations between patients, families and clinicians (Tarbi et al., 2022). Recently, Reece and colleagues (Reece et al., 2023) published a large-scale public dataset of 1656 conversations in which conversation partners were paired randomly. With this volume of data, advances in natural language processing afford new opportunities to identify characteristics and complex patterns of communication that predict important outcomes from implicit bias to social connectedness and medical care.

Unknown unknowns in psychological theory

The discovery of the double-helix structure of DNA was the culmination of interdisciplinary contributions from biology, chemistry, and physics that transformed latent constructs into observable molecules (Cobb & Comfort, 2023; J. D. Watson & Crick, 1953). This iconic discovery resulted in the “modern synthesis” of evolution via natural selection, population genetics, and Mendelian inheritance and transformed the field of biology. Psychology may well be on the brink of such a precipitous paradigm shift when it comes to interaction science. Just as advances in fields such as physics allowed the development of methods like X-ray crystallography, which were critical for discovery of DNA, so now are advances in fields such as computer science allowing the development of methods such as deep learning, which may prove crucial for breakthroughs in interaction science.

What these breakthroughs will look like is nearly impossible to predict. They may help to settle some existing theoretical debates in psychological science, in much the same way that the discovery of DNA eventually settled the debate on evolution in Darwin’s favor (Huxley, 1943). This would certainly be an important contribution. However, what excites us the most is what we cannot anticipate - the unknown unknowns that may emerge from an interaction-focused psychological paradigm, informed by the nascent methods described above. Embracing the study of interacting minds may produce a similar paradigm shift in psychology. Rather than merely providing an approach to settle our current theoretical debates, it is our hope that many of our existing theories will be amended, if not succeeded, by a deeper understanding of how interacting minds co-constitute thought and behavior.

Interacting minds as the latent level of human psychology

In 1623, John Donne wrote “No man is an island, entire of itself; every man is a piece of the continent, a part of the main.” This ancient, iconic idea is resonating anew in psychological science. Throughout our lives, social interaction is not only the primary niche for learning, it is necessary for our survival, sanity, and success. Behavioral and neurological responses collected in a single individual need to be understood as a product of interaction between the individual and their environment. And the most information-rich and dynamic environmental influence on any one mind is that of other minds. Collective behavior—a level of observation above that of dyadic interaction—is also understood as a phenomenon that shapes, and is shaped by, social interactions. For these reasons, we believe that social interaction is the latent layer of psychology that underpins who we are, how we think, and how we behave - both as individuals, and as groups.

We are neither arguing for a new subfield of psychology nor a revamped version of social or collective psychology. Instead, we propose a re-envisioning of Psychology itself. Our brains evolved in the social context. Human perceptual, sensorimotor, affective and cognitive processes are shaped by interaction and, in turn, shape future interactions. Through interaction, our brains distribute cognition, enabling widespread coordination and collective intelligence. Whether we focus on the individual or the collective, we must understand the interactions that constitute them. We are excited by the promise this perspective holds to accelerate and deepen our understanding of the human mind.

Box: Training the next generation of interaction scientists.

As we detail in this paper, a diverse range of nascent tools are poised to transform the study of naturalistic social interaction. However, to wield these tools to their fullest potential will require changes to the way we train psychologists. This may pose a challenge to the field, given the slow pace of change in graduate training. Over the four decades leading up to 2008, 70–80% of the doctoral statistics curriculum in psychology remained effectively unchanged (Aiken et al., 1990, 2008). Many of the techniques learned by today’s graduate students would not seem out of place in the classrooms of the 1970s. Here we make a set of recommendations for updating training programs to better prepare the next generation of interaction researchers to use the powerful toolkit available to them:

• Programming skills are essential for quantifying, intervening on, and modeling naturalistic social interactions. Although coding has gradually supplanted graphical interfaces in graduate statistics courses, trainees generally receive little explicit instruction on coding as part of their coursework. We recommend that departments add a programming class requirement to their coursework requirements. This will provide trainees with deeper understanding of this essential skill, and leave more room in statistics courses for statistics.

• Modeling naturalistic interactions requires statistical techniques that go far beyond the traditional curriculum in psychology, such as artificial neural networks, time series modeling, network/graph theoretical models, and dynamical systems models. Packing these materials into the 1–2 course statistics sequences common in most PhD programs is unlikely to prove feasible. Instead, we suggest that departments add a “special topics” statistics class that could rotate through these topics across different years. By aggregating students across years, this approach ensures that enrollments remain high enough to justify the existence of such a course.

• Experimentation on naturalistic interactions is a complex, multifaceted process. Existing research methods courses place little attention on essential components such as how to collect high-quality audiovisual recordings; how to design dyadic, group, or round-robin studies; and how to manage the storage, sharing, and privacy of naturalistic data. We suggest that departments introduce graduate research methods courses - if they do not currently offer them - and expand or update existing courses to cover these topics in greater depth.

The skills we outline here are highly valued both within academia and beyond it. Implementing these recommendations may thus have the beneficial side effects of both attracting a wider range of students to PhD programs in psychology and broadening career opportunities for those students.