Using second-person neuroscience to elucidate the mechanisms of social interaction

Abstract

Although a large proportion of our lives are spent participating in social interactions, the investigation of the neural mechanisms supporting these interactions has largely been restricted to situations of social observation: that is, situations in which an individual observes a social stimulus without opportunity for interaction. In recent years, efforts have been made to develop a truly social, or ‘second-person’, neuroscientific approach to these investigations in which neural processes are examined within the context of a real-time reciprocal social interaction. These developments have helped to elucidate the behavioral and neural mechanisms of social interactions; however, further theoretical and methodological innovations are still needed. Findings to date suggest that the neural mechanisms supporting social interaction differ from those involved in social observation and highlight a role of the so-called ‘mentalizing network’ as important in this distinction. Taking social interaction seriously may also be particularly important for the advancement of the neuroscientific study of different psychiatric conditions.

Introduction

Our social lives are spent primarily participating in reciprocal interactions. Despite this centrality of social interaction to our everyday lives, the study of the social-cognitive processes supporting social interaction has typically been conducted outside of a social-interactive context (for example, in studies that have investigated the passive viewing of a static image of a face or a story about a character¹). In 2013, a call was put forth for a ‘second-person’ neuroscientific approach to the study of social interaction, proposing that it is fundamentally different to the study of social observation².

The subsequent growth in second-person neuroscientific studies has, in a short period of time, significantly advanced our understanding of the neural mechanisms of social interaction within individuals and across interacting brains and has also offered new insights into the neurobiology of disorders of social interaction, such as autism and other psychiatric conditions. Further, these studies have offered growing support for the idea that neural and cognitive processes differ when examined in a second-person (that is, truly interactive) context from those observed when examined in third-person contexts that require only the observation of social stimuli. Nevertheless, many questions remain and additional research efforts to further develop interaction-based measurements, tools for cross-brain data analyses, and methods to assess their translational potential in psychiatry are required.

In this Opinion article, we review the advances in second-person neuroscience that have taken place in recent years and describe the new types of studies that have been carried out to study the neurobiology of core social-interactive behaviors. We conclude the article by highlighting and expanding on new directions in the field and the ways in which they will advance the neuroscience of social interaction in health and disease.

Second-person approaches

Second-person neuroscience is based on the assumption that social cognition during social interaction is fundamentally different than social cognition during social observation (also referred to as a third-person perspective) and that the same distinction is likely to apply to the underlying behavioral and neural mechanisms². In order to study the neurobiology underlying social interactions, it is therefore necessary to use experimental paradigms that involve participants in structured or –ideally – ecologically valid, dynamically unfolding social interactions. Second-person neuroscience approaches have been defined as studies in which individuals are participants in a social interaction and/or otherwise feel engaged with a social partner (reviewed in REF²). The interactions could be real or perceived; however they will always take place in real-time and be reciprocal, such that one partner’s actions affects the other, and vice versa. Engagement is achieved through the perception that a social partner (or agent) is communicating directly with the participant, which elicits a feeling of involvement with the other. Here we therefore refer to studies in which at least one of these criteria (real-time interaction or engagement) is met, as second-person neuroscience studies. With regard to data collection and analysis, data from either one brain that is involved in a real-time social interaction or from two interacting brains (or indeed multiple brains) can be studied to further broaden the scope of social neuroscience. Here, we refer to these two possibilities as single-brain and dual-brain approaches (Fig. 1).

Fig. 1. Single and dual brain approaches in second-person neuroscience.

The figure depicts the different approaches that are used in second-person neuroscience studies of social cognition and compares them to third-person studies. In each panel the brain image represents the brain activity that is measured from a participant (either one or two brains are shown depending on whether a single or dual-brain approach, respectively). a| In third-person approaches, a single brain is imaged while the participant, in the role of ‘observer,’ passively perceives a non-interactive stimulus, such as a picture of a face or a video of a hand action. These approaches have highlighted regions that are part of distinct ‘social’ networks, including the mentalizing, mirror neuron and reward and/or affective networks as being important for social processing¹²⁹. Several regions that are key parts of these networks are depicted. b-d| Several second-person approaches are illustrated. In each panel, the role of the participant (as the sender or receiver of communication) in the interaction is listed above the brain. These second-person methods can include single-brain approaches (b) that allow for reciprocal interaction between the participant and the interactive stimulus but in which only the brain response of the participant is examined. Examples of such an interactive stimulus include a gaze-contingent avatar, a live social partner transmitted via real-time video link or in the scanner room, or a video recording of a person directing communicative gestures towards the participant. Solid lines between brain regions depict correlations (indicating functional connectivity) within and between regions of different social networks during the sending or receiving of messages or communicative signals to an interactive stimulus^20,82,97. Dual-brain second-person approaches include both sequential and simultaneous dual-brain methods. In sequential dual-brain methods (c), the timecourse and pattern of neural activity in the sender’s brain can be compared with that of the receiver to determine how one partner’s brain activity either influences the other’s brain activity unidirectionally, through the actions that it generates, or is similar to the other’s during the sending and receiving of messages. In these studies, the ‘message’ (which may be facial displays of emotion or gestures during a charade game, for example) is first recorded by the sender while their brain activity is imaged and then presented to the receiver while their brain is imaged at a later time (depicted by a unidirectional arrow). Dashed lines represent regions in which brain activity of the sender is thought to influence or be similar to that of the receiver^6,95. In simultaneous dual-brain approaches (d) coupling (synchronous activity within similar brain regions in all participants) and inter-brain connectivity (correlations between neural activity in one region in one participant and neural activity in distinct regions in another participant) in the brains of participants engaged in direct social interaction can be measured. Examples of this coupling and inter-brain connectivity are depicted by dashed lines between brain images^9–12,73. These single- and dual-brain approaches have demonstrated simultaneous engagement and interactions among nodes of supposedly distinct networks (that is, the mentalizing, mirror neuron, and reward networks^20,28,30,60) which we propose forms an integrated social interaction network. To fully map the organization and function of this social interaction network will require further innovative methodological advances, including more ecologically-valid experimental tasks as well as quantitative measures of collective behavior and a greater focus on the emergence, development, and plasticity of this social interaction network. ACC, anterior cingulate cortex; AI, anterior insula; ATL, anterior temporal lobe; dmPFC, dorsomedial prefrontal cortex; IFG, inferior frontal gyrus; IPL, inferior parietal lobe; IPS, intraparietal sulcus; OFC, orbitofrontal cortex; PCUN, precuneus; STS, superior temporal sulcus; TPJ, temporoparietal junction; vmPFC, ventral medial prefrontal cortex; vPMC, ventral premotor cortex. VS, ventral striatum.

Single-brain approaches.

Until recently, single-brain studies represented the cornerstone of social neuroscience. Because social neuroscience as a field of research developed within the conceptual framework of cognitive neuroscience and was strongly influenced by many of its premises and assumptions, single-brain studies have mostly relied on the investigation of stimulus–response contingencies. In the social domain, this means studying the neural response to a social stimulus (such as a picture of a face, or a pre-recorded video of someone looking from right to left).

Second-person neuroscience, however, suggests that to uncover interaction-specific neural mechanisms, it is necessary to involve the study participant in a social interaction even within single-brain studies. Thus, second-person single-brain studies address how a participant’s brain responds during an interaction with a social partner (real or perceived). This approach includes studies in which participants perceive communicative cues (such as direct gaze or social gestures) communicated via video links or by computer-generated agents that respond in real time by tracking the behavior of the study participant, thus engaging them in a perceived interaction, as well as fully reciprocal, face-to-face interactions with a real social partner. Such studies therefore close the loop between the human participant and the perceived other. Single-brain studies have the advantage of allowing researchers to make use of established tools for data analysis and data collection. Also, by using anthropomorphic virtual agents and highly trained confederates they offer a relatively high level of experimental control. These tightly controlled designs allow one to address the core question of whether the brain responds in a different way when we are engaged and interacting with a social partner compared to observing them. Further, these single brain second-person neuroscience studies allow for the investigation of core social-interactive processes that cannot be probed outside of a social-interactive context.

Dual-brain approaches.

Dual-brain studies, on the other hand, offer the possibility of collecting data from two (or more) persons’ brains during a social encounter, which allows for an examination of how each brain can influence the other during social interaction (Fig. 1). Dual-brain approaches include sequential dual-brain studies and simultaneous dual-brain (also known as ‘hyperscanning’) studies. In sequential dual-brain studies the brains of participants are imaged at different times but the neural activity between sender and receiver can be compared (for example, to determine the influence of one individual’s brain — via the actions that it generates — over that of another individual or the extent to which patterns of brain activity are shared between social partners)^3,4. In simultaneous dual-brain studies two brains are imaged simultaneously during reciprocal social interactions⁵.

Sequential dual-brain studies provide insight into how information is transferred between social partners and how social partners may build shared representations. These studies examine how activation within one brain can predict the brain activation of one’s social partner during dialogue, gestural communication, and emotion processing. Examples of such sequential dual-brain studies include experimental set-ups in which one partner is imaged as they receive a ‘message’ sent by the other partner (via gestures in a game of charades or via emotional facial expressions) during a prior scanning session^3,4. For example, in a game of charades one partner (the sender) would make gestures to convey a word while their brain activity is imaged. The other partner (receiver) would later view a recording of these gestures while their brain is being imaged. Comparison of neural activity between sender and receiver (through between-brain Granger Causality methods, for example) can indicate the degree of influence the sender’s brain activity — via the actions that are generated — has over that of the receiver. Only sequential dual-brain studies that elicit perceptions of real-time interaction (through deception) and/or engagement in the interaction (by instructing couples to communicate information directly to their partners, for example) can be said to be second-person studies and will be included in the discussion below. However, non-interactive sequential dual-brain studies have also been used to shed light on some fundamental aspects of social interaction (see Box 1).

Box 1. Shared representations.

Over the course of an interaction participants build up shared representations or ‘common ground’ which brings them into conceptual alignment¹¹⁶. Pioneering sequential dual-brain studies have investigated how these shared representations may be reflected at the neural level through the use of inter-subject correlation (ISC) methods¹¹⁷. Even though many of these studies cannot be classified as second-person studies because there is either no interaction or no feeling of direct involvement in the interaction, the method conceptually appears to go beyond traditional single-brain studies because it targets the extent of shared processing (measured as inter-brain correlations) across subjects, including those assuming different roles during communication (such as speaker or listener) and focuses on the use of naturalistic stimuli (often videos)¹¹⁷.

Use of this approach has identified, for example, greater similarity at the neural level in participants adopting a shared perspective^118,119 or a similar emotional state^120,121 while viewing a movie clip. Furthermore, similarity in event-specific neural patterns within the so-called default mode network (which has significant overlap with the mentalizing network¹²²) in speaker and listeners is also seen during verbal movie recall^66, and greater neural similarity is related to listeners’ comprehension of the movie⁶⁷. Thus, these sequential dual-brain studies demonstrate how similar cognitive or emotional states are instantiated in similar brain states. This may facilitate information transfer between people and the emergence of shared representations over the course of a social interaction. Whether use of this methodological approach, which does not allow for real-time interaction, affects the extent to or manner in which brain activity is coupled is an open question. However, a recent study demonstrated greater inter-participant neural similarity within ‘core affect’ regions (such as the insula and amygdala) when participants believed they were viewing videos with a social partner rather than viewing them alone and this similarity was greater for participants who felt less alone during viewing¹²¹.

Unlike single-brain and sequential dual-brain studies, simultaneous dual-brain approaches are the only method potentially able to examine emergent dynamics between two interactors in real time. Such emergent dynamics rely on the contribution of (at least) two autonomous agents and can only be described by using interpersonal measures that capture, for instance, how the gaze behavior of person A changes contingent upon the gaze behavior of person B. Initial simultaneous dual-brain studies were limited in that they often used highly constrained tasks originating from game theory (such as economic bargaining games), which did not allow for freely forming, face-to-face interactions, but rather relied on a sequential exchange of symbols⁶. However, an important advantage of these tightly controlled dual-brain experiments was that they allowed for computational modeling approaches to be applied to task-related behavior in order to mechanistically explain the underlying cognitive processes (and their related neural correlates) and their interaction-based development over time^6–8.

In recent years, paradigms involving more ecologically valid social situations including gaze-based and verbal communication tasks^9,10 have been developed. With simultaneous dual-brain approaches researchers can, thus, identify inter-individual synchrony at the behavioral and neural level when two (or more) participants are engaged in reciprocal and freely forming interaction. Examples include studies that have engaged study participants in joint attention, mutual gaze and motor synchrony and have characterized inter-brain connectivity components that are unique to interacting individuals^11–13. Although these dual brain approaches have the advantage of being able to investigate and relate neural responses across the different interacting brains, they pose significant methodological challenges in terms of both data analyses and the implementation of suitable experimental tasks. For example, the possibility that any observed neural synchrony is due to the perfectly synchronous presentation of stimuli to both participants rather than being related to the interaction dynamics per se must be ruled out. Furthermore, dual-brain studies must use tasks that make it possible to record and quantify the relevant social behavior of both interaction partners (such as their gaze or facial expressions) and should include data analyses that extract measures of interpersonal coupling that go beyond behavioral synchrony in order to relate them to the neuroimaging data.

Components of social interaction

Several key components of social interaction have been made accessible for scientific investigation through the framework of second-person neuroscientific studies that have used single- and/or dual-brain approaches.

Mutual engagement.

A core feature of all social interaction is that both partners are mutually engaged with one another: that is, each person within the dyad knows that their actions are directed towards and relevant to their partner and vice versa². This feeling of mutual engagement is challenging to elicit in traditional cognitive neuroscience experiments that lack reciprocal social interactions or use stimuli that make it clear that no real interaction partner is co-present. However, second-person approaches have been developed to examine mutual engagement using both bottom-up and top-down engagement cues.

One bottom-up approach to achieving a feeling of engagement is to present stimuli (such as videos or still pictures) that elicit a feeling of being personally addressed. This may include viewing a person’s direct gaze, hearing one’s name, or seeing a wave hello^14–17. These communicative gestures can be contrasted with third-person directed stimuli that either have no inherent communicative value (such as scratching one’s face) or are directed towards another person. Studies on communicative gestures (including some of the first second-person studies) have revealed a key role of the dorsal medial prefrontal cortex (dMPFC), the posterior superior temporal sulcus (pSTS) and, in some cases, the inferior frontal gyrus (IFG) and amygdala in the detection of these bids for social interaction^{14,16,18–26}. These regions form part of the so-called ‘mentalizing’ and ‘mirror neuron’ networks.

One limitation of the studies that have used communicative gestures to drive a sense of engagement is that there is no reciprocal interaction. That is, the participant feels engaged but that engagement is one-way, as the participant cannot influence the agent. Perceiving contingency in response to one’s own actions is a strong indication of mutual engagement and social agency. It has been shown that when participants perceive that an avatar is responding to their actions in a human-like way (for example, with eye movements that are contingent to the participant’s gaze, but not perfectly contingent) the ventral striatum and orbitofrontal cortex, key components of the reward system, are engaged to a greater extent than when participants perceive that a computer controls the avatar²⁷. Importantly the studies of mutual engagement that allow for a comparison of neural activity during second-person and third-person processing highlight the role of specific regions and networks (such as the mentalizing and reward networks) in mutual engagement as being distinct from their roles in the simple perception or detection of social stimuli.

An alternative approach is to provide top-down information to elicit a feeling of mutual engagement – that is, to tell participants that they will be engaged directly with a live social partner in real time. Using well-controlled designs, the findings of neuroimaging studies that have used this approach provide strong support for the claim that neural responses differ during interaction and observation: in these studies the only difference between interaction and observation is the perception or belief of interaction with a real-time social partner, which elicits a feeling of mutual engagement^22,28–32. Although the specific brain regions activated differ across varied task paradigms, regions of the mentalizing network (including the ventromedial prefrontal cortex (vmPFC), dMPFC, and temporoparietal junction (TPJ)) are consistently found to be sensitive to perceived engagement with a social partner regardless of task demands^22,28–32 (Fig. 2). Similarly, electroencephalography (EEG) studies find larger amplitude responses over occipital scalp locations when contrasting responses recording during interaction with a live social partner with those recoded during the viewing of a photograph^33,34.

Fig. 2. The mentalizing network in social interaction:

a| Key regions of the mentalizing network (based on meta-analyses of theory of mind studies in adults¹) are depicted in the schematic. b| The right panel depicts a typical pattern of activation in studies of mentalizing in a third person context (that is, in which the participant is an observer (or listener) rather than interactive participant). Specifically, regions of the mentalizing network show greater activation during mental state reasoning (that is, when a participant is asked to make predictions about a social partner based on their mental states, such as their beliefs, desires, or knowledge) than during non-mental state reasoning (that is, when they are asked to make predictions based on information about the person that does not involve mental states)¹. However, in second-person studies (in which participants are engaged in social interaction) regions in the mentalizing network respond similarly during social interaction with or without explicit task demands to engage in mental state reasoning²⁸. This comparison of mentalizing network activation in second-compared to third-person contexts highlights the importance of studying social processes within social interaction because it suggests how the brain responds differs when in interaction compared to observation. TPJ: Temporoparietal junction. IFG: Inferior frontal gyrus. aTL: anterior Temporal lobe. dMPFC: dorsal medial prefrontal cortex. vMPFC: ventral medial prefrontal cortex. PCC: Posterior Cingulate Cortex.

Researchers have developed several clever ways to engage participants in interaction with an experimenter during neuroimaging data collection. For MRI studies, some examples include having the experimenter physically within the scanner room either sitting behind the scanner bore to make eye contact with the participant³⁵, holding a participant’s hand³⁶ or playing a cooperative joint action game with the participant³⁷. Studies with functional near-infrared spectroscopy (fNIRS) or EEG provide greater opportunities for naturalistic interactions, without the constraints of needing to lie motionless and supine within a scanner bore^9,35,38–40. With these methods, researchers have found that the neural response to live eye contact differs from that of to ‘eye contact’ with a photo, particularly within prefrontal regions^9,35. Further, dual-brain hyperscanning methods examining inter-brain synchrony during eye contact demonstrate coupling (that is, correlated activity between participant’s brains) within the IFG^12,41 and in regions associated with language processing⁹ when partners engage in mutual gaze (that is, when both individuals in the social dyad are making sustained eye contact).

Behavioral alignment.

For an individual engaged in a dyadic social interaction, reciprocal relations (that is the mutual effect of each participant on the other as a result of actions that they decide to carry out during the interaction) do not constitute the only way of learning about the interaction partner. Our knowledge of the other may also - in part - reside in the social interaction dynamics: that is, in processes of behavioral alignment such as mimicking the other person’s gestures or facial expressions or coming into conversational alignment. Furthermore, understanding another’s mind may hinge upon emotional engagement and a bodily responsiveness to another person’s state, which could complement and/or facilitate more cognitive ways of understanding the other. Such processes of behavioral mimicry and/or synchrony and alignment have been the focus of some research: for instance, the perception of structure in social interactions in order to generate common ground and the specific timing of behavioral cues in order to create situations for learning has been shown to guide vocal development and language learning^42,43. In other areas of research, adults engaged in a social interaction have been shown to automatically coordinate in turn-taking, agree upon names for objects⁴⁴, approach each other’s accents⁴⁵, sway their bodies in synchrony⁴⁶, mimic each other’s facial expressions⁴⁷ and visually coordinate their attention through synchronized eye movements⁴⁸. Furthermore, the phenomenon of social conformity could also be regarded as a case of high-level or cognitive alignment with other individuals, which is also affected by behavioral mimicry⁴⁹.

A unifying theoretical account of the role of social interaction in understanding other minds has recently been proposed, according to which social interaction dynamics precede and modulate the development of individual capacities, such as being able to speak a language⁵⁰. Nonetheless, the neurobiology of behavioral and psychological alignment in freely forming social interactions is not well understood. Studies have demonstrated that the development of social conformity complies with principles of reinforcement learning and that adaptations to the group norm are linked to so-called prediction error signals in the ventral striatum in the adapting individuals^51,52. Although these studies on social conformity were performed in individuals and thus do not address alignment in real-time social interaction, second-person studies have used hyperscanning to examine how behavioral mimicry³⁹ or motor synchrony (synchronous movement between partners) is reflected in neural coupling between brains⁵³. The neural synchrony between the brains of social partners, for instance, predicts their behavioral success in a visual search task⁵⁴ and neural synchrony with their classmates predicts a student’s engagement during a classroom lecture⁵⁵. Neural synchrony also increases as a function of social coordination^13,56. Also, cooperative interactions, in which participants were asked to work together, increased implicit behavioral synchrony in fingertip movement, which was reflected both at the level of the individual brain with greater activity within the posterior middle temporal gyrus, and across brains through increased phase synchrony in inter-brain connections after training¹³. Importantly, the modulatory effect of cooperative social interaction on asymmetric phase synchrony (coherence between different brain regions) was restricted to inter-brain (rather than intra-brain) measures, which underlines the importance of extending the neurobiological investigations of social interaction to interacting brains.

Newer approaches go beyond identifying neural synchrony to begin to classify patterns of coherent activity across two brains (inter-brain multivariate decoding) and have demonstrated that distinct asymmetric patterns of frontal alpha oscillations occur during interactive motor synchrony in which pairs of individuals synchronize their finger tapping to each other but not during non-interactive synchrony with a computer metronome. This asymmetry reflected the emergent ‘leader’ and ‘follower’ roles of participants in the finger-tapping task such that leaders demonstrated stronger frontal alpha suppression than followers⁵⁷. EEG source localization methods have localized neural synchrony during tasks of motor synchrony to regions associated with the putative mirror neuron system (pMNS)⁵⁶. However, it will be important to investigate neural signatures of the other types of behavioral alignment that emerge during more naturalistic social interactions (such as conversation).

Reciprocal interaction.

Dyadic reciprocal social interaction requires social partners to explicitly take on complementary and alternating roles throughout the course of an interaction (such as initiator and responder, speaker and listener or sender and receiver). Although past work examined each of these roles separately from a third-person perspective, recent second-person neuroscience studies have begun to elucidate the neural correlates of these roles during a social interaction. This has included investigating how partners establish joint attention, how information flows between sender and receiver and how previous interaction influences subsequent interactions.

Joint attention.

Joint attention occurs when one person initiates a bid for joint attention, the other responds to that bid, and both hold the intention to share attention. Initial studies aiming to understand the neural correlates of this foundational social ability took a third-person ‘observer’ approach (for review see⁵⁸). In these studies, participants solely played the role of responder and designs focused on components of joint attention (such as responding to another person’s shift in gaze). These studies implicated brain areas that had previously been associated with gaze processing (such as STS) in joint attention⁵⁸. Novel second-person approaches, however, have highlighted a more complex network of brain regions associated with mentalizing, reward processing and attention as being involved in reciprocal joint attention, with shared and distinct regions being important for the initiator and responder roles^30,59–64 (for review see⁶⁵). Further, dual-brain studies have demonstrated neural coupling between social partners during a joint attention task within the right TPJ¹¹.

Communication.

In a conversation, social partners convey and interpret messages to and from their social partner through both linguistic means and nonlinguistic means (such as gestures). Studying these dyadic processes during interaction is important for the identification of key properties of communication such as speaker-listener coupling, turn-taking, and alignment^66,67. Although traditional cognitive neuroscience methods can be used to examine the brain responses to receiving a message or sending a message, dual-brain approaches are necessary to examine the coupling and mutual influence between social partners’ brains. For example, second-person sequential dual-brain studies have examined direct communication between romantic partners in which the actions of one partner were intended directly for the other^3,4. Using Granger causality or dynamic multivariate decoding methods, these studies have shown that activity within the sender’s brain can predict that of the receiver’s with a temporal delay, but only if the sender and receiver are from the same interactive dyad^3,4. These studies suggest a possible neural resonance (or similarity in neural activity patterns) between sender and receiver that is important for communication. In a study that examined emotion sharing, the time delay between the sender’s neural activity and the peak similarity of the receiver’s neural activity was reduced over the course of several rounds of emotion sharing, suggesting that the partners were building up shared emotional representations during communication⁴. Such processes may also be important within a learning or a clinical context. For example, in a learning context higher inter-brain EEG coherence across students within a classroom was associated with greater engagement in the class as reflected in student course ratings and this inter-brain coherence was greatest between pairs of students who engaged in eye contact⁵⁵. In a clinical context, alterations of inter-brain coupling that could be detected using dual brain neuroimaging in patients with borderline personality disorder (BPD) characterized by interpersonal conflict and unstable relationships were normalized in remitted patients who had received psychotherapy that targets their social interaction difficulties¹¹.

Social decision-making.

Real-world social interactions require an individual to use previously-acquired information about a social partner to iteratively predict their subsequent actions. Second-person social decision-making studies examine this process by contrasting conditions in which a participant plays strategic games with a human social partner to those in which they play against a computer using both dual- and single-brain methods. Initial studies demonstrated that strategizing against a human leads to a greater engagement of mentalizing network regions (particularly the medial prefrontal cortex (mPFC)) than playing against a computer^68–70. Subsequent studies identified a unique role of the TPJ in predicting an opponent’s actions^71,72 and demonstrated that TPJ activation and the synchrony of activity within the TPJ between social partners can be modulated by the perceived “socialness” of the agent (that is, whether the interaction is face-to-face⁷³ or whether the interaction occurs with a computer compared to a contingent robot compared to a human social partner⁷⁴).

Second-person social decision-making studies have also shed light on the neural correlates of two core social-interactive processes: trust and reciprocity. Regions of the reward and salience brain networks have been identified as being important in developing a model of an opponent’s actions that affects decisions of trust and reciprocity, with the nucleus accumbens playing a role in trust decisions and the anterior insula and caudate involved in reciprocity decisions and feedback learning (for review and meta-analysis see⁷⁵). These studies have primarily relied on iterative trust games in which an investor decides how much money to share with a trustee (a measure of intention to trust) and then the trustee decides how much to return (a measure of reciprocity). The use of iterative, multi-round games provides the ability to develop computational models of how players build up models of their opponents’ intentions and use these models to adjust their own and predict their opponents’ actions. In pioneering work using a hyperscanning (simultaneous dual-brain) approach, one study compared neural signals from the caudate of the trustee and the cingulate of the investor during trust decisions⁶. Over the course of the trust game, activity within the caudate of the trustee shifted: the peak response initially occurred after each repayment amount was revealed but, as the game progressed, the peak response began to occur before the repayment amount was revealed, reflecting a shift in the involvement of this region in the response to the investor’s actions to its involvement in the prediction of the investor’s actions.

A significant strength of two-person exchange games is the ability to quantify how interpersonal dynamics during the two-person exchange can predict behavior and distinguish between neurotypical and atypical populations^7,8,76–78. For example, in a prisoner’s dilemma game in which players’ choices to cooperate with or defect from their social partner affect monetary gains granger causality and graph theory methods were used to identify patterns of correlated activity between players brains that would predict which pairs adopted a non-cooperative strategy⁷⁹. Further, during an iterative trust game the sophistication of a single player’s model of their social partner (roughly indicative of the extent to which the player can engage in recursive mentalizing) differed depending on whether the social partner was a neurotypical control or a participant with a diagnosis of BPD⁷⁶, demonstrating that interpersonal dynamics inherent to the dyad affect cognitive and neural profiles of social interaction.

Although these neuroeconomic studies were the first to bring a truly second-person perspective to social neuroscience, they were limited in the types of competitive interactions examined, many of which are not characteristic of most everyday social interactions. Most of these studies involved competing or cooperating with a partner for personal monetary gain rather than for the social rewards inherent in everyday social interaction. Although more recent studies have included more real-world scenarios, such as participants receiving helpful information from an advisor⁸⁰, they have not yet included dynamic and unconstrained social interactions. Doing so will provide a more comprehensive and ecologically valid characterization of the neural computations of social interaction.

Group dynamics.

Social interactions often include more than a dyad and these group dynamics are thought to influence neural processing. One example of this is the feeling of exclusion from a group. Second-person single-brain studies have examined how the brain responds to feelings of inclusion or exclusion from a group primarily through ‘Cyberball’ games in which participants engage in a virtual ball-tossing game with other players who then stop throwing to them during exclusion periods⁸¹. By contrasting exclusion with inclusion these studies cannot isolate the effect of social interaction but instead demonstrate a role of emotional processing regions, including the insula, anterior cingulate, mPFC and precuneus, in perceptions of exclusion from a group (for review see⁸¹). A second-person fMRI study has shown that similar emotion processing regions are engaged and functionally connected with mentalizing network regions when participants experience public embarrassment because they believe they failed at a cognitive estimation task in front of a panel of peers⁸². Another study has shown that greater functional connectivity within the mentalizing network during social exclusion in a cyberball paradigm is negatively associated with the density of the participant’s real-life social networks (defined by how many friends they have, but also whether their friends are friends with each other)⁸³. Thus, integration between mentalizing and arousal systems may underlie feelings of real-time group rejection or embarrassment.

Another second-person study that has made use of real-life social networks has demonstrated that the responsiveness of reward-related neurocircuitry during a joint attention game is related to the social network size of the participant⁸⁴. Further, using inter-subject correlation (sequential dual-brain) methods, another study has shown that the neural responses of individuals that are closer in a social network are more similar to each other than to those that are more distant in the network⁸⁵. Although this study was not a second-person neuroscience study (because participants were simply watching the same movie with no perceived social interaction), this line of research could be extended to enable simultaneous dual-brain approaches to be used. These might not only take into account individual-based measures, but also difference scores between social partners on interpersonal competence, such as the ability to effectively interact with others and maintain friendships, in order to investigate whether those predict interaction success.

Social cognition in interaction

Across studies of mutual engagement, reciprocal interaction and communication, second-person neuroscience findings have challenged the traditional view of how the brain infers and reasons about others’ mental states, including their goals, intentions, and beliefs. The so-called mentalizing network (Fig. 2) was initially defined based on third-person ‘observer’ tasks that typically involve explicit mentalizing, in which participants are asked to make an explicit judgment based on a mental state (one’s own or another’s; for review see¹). These mental state reasoning conditions are contrasted with non-mental reasoning; for example, reflecting on social (but not mental) aspects of a person (Fig. 2). A seemingly distinct system, the putative mirror neuron system (Fig.1) has been shown to be engaged when participants make inferences about a person’s intentions or goals based on their actions and, because these regions are also engaged during execution of the same action or goal, this system is sometimes thought to support a simulation process that allows for understanding of other’s action goals⁸⁶. In traditional models of action understanding, these systems therefore support distinct computations, with the pMNS supporting inferences related to the ‘how’ (kinematics) and ‘what’ (goal) of the action and the mentalizing system supporting inferences related to the ‘why’ (explicit reflection on intentions)^87–89. Second-person neuroscience studies have challenged this view in two ways.

First, in contrast to third-person studies, second-person neuroscience studies have demonstrated that regions of the mentalizing network are recruited during perceived real-time social interaction even when no explicit mental state demands are present^{22,28,30,31,63} (Fig. 2). For example, hearing speech (with no mental state content) from a live (not recorded) social partner or engaging in a gaze-based interaction with another person engages regions overlapping with those involved in explicit mental state reasoning^{22,30,31, 63}. Further, making non-mental judgments about a real online peer (such as matching the fact that the peer has a kite to it being a windy day versus a rainy day) engages regions of the mentalizing network more than similar judgments about a fictional character²⁸. When interacting with a peer these mentalizing network regions show similar levels of activation for both mental and non-mental judgments (unlike the activation pattern seen in the character condition)²⁸, highlighting the importance of studying social processing within a second-person framework. An important question for future second-person neuroscientific research will be whether activation of the mentalizing network for non-mental state reasoning about a social partner is driven by spontaneous mentalizing or by a more primary computation critical to social interaction that is not typically elicited in ‘observation’ studies of social processing, such as coordinating attention or perspectives with a real-time social partner. Support for the latter idea comes from studies of human infants and nonhuman primates with relatively impoverished mental state reasoning abilities that demonstrate sensitivity of this network to social interaction^2,65,90–94.

Second, second-person neuroscience studies have questioned the distinction between (that is, the independent engagement of) the mentalizing and mirror networks^3,20,95–97 (Fig. 1). For example, when viewing communicative bids, functional connectivity between regions of the mentalizing network and mirror systems is increased²⁰. Further, during an interactive game of charades, inter-brain functional connectivity is seen between the mirror and mentalizing networks such that neural activity within the gesturer’s pMNS can predict activity within the guesser’s pMNS and within a region of the mentalizing network (the vmPFC). Why might these networks act in concert during social interaction but not during observation? Studies investigating inhibition of spontaneous mimicry give some insight by suggesting that mentalizing regions (particularly the mPFC) may act to control the automatic shared representations between social partners (reflected in pMNS activation)^98,99. Indeed, the research described in this article describes greater resonance, or shared representations, during social interaction. However, this is just one possibility and indeed is incongruent with findings of concurrent engagement of mentalizing and mirror networks in interaction⁹⁵. Further, these mentalizing network interactions likely extend beyond the mirror neuron network as second-person studies have highlighted concurrent activation and functional connectivity between reward or arousal and mentalizing networks during interaction^28,63,82,100.

In sum, while data are still limited, second-person neuroscience suggests differences in network organization and/or different computations of nodes within those networks during social interaction and social observation. However, the extent to which functional brain organization fundamentally differs when participating in interaction or observation remains an important question. We propose that pursuit of this question may result in a redefinition of the boundaries and weights of current network structure such that a ‘social interaction network’ may emerge as some combination of nodes across these so-called distinct networks (Fig. 1).

Future Directions

Although second-person neuroscience has begun to shed new light onto the behavioral and neural mechanisms of social interaction, this field of research is still in its infancy. We see several particularly promising areas of research that could be extended in the future.

Interaction dynamics.

Even within dual-brain studies task set-ups are typically fairly constrained and predictable, which does not allow for an understanding of the interaction dynamics that emerge across the course of a natural social interaction. EEG and fNIRS methods have allowed for more freely forming interactions and the ability to capture emergent inter-brain properties (reviewed in REF⁵³). However, many of these inter-brain analyses have been heavily dominated by a search for synchrony, or mirroring, between brains. Interpretations of synchrony, however, are limited as such synchrony may arise due to alignment between brains or due to the alignment of both brains to precise timing of the events of the interaction, which does not require social interaction. Further, neural synchrony is just one possible neural signature of dynamic, reciprocal social interaction and may not capture the distinct and complementary roles that are inherent to dyads in everyday interactions (reviewed in REFs^101,102). Greater utilization and development of analysis tools that provide a multidimensional approach to capture spatial and temporal similarities and differences across individuals during social interaction, such as inter-brain multivariate decoding⁵⁷, asymmetric phase synchrony³⁹ or graph theoretic approaches to characterize interpersonal behavioral and inter-brain organization^79,103 are needed. Further approaches that examine the reciprocal influence between brains within freely forming interactions (such as granger causality analyses³ or computational modeling¹⁰⁴) will elucidate how brains construct, update, and influence their own and their partners’ joint representations of the interaction in real time. Importantly, these methodological advances should be coupled with greater theoretical attention to how the emergent properties of an interaction would be reflected in neural activity. Finally, the question of how inter-individual differences at the level of brain structure and function across two interaction partners are related to success or impairment in real-world social interactions and social-interactive impairments requires greater study (Box 2). Developing methods to allow for greater reciprocity and unpredictability will be especially important in understanding disorders of social interaction, such as autism spectrum disorder (ASD), in which the unpredictability of the social environment may be a key factor in the observed social-interactive difficulties. In one example of this approach, a study investigating face-to-face spontaneous conversation in adults with ASD has demonstrated hyper-connectivity within social processing brain regions during spontaneous conversation, as compared to the connectivity of these regions during vocal repetition¹⁰⁵.

Box 2: Second-person neuropsychiatry.

Although second-person neuroscience focuses on neurotypical individuals, the second-person approach is also critical for our understanding of psychiatric disorders such as autism, schizophrenia, social anxiety, depression and personality disorders, which can be construed as disorders of social interaction. More specifically, experimental tasks that focus on social interaction and are high in ecological validity are likely to be more sensitive in their objective assessment of those social impairments that are most therapeutically relevant¹²³ and, when combined with computational approaches, may be used to develop more sensitive neural signatures of atypical social interaction^{7,8,76,78,124}. Furthermore, a mechanistic understanding of social impairments will require us to study (at least) dyads of individuals, given social impairments appear to be affected by the characteristics of the individuals involved. For example, social impairments have been shown to be less pronounced (or even completely absent) when two people with autism interact with each other than they are in a situation in which one person with autism and one person without autism interact. This clinical observation might be related to evidence indicating that the empathy shown by individuals is greatest when it is directed to others with autism¹²⁵. Conversely, individuals without autism find it easier to infer the mental states of individuals without autism than those of individuals with autism¹²⁶.

These findings suggest that social impairments might be more closely related to (dis-) similarities between interaction partners than they are to the characteristics of each individual, perhaps because an interaction partner’s behavior can be more easily and accurately predicted when he or she is similar to oneself. This has become known as the social interaction mismatch hypothesis^50,127 and has been the foundation for the development of group psychotherapy for adults with high-functioning autism¹²⁸. Future research may help to further address this issue by systematically manipulating interpersonal differences across dyads in order to assess their impact on social interactions and their relationship to brain structure and function of both interaction partners.

Network approaches.

We also argue for a greater focus on how brain network dynamics are altered during participation in social interaction. Although most of the studies reviewed above have focused on regional activation or synchronization in the activation of different regions, understanding the functional relevance of a region requires understanding both its role within a broader network and the inputs it receives from other nodes of the network and from regions of other canonical brain networks. This approach could extend to examine inter-brain network connectivity – that is, examining how regions and networks within one brain couple to multiple regions or networks within another person’s brain. Importantly, this network organization may dynamically shift over the course of an interaction or repeated interactions, such as in psychotherapeutic settings or the development of relationships.

The importance of development.

The distinction between interaction and observation emerges early. By their first months of life infants are sensitive to the temporal contingency present in an online social interaction¹⁰⁶ and demonstrate biobehavioral synchrony with caregivers¹⁰⁷. At 9 months, infants learn language better from a live (compared to recorded) actor¹⁰⁸ or in the presence of a peer¹⁰⁹. At two years of age toddlers synchronize their own actions to those of a live human partner but not to those of a machine¹¹⁰. Recent evidence suggests this behavioral synchrony is reflected in neural synchrony between infant and caregiver, particularly during direct gaze in live contexts¹¹¹. Despite the importance of social interaction in shaping development, relatively little neuroscientific work has taken a developmental second-person neuroscience perspective (for review see⁹²). A second-person neuroscience approach to development will be crucial to understand the development of affiliative bonds and the mechanisms by which brain regions specialize for specific social-interactive computations. For example, it is possible that the explicit mentalizing abilities within the dMPFC and TPJ may emerge from a more basic functional role of these brain regions during social interaction. Longitudinal studies could examine the extent to which sensitivity of these regions to social interaction is predictive of later explicit mentalizing abilities and neural functional specialization. Investigating the behavioral and neural mechanisms of social interaction across development will also need to go beyond the study of behavioral synchrony to also focus on neural mechanisms underlying how individuals generate complementary actions¹¹², coordinate attention¹¹¹, and take on distinct communicative roles within naturalistic social interactions.

Relationships.

Although second-person neuroscience has brought greater ecological validity to the study of social interaction, studies thus far often overlooked the critically important dimension of relationships between dyads. From extensive literature in developmental and social psychology, we know that individual differences in relationship quality robustly predict social and emotional abilities. Further these individual differences in relationships influence behavioral synchrony between partners¹⁰⁷. More recent second-person neuroscience studies have revealed greater neural synchrony between romantic couples compared to strangers during social gaze¹⁰ and target detection¹¹³. Further, hand holding from a romantic partner acts to reduce amygdala reactivity³⁶ and increases neural coupling between partners during pain administration¹¹⁴. Among partners, social closeness modulates the reward system when partners share monetary gains¹¹⁵ or share attention on emotional videos⁶⁴. Finally, activation in regions associated with social perception is greater when engaging in a joint attention game with a familiar than unfamiliar person⁶¹. Future second-person neuroscience studies should therefore consider how interpersonal relationships influence neural dynamics between interacting brains.

Concluding remarks

Since the initial call for a second-person neuroscience we have gained striking new insights into the workings of the social brain in interaction. Findings are converging on a set of brain regions across mentalizing, reward, and mirror neuron systems that play key roles and interact closely in order to support social behavior in ecologically valid contexts (Fig. 1). Preliminary evidence indicates that studying two brains in interaction not only allows us to ask new questions about how social behavior is realized at an interpersonal level and how this is supported by inter-brain neural activity, but also how social interaction difficulties might be related to alterations of inter-brain rather than single-brain network activity. In spite of these ground-breaking insights and the translational potential of second-person neuroscience, the approach remains in its infancy and there are many questions that will require even greater innovation, both methodologically and conceptually, to answer. We have highlighted a few that we believe will be instrumental in further advancing the neuroscientific knowledge of what is at the core of being human: social interactions.