Beyond speaking: neurocognitive perspectives on language production in social interaction

Abstract

The human faculty to speak has evolved, so has been argued, for communicating with others and for engaging in social interactions. Hence the human cognitive system should be equipped to address the demands that social interaction places on the language production system. These demands include the need to coordinate speaking with listening, the need to integrate own (verbal) actions with the interlocutor's actions, and the need to adapt language flexibly to the interlocutor and the social context. In order to meet these demands, core processes of language production are supported by cognitive processes that enable interpersonal coordination and social cognition. To fully understand the cognitive architecture and its neural implementation enabling humans to speak in social interaction, our understanding of how humans produce language needs to be connected to our understanding of how humans gain insights into other people's mental states and coordinate in social interaction. This article reviews theories and neurocognitive experiments that make this connection and can contribute to advancing our understanding of speaking in social interaction.

This article is part of a discussion meeting issue ‘Face2face: advancing the science of social interaction’.

1. Introduction

As humans we have the basic need to connect with each other, to share our ideas, thoughts, and feelings. Language is the central instrument to meet this need. When we speak to communicate, we typically engage in social interaction with others. Yet mainstream psycholinguistic research mainly investigates language production in settings devoid of any form of social interaction. This approach corresponds to an understanding of language as an instrument of the human mind for processing information and structuring thinking (versus as an instrument enabling social interaction, for discussion see [1,2]). Research in this tradition has provided the basis for mechanistic theories of language production, which dissect into subsystems the processing steps required for an individual speaker to get from forming a communicative intention to generating overt speech. While these theories have been instrumental for understanding the cognitive processes underlying isolated language production, it is vital to scale up to settings in which language is typically produced, namely, in social interaction (see [1–3]).

The core processes of language production are shaped by the unique demands social interaction places on our cognitive system. Moreover, the demands of social interaction are met by engaging additional cognitive processes and neural structures; those that support social cognition and interpersonal coordination. Theories of language production need to be embedded in a larger understanding of the human cognitive system and the mechanisms that support successful social (inter-)actions. Recent advancements in this direction cross disciplinary boundaries and adapt methods from experimental psychology, social psychology and cognitive neuroscience, and connect our understanding of how humans produce language to our understanding of how humans gain insight into other people's minds and coordinate their actions in social encounters.

The goal of this article is to pull together insights and perspectives on how neurocognitive processes underlying speaking coordinate with, and are shaped by, processes supporting social cognition and social interaction. We will focus on three demands that social interaction places on the language production system: (i) the need to coordinate speaking with listening, (ii) the need to coordinate own behaviour with the conversational partner's behaviour as part of a joint action, and (iii) the need to flexibly adapt speaking to a particular conversational partner.

2. Language production in social interaction: more than speaking

Speaking in a social interaction is something we do every day and without thinking much about it. Yet, as simple as it may appear, it is a highly complex task. Interlocutors need to accomplish the sheer act of producing speech, starting with forming the intention to communicate, generating the message, selecting the right words to express this message, encoding it as grammatical and phonological information, and finally, translating this into motor commands that execute articulation ([4]; for recent review see [5]). Speaking is also commonly accompanied by hand gestures, facial displays, or other bodily cues that need to be integrated, both in time and content, with the spoken message (e.g. [6]). What is more, in a social setting, additional demands are placed upon a speaker's cognitive system that go beyond the core processes of language production. These demands are met by cognitive mechanisms and neural structures supporting social cognition and joint action coordination.

3. Integrating speaking and listening

One source of insights into the nature of the demands spoken social interactions place on speakers have been ethnomethodological studies that describe and analyse patterns of behaviour through observations of natural occurring spoken interactions (e.g. [7–9]). A central topic in this research tradition has been turn-taking, which describes the alternation of speaking turns between different speakers (e.g. [9]). This characteristic of conversational speech comes with consequences for the cognitive system and has informed and inspired experimental psychologists to investigate the cognitive mechanisms that could support such behaviour (for discussion see [10]). As we will see, the human cognitive system seems well equipped to meet this challenge.

One basic consequence that follows from the turn-taking structure of conversation is the tight integration of speaking and listening: processes of language production become interwoven with processes of language comprehension—and may happen in parallel. Crucial insights how this may be achieved have come from connecting language production to general principles of human cognition: cognitive theories of action planning and action execution have proposed that the perception of action and the execution of action share common representational structures [11]. Mirror neurons in the brain that fire both in response to viewing an action and in response to executing an action, provide a neuroscientific basis for this mechanism (e.g. [12–14]).

In language, as well, the argument has been made that the representations underlying language comprehension and those underlying language production are of the same format (e.g. [15–18]). In fact, this principle has been proposed to be a fundamental building block of language use in social interaction [3]. According to this proposal, speaking is facilitated by processing the partner's utterance: via a simple priming mechanism, it is efficient for speakers to re-use linguistic structures (e.g. the same lexical expression or syntactic construction) previously used by their conversational partner. This principle has been offered as a mechanistic explanation for the frequently observed behaviour of interlocutors to entrain on their linguistic behaviour and become more similar to each other over time. Indeed, a large body of literature has demonstrated that conversational partners converge over the course of an interaction on different levels of representation, ranging from the phonetic level (e.g. [19]) to the use of similar speech-accompanying gestures (e.g. [20]). What is more, such entrainment of linguistic representations is said to form the basis for mutual understanding [21].

(a) . Coordination of spatial and temporal neural activity between speakers and listeners

Corresponding to the assumption of a shared representational format for comprehending and producing language, on a neural level, neural circuitry underlying language comprehension and production have been shown to overlap (e.g. [22–24]). This provides the basis for hypothesizing that neural states of communicating individuals should become more similar to each other as they activate similar representations while speaking and listening. This was first demonstrated by a seminal functional magnetic resonance imaging (fMRI) study comparing the brain activity of a speaker telling a story to the brain activity of several listeners hearing the story [25]. By relating the two patterns of neural activity to each other the authors were able to demonstrate that speaker's and listeners’ brain activity coordinated both temporally and spatially. Temporally, listeners' brain activity mainly followed speakers’ brain activity with a delay of several seconds as the signal passed from speaker to listeners. A delay in neural coordination has since been also found in other dual-brain studies and has been related to different levels in the processing hierarchy (e.g. [26,27]) and to communicative success (e.g. [28]). In Stephens and colleagues' study [25] listeners’ brain activity preceded speaker's brain activity in some cases, which the authors interpreted as anticipatory activation (for more recent findings on the role of prediction in speaker–listener coordination, see [29]).

Spatially, interpersonal neural coordination was observed in corresponding brain areas in speakers and listeners, providing support for the representational similarity between these two processes. Interpersonal neural coordination is associated to communicative success: the degree of spatial coordination between speaker and listener correlated with listeners' story comprehension, and, when the story was told in a language the listeners could not understand, no significant coupling was observed.

A coordination between speakers and listeners may not exclusively rely on activating identical brain areas and processes: in a similar storytelling setting, a dual-brain electroencephalography (EEG) study reports coordination between speaker's and listeners’ electrophysiological activity involving different locations across the skull ([30]; for comparable findings of coordination between different brain areas in nonverbal interaction, see [31]). This study's design explicitly excluded the possibility that neural coordination between speakers and listeners results merely from processing a shared perceptual environment. Instead neural coordination corresponds to processing the content of the communicated linguistic information: the recorded narrations of two speakers were superimposed on each other and listeners were instructed to attend to either one or the other speaker. Listeners' EEG coordinated with the attended speaker's EEG significantly more than with the unattended speaker's EEG.

This is in line with more recent contributions, which have argued that mutual understanding is more than a one-directional flow of information, encoding a message on the one side and decoding the message on the other side. Instead, mutual understanding is actively generated and involves dynamic adaptation and the deliberate building of a shared conceptual space between speakers and listeners [32–35]. That mutual understanding goes beyond producing and receiving speech signals (and activating corresponding representations) is also apparent by dual-brain studies indicating a speaker–listener coordination in brain areas that extend beyond the core language network and engages additional cognitive processes, such as working memory [36], shared attention [37] and social cognition [25,30,31,38].

Studies that relate neural activity across multiple individuals have pioneered methodological advances that allow investigating and quantifying neural coordination between two or more brains (e.g. [39–41]). Multi-brain studies can offer insights that go beyond the individual mind and brain, and investigate the dynamic coordination between two or more communicating individuals. Such an approach can further theory-building and theory-testing, and advance our understanding of language in social interaction (see [42] for a recent review of this research area).

(b) . Coordination of behaviour and cognition during turn-taking

Another feature of turn-taking that has sparked a rapidly growing body of experimental research, is the speed by which speakers and listeners alternate. A starting point for these investigations has been the observation that, across cultures, the average inter-turn interval, the gap in between one speaker's speaking turn and the next speaker's turn, is around 200 ms [43,44]. This is in stark contrast with psycholinguistic laboratory studies on language production in which the production of a word (let alone a sentence) takes already 600 ms (e.g. [45,46]). This puzzle has been addressed by assuming a cognitive architecture that can comprehend and produce language in parallel and relies on predictive mechanisms that allow the next speaker to prepare their speaking turn once the gist of the incoming turn can be predicted and to launch it once the current turn is completed (e.g. [47–52]).

A seminal study of turn-taking manipulated how early an interlocutor can predict how their conversational partner's speaking turn will end and hence, how early the planning of a response can begin [49]. Resorting to recordings of electrophysiological activity in the interlocutor's brain the researchers were able to demonstrate that speech planning begins already during the conversational partner's speaking turn, and begins earlier when the gist of the partner's speaking becomes predictable earlier in the sentence. Comparable conclusions have been reached by Corps et al. [51]. Noteworthy is also a recent study by Bögels [50] which replicated the finding that conversational partners can and do start planning their turn already about one-third into the partner's utterances in a more natural setting with a mix of pre-scripted and spontaneous turns.

While there seems to be agreement in the field that speech planning can and does occur in parallel to comprehending the partner's speech, it is not clear to what degree the two processes may conflict with each other. If a person begins to plan their response while their partner is still speaking, this may reduce the depth with which the partner's utterances can be processed. First investigations have addressed the costs of early planning on processing load (e.g. [53,54]). The theoretical basis of these investigations is that preparing speaking while simultaneously processing incoming speech taps into the same mental representations, limiting the resources that can be dedicated to each. Yet a recent neurosurgical study using intracranial electrocorticographical recordings comes to the conclusion that speech planning is functionally and anatomically distinct from articulating speech or listening [55]. This study temporally isolated the initiation of speech planning processes by varying the point at which critical information is presented in the partner's turn (cf. [49]), and then compared the observed neural activity to neural activity observed during episodes of listening and speaking. The identified signatures of speech-selective planning were then verified in unconstrained natural dialogue. If indeed, as this study suggests, separate modules are dedicated to speech planning and speech processing, this could mean that the two processes can run in parallel without impeding each other.

A recent analysis of corpus data of naturalistic spoken conversations challenges the proposal that preparing speech production during comprehension is pervasive in everyday conversation [56]. An analysis of the length of speech segments suggests that speaking turns may often not be long enough to allow their partners to complete, or even initiate, the planning of their response. According to the authors, further strategies may be in place to assure timely turn-taking. For example, interlocutors may not respond to the immediately preceding speaking turn, but to a topic introduced earlier in the conversation. Essentially, interlocutors may each be contributing to the conversation but may not be directly refering to each other's contributions. This proposal fits with the proposal that conversational partners may strive for a level of understanding that is good enough for the current purpose [57,58], and which may sometimes be built on an illusion of understanding [59].

Lastly, another open question arising from this research field is which cues speakers rely on for predicting their partner's turn end so that they can launch their response. This response needs to be not only appropriate in content [60] but also appropriate in time (e.g. [61–66]) as the time interval that elapses between two speaking turns carries pragmatic information (e.g. making certain replies more likely than others, see [67]). Different linguistic and paralinguistic cues have been demonstrated to mark upcoming turn ends, for example, syntactic and lexical markers [68] or prosodic properties of speech [62], and may be used to facilitate turn transitions. Lastly, it has been proposed that the inherent temporal structure of conversation itself may facilitate the timing of speaking turns [66,69]. For example, a conversational partner may rhythmically entrain on the current speaker's syllable rate. Applying a dynamical systems perspective, the temporal coordination observed during turn-taking may thus be described by self-organizing dynamics between coupled oscillators. Similar proposals have also been made for non-verbal types of social interaction, such as finger tapping [70], or body sway [71].

While theory-building in this research field continues, the question of how speaking and listening become so tightly interweaved in social interaction remains valid and is an important puzzle piece to theories of language production in social interaction. Next we will turn to another demand social interaction places on language production, namely going beyond actions of individuals and merging these into a joint action.

4. Representing the own and the partner's actions

Language production, when used for social interaction, is not simply an individual process. Speaking becomes a coordinated, joint action between two or more socially interacting individuals (e.g. [1,3,18,72–74]).

Over the last decades, this perspective has gained from cross-vertilization from research on the perceptual, motor, and cognitive basis of joint actions (for overview, see [75]). Joint action has been defined as an action for which two or more individuals coordinate their individual behaviour in time and space to achieve a shared goal [76]. One prominent theoretical proposal has been that coordination between multiple individuals relies on an individual's capacity to represent the joint task, to predict the outcome of the own and the partner's contributions, and to integrate these [77]. Empirical support for this comes from studies on action planning in a joint spatial compatibility task, demonstrating that co-acting individuals represent not only their own actions but also their partner's actions (e.g. [78]; for recent meta-analysis, [79]).

These theoretical assumptions have recently been applied also to joint actions involving speaking. A seminal study by Baus et al. [80] recorded electrophysiological activity (EEG) during a joint picture naming task in which two task partners took turns naming pictures. Using electrophysiological signatures of lexical frequency, the authors were able to demonstrate that speakers engage in lexical processing not only when it is their turn to name a picture but also when it is their partner's turn to name the picture. The authors propose that representing the partner's speaking enables interlocutors to predict their partner's verbal behaviour. This is in line with recent mechanistic accounts of dialogue which propose that predicting the partner's verbal behaviour engages the own speech production system and serves to facilitate language processing and the coordination of utterances across speakers both in their timing and content (e.g. [18,81,82]).

An ongoing debate in the field of joint action research is the nature of these shared task representations. While some propose that the partner's specific action is simulated and represented on a level of detail similar to one's own action [83,84], others have proposed that represented is not the partner's action itself but merely the fact that the partner acts [85–89].

Recent speech production studies investigating language production in a joint action framing can be applied to this debate; albeit providing mixed evidence. First evidence in favour of the proposal that the partner's speaking is represented as detailed as own speaking comes from the study by Baus et al. discussed above: since the neurophysiological activity elicited by the partner's naming were sensitive to lexical frequency this suggests that at least this aspect of the partner's action was represented. Further evidence comes from a study by us [90] in which we built upon a well-known semantic interference effect characterized by slower picture naming when successively naming pictures of the same semantic category [91]. Placed in a joint picture naming setting in which two task partners take turns naming pictures, this study was able to demonstrate that semantic interference can be elicited not only by own prior naming of semantically related pictures but also by the partner's naming of semantically related pictures. Importantly, partner-elicited semantic interference was not based on hearing the partner name the picture: the same pattern of results emerged when the partner's naming was masked through noise-canceling headphones (Kuhlen & Abdel Rahman, experiment 3), and when task partners were naming pictures located in different rooms (Kuhlen & Abdel Rahman, experiment 2). This pattern of results supports proposals that the partner's utterances are simulated based on the beliefs about the partner's action and, crucially that this simulation is at the level of seeking lexical access on behalf of the partner. Consequently, the amount of interference experienced increases when subsequently naming semantically related pictures. Converging evidence for this finding comes from a different team of researchers who applied the same paradigm also in a joint picture naming setting [92] and from a mega-analysis, combining several studies within this setting [93]. These studies align with accounts of joint action which suggest that the partner's actions are co-represented and suggests that the partner's speaking can have profound and lasting effects on own speech production.

Other studies, however, find little evidence for the claim that the partner's utterances are co-represented—at least not to the level of seeking lexical access on behalf of the partner. For example, a study by Gambi et al. [94] found that speech production was affected by the knowledge that the speakers' task partner was concurrently also naming a picture. But speakers were not affected by whether the picture was related or unrelated to the pictures the partner had to name [94]. Similar conclusions have been reached by other studies employing variants of joint picture naming tasks [95,96].

Whether a partner's utterances are represented, and to what level of detail, may depend on the nature of the social setting, the role the task partner plays in the interaction, and the saliency of their actions. For example, in a follow-up study on the cumulative semantic interference effect in a joint picture naming setting we were not able to find partner-elicited interference [97]. In this study we recorded electrophysiological activity of the speakers, which required a large number of trials and, to avoid the confound of hearing the partner name the picture, required that the two task partners completed the joint picture naming task sitting in two separate rooms. This may have weakened the effect: a mega-analysis accumulating six experiments on cumulative partner-elicited semantic interference suggests that partner-effects may be particularly pronounced after just having interacted (in physical presence) with the partner [93]. How a partner's actions are represented may also depend on the identity of the task partner: studies on human–robot interaction indicate that a robot's verbal actions may be represented on the conceptual, but not on the lexical level ([98]; more on conceptual alignment with robot partners [99]). Lastly, literature from research on other types of joint action suggests that the goals of the individual co-actors, whether they complement or compete with another, may influence partner co-representations [100,–102]. Future studies will need to further elucidate how these factors affect the degree to which the partner's actions are represented.

The body of research reviewed here has scaled up controlled studies of language production, specifically on the process of lexical selection, to joint action settings in which two speakers take turns speaking. Yet the employed picture naming tasks provide, at best, only very minimal communicative purpose for the speaker's utterances. Speaking with the goal to communicate is arguably an important aspect of language use in social interaction. What is more, conversational speech typically involves more than producing isolated words. Instead these are most often embedded in a larger discourse context. These aspects may shape the processes by which individuals produce language. This is indeed suggested by a recent joint picture naming study in which we embedded a picture–word interference task in a conversational turn sequence [97]. In this study, two participants played a simple game that involved naming and matching pictures displayed on their playing cards: in a given trial, the first player requests the information which cards should be placed on top of each other (e.g. ‘which card comes on apple?’) thereby producing the distractor word (apple). The second player then responded by naming the appropriate card thereby naming the target picture that was either semantically related or unrelated to the distractor (e.g. ‘pear’ versus ‘car’). Hence, the experimental procedures essentially resembled a classic picture-word interference task (compare e.g. [103–105]), but placed in a social and communicatively meaningful exchange. The large body of literature on picture-word interference in single-speaker settings would predict that speakers should experience semantic interference, that is slower naming latencies, when hearing a semantically related distractor word immediately prior to naming the target picture. Yet, in this communicative setting, no interference was observed. What is more, when the communicative game was slightly altered to increase participants' focus on the conceptual relationship between target and distractor (e.g. 'what matches apple?'), the partner's utterance, when presented 650 ms before the onset of the target picture, even had the potential to prime and facilitate subsequent speaking. We propose that, in communicative settings, the partner's utterances are considered in relationship to own utterances and can thus facilitate own speaking. In the framework of picture–word interference, this means a shift towards processing the distractor words in its conceptual relationship to the target word (instead of focusing on its lexical representation), thus moving from lexical interference to semantic priming.

Along similar lines, other experiments embedding picture-word interference in a larger discourse context have also reported decreased semantic interference ([106,107], experiment 1). Yet, the mechanism behind this effect is not quite clear. While we have argued that the decrease of semantic interference results from a greater emphasis on conceptual processing and enhances semantic priming [97], Shao & Rommers [107] suggest the preceding discourse context constrains semantic processing by narrowing down the number of possible lexical candidates (but see [108,109], discussed below, for empirical evidence against this degree of flexibility). Tufft & Richardson [106] propose a non-linguistic origin for decreased semantic interference, resulting from the possibility of offloading responsibility for processing the distractor in a dual-task setting. While the precise mechanism of this effect requires further investigation, these studies suggest that the communicative nature of the social interaction can have an impact on how the partner's utterances are integrated with, and shape, own language production.

5. Adapting speaking to the conversational partner

When speaking in a social setting, utterances are adapted to the specific conversational partner. Such adaptations have been called audience or recipient design [110–112] and can shape all levels of linguistic representations: for example, the choice of a referential expression depending on the partner's perspective (e.g. [113–120]). The choice of syntactic constructions depending on information being new or already known to the addressee (e.g. [121,122]), and the clarity with which this information is articulated [122,123]. These types of flexible adaptations in language use have been suggested to be supported by the human ability to take another person's perspective and to infer their mental states [1]; an ability typically referred to as mentalizing or Theory of Mind. The ability to mentalize may be an important additional process engaged when language is produced in social interaction. In addition to engaging social cognition, adaptations to the conversational partner (and the conversational context more generally) may trigger adaptations in semantic processing. In the following we will discuss empirical evidence for each of these aspects underlying partner-adapted language production.

(a) . Mentalizing supports partner-adapted speech production

Numerous studies on speech comprehension have pointed towards the need to engage in pragmatic inferencing in order to derive from linguistic meaning intended speaker meaning (for review [124]). These inferences are typically associated with social cognition, and a person's ability to read another person's (communicative) intentions and mental states, also called mentalizing or Theory of Mind.

Also when speaking, mentalizing is routinely engaged. For example, an fMRI study investigated neural activity of speakers while engaged in a communicative game with a task partner located outside of the scanner [125]. Specifically, the study contrasted a communicative condition, in which speakers referred to objects whose identity their task partner had to guess, with a non-communicative condition, in which the addressee knew in advance which object was being referred to (thereby eliminating any need to communicate this information). Areas that social neuroscience has associated with mentalizing, most notably the dorsomedial prefrontal cortex, differentiated these conditions. The authors concluded that mentalizing is a crucial process when communicating to a conversational partner.

Similar conclusions are reached by another fMRI study in which speakers were asked to give simple spatial instructions either to a task partner located outside of the scanner (communicative condition), or for the purpose of testing the microphone (non-communicative condition; [126]). Neural activity was measured prior to speaking at a time point when speakers knew that they would be speaking with or without communicative intentionality (‘who’: partner versus microphone test), but they did not know yet which instructions they should give ('what'). Multivariate pattern analysis decoding regional activation patterns associated with a particular task set [127–129] revealed that the communicative intentionality of the setting is encoded in the ventral-medial prefrontal cortex. This brain area has been identified in lesion studies to be responsible for tailoring a communicative message to characteristics of the task partner [130]. This study supports the conclusion that mentalizing is routinely engaged during speaking in social interaction and that mentalizing becomes engaged already during preparatory processes of speaking.

The assumption that pragmatic processes shape speaking already in preparatory stages of speech production is further supported by a recent EEG study in which participants were asked to speak with the intention of producing two different types of speech acts—either for the purpose of naming the displayed object or with the intention to request the object [131]. Electrophysiological signatures, most notably a readiness potential that the authors label pragmatic prediction potential, distinguishes these two speech acts already 600 ms prior to speaking. Based on a source analysis of the observed predictive potential, which indicates the involvement of the motor cortex, the authors suggest that preparing a request engages predictions about specific action-related consequences. This study supports accounts that a speaker's communicative intentions, and possibly the anticipation of the addressees' reaction, inform preparatory processes of language production.

The above reviewed studies indicate that pragmatic processes, most notably the ability to infer and anticipate another person's intention and mental states, become engaged when speakers speak in the context of a communicative, social exchange. A recent fMRI study ties activation of the mentalizing network to the ability to adapt speaking to a conversational partner's informational needs [132]. In this study participants, located inside the scanner, gave their task partner, located outside the scanner, instructions to select objects placed on a grid, a classic referential communication game for investigating audience design (see [133–135]). In a communicative condition speakers' utterances served the purpose to instruct the task partner which object to select. This condition was compared to a noncommunicative condition, in which speakers produced the referential expressions without having a task partner. Neural activity was recorded during speech planning, which was experimentally separated from speech production by requesting participants to press a button prior to speaking. Crucially, in some trials the grid contained objects that competed with the target object (a small glass and a medium-sized glass), and these were either visually accessible, or occluded from their partner's perspective on the grid. This required speakers to specify their referring expression depending on their partner's perspective. The pattern of results suggests the involvement of two core structures of the mentalizing network, the medial prefrontal cortex (mPFC) and the temporal parietal junction (TPJ), when speakers produced speech for the purpose to communicate and when their utterances required an adaptation to the partner's perspective. These two brain areas may serve a differential role: activity in the mPFC was observed under communicative (versus noncommunicative) conditions in which speakers had to adjust their utterances to their task partner's perspective. Activity in the bilateral TPJ was exclusively observed within communicative trials and marked whether the speaker's and their addressee's perspective differed. Together these findings support theories that propose the mPFC distinguishes communicative from noncommunicative actions (e.g. [125,126]) and further suggests that the TPJ supports flexibly adaptations to the specific needs of the addressee.

One disputed question in the field has been how essential mentalizing is to speaking in social settings. While the just reviewed work suggests that mentalizing is routinely engaged already during early moments of language production, other scholars have proposed that speaking is initially planned based on the speaker's own perspective and mentalizing is employed only later in processing in case a repair is needed (e.g. [136–140]). Neuroscientific support for this claim comes from a magnetoencephalography study on language comprehension in which addressees processed referential expressions that had either been established as common ground with the speaker in a prior interaction, or that violated this common ground [141]. Mentalizing activity was only observed when speakers violated the common ground established with their addressee. The authors conclude that mentalizing is engaged as a reaction to pragmatic violations, but not for making partner-specific predictions about the partner's upcoming utterances.

While the reviewed studies on speech production suggest that mentalizing is an essential component when speakers speak with communicative intentionality and allows speakers to adapt their speaking to the conversational context, the connection between brain areas associated with speaking and those associated with mentalizing are not yet well understood. On one hand, the literature supports the view that mentalizing and linguistic processing are separate faculties: for example, speakers with aphasia show difficulties speaking while their mentalizing skills are typically intact (e.g. [142–146]); and, vice-versa, speakers with autism spectrum disorder show deficits in mentalizing while typically their linguistic abilities are intact (e.g. [147–152]). On the other hand, the here reviewed studies suggest a close integration of the ability to speak and the ability to mentalize. This implies a connection between areas supporting language production and those supporting mentalizing. Indeed, Deen et al. [153] report a small amount of overlap between language and mentalizing activations in the left superior temporal cortex. Moreover, a recent study involving three fMRI experiments reports synchronized activity between language and mentalizing networks at rest and during story comprehension [154], suggesting a functional integration of these two abilities. More work is needed, though, to understand the nature of this integration and how it plays out over the course of producing a communicative utterance. Employing more time-sensitive neuroscientific methods, such as EEG, could allow future research to address these questions.

(b) . Knowledge about the partner shapes semantic processing

Partner-adapted speaking may not only involve thinking about the communication partner's mental states and informational needs, it may also carry consequences for semantic processing. Along these lines, research on language comprehension has demonstrated that semantic processing is shaped by knowledge about the conversational partner: for example, listeners use cues available in the speaker's accents to disambiguate homonyms [155], and the speakers' voice or face can serve as a cue to the speaker's identity and inform predictions about upcoming words [156,157]. Also, knowledge about the partner's profession can shape word processing such that atypical category exemplars become more accessible [158]. These studies point towards a flexibility in semantic processing that enable adapting language comprehension to the conversational partner.

Similar flexibility may be observed in language production. In support of this proposal it has been demonstrated that semantic relationships can be constructed on the fly given a certain semantic context and that these ad-hoc semantic relationships affect speech production [159]. While objects like ‘stool’, ‘bucket’, ‘knife’ and ‘river’ are typically perceived as semantically rather unrelated, they are perceived as related when presented together with an overarching theme, in this case, a fishing trip. Once this theme is available to speakers the generated semantic relationship between these items leads to interference during lexical selection. The same principle may be at work in a conversational context. First evidence for this comes from a study demonstrating that thematic context can be elicited by information provided by a task speaker [160]. In this study formerly unrelated objects were associated to a common theme through a short narrative provided by the task partner (e.g. a narrative about going on a fishing trip). Similar to Abdel Rahman & Melinger's study [159], the thematic context generated by the partner elicited, under certain circumstances, interference in a subsequent picture naming task.

Yet other studies have pointed to limited flexibility of speakers to take the social or pragmatic context into account. In a joint picture naming study [161], speakers took turns naming pictures. In some trials, the preferred basic level term for a given picture became pragmatically infelicitous because it failed to distinguish between two objects of the same category (e.g. naming the picture of a shark ‘fish’ in the presence of another fish). Yet, when words phonologically related to this basic level term were presented as distractors in a picture-word interference paradigm, they interfered with speech production. This indicates that lexical entries for the basic level term were covertly activated during speech production even though they were pragmatically infelicitous, suggesting that the semantic system shows little flexibility in taking pragmatic context into account (for comparable findings, see [108]).

Future studies will need to pinpoint the degree of flexibility, and its limits, for pragmatic and social context to constrain semantic processing during language production. This will inform models of language production and help distinguish accounts that view word retrieval as a largely static process (e.g. [162–170]) from accounts that allow for more context-dependent flexibility (e.g. [171,172]).

6. How do we get to a framework for language production in social interaction

Speaking is a fundamentally social activity: when we speak, we typically speak in the context of a social interaction. In order to do so successfully, our cognitive system does not only need to solve the challenge of producing fluent speech; it must also meet the demands social interaction places on the interacting individuals. In this paper we have focused on three main demands: (i) the need to integrate speaking with listening, (ii) the need to build from single actions a joint action, and (iii) the need to flexibly adapt to the conversational partner.

To meet these demands, additional processes are engaged that go beyond the core faculty of speaking. To allow for timely turn-taking, speakers must plan their speech and, in parallel, process the incoming speech from their partner's turn. Furthermore, speakers plan, represent and execute not only their own verbal actions, they also predict and represent their partner's actions in order to coordinate, in time and content, their own and their partner's behaviour. This requires taking into account the partner's unique perspective, knowledge background and informational needs, involving processes of social cognition and mentalizing. Speaking in social interaction therefore includes a series of additional cognitive processes. This argument is in line with other recent calls for extending the traditional language network [124,173,174].

What is more, the processes of language production are shaped by social interaction. The verbal actions of the partner can have a profound and lasting effect on own language production. For example, the utterances of the partner contribute to the current semantic context and can as such interfere with a speaker's lexical access [90,92], or facilitate this access [97,106,107]. Moreover, a given semantic context can be shaped by knowledge about or information shared with the conversational partner [160]. Language comprehension may be facilitated when combined with episodes of speaking (e.g. [81,175]) and may be affected by the knowledge state of the conversational partner (e.g. [176]).

In order to develop a theoretical account of language production in social interaction it is necessary to integrate our understanding of language production with our understanding of how people understand others' intentions and mental state and how multiple actors coordinate their behaviour and mental states in social interactions more generally. Indeed, in recent years these formerly separate research areas have grown together as cognitive psychologists are becoming increasingly interested in social settings; and social psychologists are becoming increasingly interested in cognitive accounts for social behaviour [177].

Advancing this field requires methodological innovations that allow experimental investigations of language use in social settings. This may require innovating or adapting existing experimental procedures and designs so that two or more individuals can interact in controlled laboratory settings. Some questions may be addressed best by recording not only one individual's behaviour, cognition or neural states but also of those of the interaction partner. This shadows recent calls for a paradigmatic shift in social neuroscience to a two-person approach (e.g. [40,178,179]).

Methodological advances also require more interactive settings that allow research participants to actively engage in social interactions instead of just passively observing them [180–182]. This development is also met by recent calls for a shift in the study of social cognition towards more ecologically valid settings in dynamic, multimodal, context-embedded and interactive environments [183]. Last but not least, when bringing social interaction into the laboratory, the research participants' behaviour is influenced not only by the presented stimuli, but also by the interaction partner. Researchers are therefore cautioned to consider, just as carefully as they consider the selection of their stimuli, how an interaction partner is introduced into the experimental setting [73].

7. Conclusion

Investigating language production in socially isolated laboratory settings has laid the ground works for understanding the cognitive and neural underpinnings of speaking. Language production in social interaction is not fundamentally different from language production in isolation. Yet social interaction places important constraints on language production that are met by engaging additional processes and shape the processes underlying language production. It is time to scale up our understanding of language production to settings in which two or more speakers communicate in social interaction.

Data accessibility

This article has no additional data.

Authors' contributions

A.K.K.: conceptualization, writing—original draft; R.A.R.: writing—review and editing.

Both authors gave final approval for publication and agreed to be held accountable for the work performed therein.

Conflict of interest declaration

We declare we have no competing interests.

Funding

This research was supported by grants KU3236/3 and AB277/11 from the German Research Council and the Career Development Award 420_CDA_7 awarded to the first author by the Berlin University Alliance.