Cortical-subcortical production of formulaic language: A review of linguistic, brain disorder, and functional imaging studies leading to a production model

Abstract

Formulaic language forms about one-fourth of everyday talk. Formulaic (fixed expressions) and novel (grammatical language) differ in important characteristics. The features of idioms, slang, expletives, proverbs, aphorisms, conversational speech formulas, and other fixed expressions include ranges of length, flexible cohesion, memory storage, nonliteral and situation meaning, and affective content. Neurolinguistic observations in persons with focal brain damage or progressive neurological disease suggest that producing formulaic expressions can be achieved by interactions between the right hemisphere and subcortical structures. The known functional characteristics of these structures form a compatible substrate for production of formulaic expressions. Functional imaging using a performance-based analysis supported a right hemisphere involvement in producing conversational speech formulas, while indicating that the pause fillers, uh and um, engage the left hemisphere and function like lexical items. Together these findings support a dual-process model of language, whereby formulaic and grammatical language are modulated by different cerebral structures.

Introduction

In a New York Times article, a lawyer uses the aphorism Where there’s smoke, there’s fire to convey the gravity of the case against Enron and its accounting firm, Arthur Andersen:

“The destruction of documents would indicate some intent to deceive,” said Franklin B. Velie, a former federal prosecutor who is now a partner at the Salans law firm in New York. “Where there’s smoke there’s fire, and where there is a lot of smoke, like the destruction of documents, there is a lot of fire. This is really beginning to look like a fraud scenario” (Mitchell, 2002).

Key characteristics of formulaic language, consisting of idioms, proverbs, expletives, pause fillers, aphorisms, conversational speech formulas, and a variety of other fixed, conventional expressions, are manifest in this example: length, cohesion with decompositionality, memory storage (known to speakers in the language community), nonliteral, situation-bound meaning, and affective/attitudinal content. It is the purpose of this paper to review these characteristics and to place them in the context of brain processing, highlighting cortical-subcortical relationships. Evidence is drawn from linguistic studies as well as brain disorder and functional imaging studies. It will be seen that the characteristics of formulaic expressions (FEs) are compatible with known characteristics of the brain structures that modulate them. An original finding (on pause fillers) is offered to contribute to a speech production model of how formulaic and propositional expressions are produced.

Background

Studies of formulaic language have gained ground in linguistics, sociolinguistics, pragmatic studies, neurolinguistics and first and second language learning (Altenberg, 1998; Bolinger, 1976, 1977; Clark, 1970; Gibbs & Gonzales, 1985; Jespersen, 1933; Katz, 1973; Lyons, 1968; Mackin, 1978; Makkai, 1972; Moon, 1998,a, b; Nunberg, Sag, & Wasow, 1994; Redfern, 1989; Wray, 2002). Formulaic language plays an important role in normal verbal communication (Alexander, 1978; Fillmore, 1979; Kuiper, 2004, 2009; Pawley & Syder, 1983) and has properties that distinguish it crucially from grammatical language.

Length and number

The FE example in the New York Times excerpt above is 5 words in length (not counting contractions), which is fairly long to be retained in memory (Miller, 1956; Simon, 1974). FEs are comprised of from one to several words, and very long fixed utterances have been reported, such as poetic and literary expressions (Hoblitzelle, 2008); songs and titles from personal biographical history (Pena-Casanova, Bertran-Serra, Serra, & Bori, 2002); Buddhist prayers (Shinoura et al., 2010), and schemata (fixed expressions with one or more open slots) such as You can take the __ out of the __, but you can’t take the __ out of the __ (Van Lancker Sidtis, Cameron, Bridges, & Sidtis, 2015). The fire and smoke aphorism is one of a very large collection in a speech community. It is probably impossible to reliably establish total numbers of formulaic expressions in the native speaker’s repertory. A dictionary of contemporary Czech phraseology, featuring idioms, provides 35,000 entries (Cermák, 1994). The number of known fixed expressions in the cultural cache of a language is probably very much larger (Jackendoff, 1995)—and no upper limit has been identified. A recent study provides an estimate of 42,000 lemmas (unique word stems) present in an adult speaker (Brysbaert, Stevens, Mandera & Keuleers, 2016), with considerable combinatorial potential for a much larger number of multiword expressions. Studies of incidence in proportion of talk in everyday discourse range from 25–60%, depending on types of FEs quantified, topics and speakers, and whether the corpora are written or spoken (Biber, 2009; Biber & Conrad, 1999; Erman & Warren, 2000; Foster, 2001; Sinclair, 1991; Van Lancker Sidtis & Rallon, 2004).

Cohesion and decomposability

Formulaic expressions, by definition, are unitary, which is to say they are canonically fixed with specific words in a certain order on a stereotyped intonation contour (Hallin & Van Lancker Sidtis, 2015; Lieberman, 1963, 2001; Lin & Adolphs, 2009; McGlone & Tofighbakhsh, 2000; Van Lancker, Canter, & Terbeek, 1981). Formulaic expressions have been shown in experimental studies to be cohesive, holistic, fixed, or unitary in structure (Horowitz & Manelis, 1973; Kuiper, van Egmond, Kempen, & Sprenger, 2007; Osgood & Housain, 1974; Pickens & Pollio, 1979; Simon, 1974; Siyanova-Chanturia, Conklin, & van Heuven, 2011; Swinney & Cutler, 1979), but they can also be manipulated using grammatical rules and extra words. They are flexibly cohesive. In the New York Times example, a lot was inserted twice, and a phrase intervened in the middle of the aphorism: “where there is a lot of smoke, like the destruction of documents, there is a lot of fire.” These kinds of changes are frequent and allowable, as long as the canonical form is recoverable (Kuiper, 2009). Demonstrated compositionality in some experimental contexts—e.g., applying grammatical rules or lexical insertion—has led to a proposed hybrid model (Sprenger, 2003; Sprenger, Levelt & Kempen, 2006), whereby formulaic expressions occur as fixed—existing as a recognizable formuleme—and decomposable into structured parts (Libben & Titone, 2008)—on different “levels.” In any case, their integrity and recognizability as a unit distinguishes them importantly from novel, newly created language.

Known: Familiar and stored as memory traces

The lawyer, Mr. Velie, in using the aphorism, assumes that the readers will recognize the fixed phrase with its concomitant nonliteral meaning. The effect of using the phrase is contingent on familiarity recognition by the reader. FEs are known (stored in toto as memory traces), with details of shape, meaning, and social contingencies, to native speakers of the language (Wray and Perkins, 2000). This has been demonstrated in listening studies (Rammell, Pisoni, & Van Lancker Sidtis, 2018; Van Lancker Sidtis, 2003) and surveys (Van Lancker Sidtis & Rallon, 2004). The FEs are familiar, in the sense that faces, voices, persons, and geographical locations can be personally familiar (Kreiman & Sidtis, 2011, Chapter 6).

Meaning: nonliteral and situation-bound

FEs are typically nonliteral in meaning and to make sense, they are tied closely to social and interlocutory context. Mr. Velie obviously does not intend to directly refer to fire or smoke in his message. Nonliteral meanings have an indirect effect, which, paradoxically, can have more formidable impact than using a literal message. These qualities emerge in the smoke and fire example. Formulaic expressions communicate complex and intricately woven scenarios, and the very vagueness of their meanings gives them power. FEs have been seen to be brought to bear in complaint management (Drew, & Holt, 1988), and they often depict complicated social relationships between players, as in She has him eating out of her hand.

Meaning: emotion, affect, and empathy

The fire and smoke aphorism carries emotional content. Nuance, connotations, affect, and attitudinal-emotional meanings inhere essentially in formulaic expressions. Expletives (Wow!, dammit, shucks, good heavens) make this point easily, as the cardinal purpose of these expressions is to communicate anger, surprise, shock, disapproval, or excitement (Foote & Woodward, 1973; Gallahorn, 1971; Hughes, 1991; Jay, 2000; Montagu, 1967; Munro, 1989; Van Lancker & Cummings, 1999). The idiom he’s out on a limb communicates worry, risk, failure, anxiety, and any number of negative affective associations, while a matched literal sentence, he’s out in a boat, without modifying verbal material (e.g., a leaky boat, in a storm, relaxing, on a perfect day) is neutral regarding attitudinal or emotional valence. Perusal of lists of idioms reveals this as a consistent element. Idioms engage emotional arousal, subtle or strong, positive or negative. Don’t bite the hand that feeds you carries a warning and a criticism; He pulled the rug out from under us implies disappointment, dismay, and reproachful anger. Use of an idiom strongly aligns the co-participant with its meaning (Drew & Holt, 1988); idioms manifest “a special resistance to being challenged” (p. 411), due to their success in achieving affiliative responses, partly due to their generality and the assumption of general knowledge in the culture (Kitzinger, 2000).

Conversational speech formulas, such as right, if you say so, whatever! weave together affect and attitude, which may be empathetic, reproachful, suspicious, or encouraging. Routine speech formulas form a large part of daily talk, functioning to communicate “beliefs, wants, wishes, preferences, norms and values” (Coulmas, 1979, p. 239) and allowing participants to engage in language play (Bell, 2012). The bonding and affiliative functions of formulaic expressions in conversation have been amply described (Edwards & Potter, 1992; Kecskes, 2000; Potter, 1996; Wray & Perkins, 2000), including in children (Corsaro, 1979; Gleason & Weintraub, 1976). A poignant example of comes from a World War II diary, where, toward the end of the war when Hitler’s influence was diminishing, a German citizen noted the return of the traditional greeting “Gruß Gott,” (a shortened version of “God greets you”) previously suppressed, on the streets of his Bavarian town (Breloer, 1984; reported in Kershaw, 2000). Bonding and group identification is seen in the extensive repertories of formulaic expressions in sports, work settings, families, poetry, and many other social domains (Hickey & Kuiper, 2000; Kiparsky, 1976; Pawley, 1991).

Experimental studies also provide evidence of the role of formulaic expressions in communicating affect. Formulaic expressions were frequently used in autobiographical accounts (Fainsilber & Ortony, 1987; Fussell & Moss, 1998). It may be easier for people to express emotion in the indirect language of nonliteral utterances. Formulaic conventional expressions are liberally used in expressing affection and friendship. In a study of over 100 couples, solidarity in the relationship was positively correlated with types and total numbers of FEs reported by the couples (Bell & Healey, 1992). In another study, “loving, commitment, and closeness” were correlated with the number of idioms expressing affection and sexual activity (Bell, Beurkel-Rothfuss, & Gore, 1987, p. 47). Marital satisfaction, rated in newly and long-term married couples, was associated with using more of such expressions (Brues & Pearson, 1993) and couples develop a personal repertory of FEs (Hopper, Knapp, & Scott, 1981). Conversely, couples whose relationship is in decline reported fewer FEs (Dunleavy & Booth-Butterfield, 2009). FEs with taboo and importuning content elicit measurable autonomic reactions (Harris, Aycicegi, & Berko Gleason, 2003). Computer approaches, such as those employing sentiment analysis, which automatically interprets attitudes and emotions in written text, may serve to quantify the emotional valences of FEs (Williams, Bannister, Arribas-Ayllon, Preece, & Spasić, 2015)

The Neurolinguistic Perspective

Lesion studies

The early impetus for recognizing the role of FEs in speaking arose from neurolinguistic observations in persons with aphasia. Starting with J. Hughlings Jackson in the 19^th century (Hughlings Jackson, 1874/1932, 1878/1932), scientists and clinicians with exposure to aphasic speech universally noted that some kinds of speech—usually fixed, familiar utterances-- are well preserved even in the context of severe phonological, lexical, and syntactic language impairment (Bay, 1964; Benson, 1979; Critchley, 1970; Darley, 1982; Espir & Rose, 1970; Gloning, Gloning, & Hoff, 1963; Goldstein, 1948; Goodglass & Mayer, 1958; Head, 1926; Kreindler & Fradis, 1968; Luria, 1966; Marie, 1925/71; McElduff & Drummond, 1991; Pick, 1931/1973; Wepman, Bock, Jones, & Van Pelt, 1956). Utterances noted by neurologists and other clinicians constitute the subsets now recognized—and enlarged-- as comprising formulaic language. Clinical assays using the Wada technique, where by cognitive behaviors are evaluation during anesthetization of one and then the other cerebral hemisphere (Loring, Meador, Lee, & King, 1992) suggest that primary language functions reside in the LH in the large majority of right handed persons.

To substantiate these anecdotal reports, surveys were distributed to speech clinics (Blanken, Wallesch & Papagno, 1990; Code, 1989). Responses documented that in severe aphasia, residual speech was made up of swearwords, interjections & greetings, sentence initials (I can’t, I think), and a broad range of fixed, conventional expressions. The results were comparable for English, German, and Chinese (Blanken, 1991; Chung, Code, & Ball, 2004; Code, 1982). Structured testing comparing automatic and propositional speech production and comprehension in aphasia further verified this dual functionality (Lum & Ellis, 1994; Van Lancker Sidtis & Yang, 2016). Persons with the diagnosis of transcortical sensory aphasia (Berthier, 1999) or with semantic deficits due to isolation of the speech area, while having severely impoverished semantic competence, are able to complete formulaic expressions, such as idioms and proverbs (Nakagawa et al., 1993; Sidtis et al., 2009; Whitaker, 1976).

Residual speech following extensive brain damage can produce a compelling display. In a famous case, Baudelaire’s post stroke recurrent utterance consisted exclusively of a fragment of a French expletive—cré nom. (Dieguez & Bougousslavsky, 2007). An early study of a normally developing adult who underwent a left hemispherectomy for cancer treatment, resulting in a profound aphasia, revealed a store of residual expressions including expletives, pause fillers, and conversational speech formulas (Smith, 1966; Smith & Burklund, 1966); other such cases revealed similar phenomena, strongly attributing this residual, pragmatically-based speech competence to the remaining right hemisphere (Crockett & Estridge, 1951; Hillier, 1954; Zangwill, 1967; Zollinger, 1935). In an extensive study using the Wada technique in persons with aphasia, a strong RH role in preserved, nongrammatical speech was demonstrated (Czopf, 1972).

More recently, experimental studies of groups of individuals with unilateral, focal lesions following stroke have confirmed and extended this early perspective (e.g., Baldo et al., 2016). Monologues, consisting of spontaneous speech over a range of topics chosen by the speakers, were obtained from persons with left hemisphere (LH) damage and aphasia and from persons with RH damage who were comparable in age and education. Discourse samples from matched healthy controls were also recorded and transcribed. Numbers of words in formulaic expressions were assessed and calculated as a proportion of the word count of the discourse sample word count. Examination of these samples of discourse revealed that LH damage was associated with proportions of FEs that were significantly higher than those obtained from healthy controls. In stark contrast, persons with right hemisphere (RH) damage produced significantly lower proportions of words in formulaic expressions (Van Lancker Sidtis & Postman, 2006). Similar results were obtained for Korean expressions produced by Korean-speaking persons with unilateral brain damage following stroke, using listeners’ ratings as measures (Yang & Van Lancker Sidtis, 2016). In a study of repetition phenomena in spontaneous speech, persons with RH damage repeated fewer FEs than their LH damaged counterparts (Wolf, Van Lancker Sidtis, & Sidtis, 2012). These empirically derived results and the hemispherectomy and other neurological observations above lend strong support to Jackson’s original impression attributing FE production to the RH.

It has also been reported that focal subcortical lesions, due to stroke, are associated with reduced proportions of formulaic expressions or a loss of known expressions. The first such reported case described a loss of the ability to produce well-known prayers following subcortical stroke (Speedie, Wertman, T’air, & Heilman, 1993). A later study examined proportions of FEs in the spontaneous speech of two persons following focal strokes to basal ganglia nuclei (Sidtis, Canterucci, & Katsnelson, 2009), and found a significantly diminished proportion in both cases. For one person, premorbid speech was available for comparison; the diminished FE proportion held for the poststroke speech only.

Progressive Neurological Diseases

It has often been observed clinically that individuals with Alzheimer’s disease (AD), although severely impaired in semantic abilities, nonetheless speak fluently, utilizing a noticeable richness of formulaic expressions, such as nice to see you again, see you later, so long, excuse me, and thank you very much. A research study quantified this clinical observation: persons diagnosed with AD produced significantly more FEs in spontaneous speech than healthy controls (Bridges, & Van Lancker Sidtis, 2013). Throughout the progression of the disease, AD leaves much of motor function intact (driving, cooking, walking, manipulating objects) due to the fact that the basal ganglia, site of procedural motor function, remain relatively functional almost until the end phase (Cummings, 1985). Retained motor functionality has been shown by intact activities of daily living in AD, such as piano playing (Beatty et al., 1999) and games and card playing (Beatty et al., 1994), despite extreme cognitive impairments in memory and language. For example, in Beatty et al. (1994), a person diagnosed with AD continued to play contract bridge, while, due to severe cognitive impairment, he was unable to name the suits used in the card game. The retention of fixed, overlearned FEs in AD can also be attributed to these retained implicit motor abilities provided by a relatively intact basal ganglia.

In contrast to the AD neurological disorder, Parkinsonian disease (PD) arises from impaired subcortical nuclei, leading to motor disabilities manifest in tremor, rigidity, slowing, and gait disturbance. Speech is often slurred and low in volume. Further, PD speech, clinically observed, is relatively impoverished in use of FEs (Illes, 1989). A recent study compared spontaneous speech samples from AD and PD speakers (Van Lancker Sidtis, Choi, Alken, & Sidtis, 2016). In these experimental studies, again utilizing freely produced monologues that were recorded and transcribed, AD speakers’ proportions of FEs averaged significantly higher than the proportions measured in healthy speakers, while PD was associated with deficient output of FEs. In another measure of the production of fixed expressions, persons with PD made significantly more errors than the healthy control group in reciting known rhymes and prayers (Bridges, Van Lancker Sidtis, & Sidtis, 2013). Results from studies of proportions of FE in spontaneous speech in persons with left, right, and subcortical lesions and with AD or PD, compared with healthy adult speakers are shown in Figure 1.

Figure 1. Incidence of FEs in spontaneous speech in association with LH damage, RH damage, and subcortical disturbance.

Overview of single case and group studies assessing proportions of words in formulaic expressions. Number indicates n of participants: PD = Parkinson’s disease, SC = subcortical lesion, RH = right hemisphere lesion, LH = left hemisphere lesion, AD = Alzheimer’s disease.

Compatibility of RH and Subcortical Structures for FE production

The cerebral hemispheres are specialized for modes of functions (Bever, 1975; Bogen, 1969; Bradshaw & Nettleton, 1983; Graves & Landis, 1985; Heilige, 1993; Kosslyn et al., 1989; Martin, 1979; McGilchrist, 2009; Robertson, 1995). This fact arises from several decades of studies in neurological and healthy subjects using an array of experimental paradigms, a body of evidence too extensive to be reviewed here in detail. Descriptions of RH function, inferred from deficits following unilateral lesions and from psychophysiological studies, include competence for inference judgments (Brownell, Potter, Bihrle, & Gardner, 1986; Weylman, Brownell, Roman, & Gardner, 1989), functional, pragmatic communication and use of language in context (Bryan, 1988; Joanette & Brownell, 1990; Joanette, Goulet & Hannequin, 1990; Kaplan, Brownell, Jacobs, & Gardner, 1990), and nonverbal expressiveness (Buck & Duffy, 1980). Visual processing modes differ between left and right sides (Van Kleeck, 1989). Handling of lexical processing reveals differential sensitivity to semantic relationships, with peripheral and extended meanings better performed in the RH (Brownell et al., 1986; Drews, 1987; Sidtis, Volpe, Holtzman, Wilson, & Gazzaniga, 1981; Titone, 1998). The RH is superior to the LH in processing configurations, including visual and auditory Gestalts, such as faces, complex pitch patterns, and voice patterns (Sidtis, 1980; Sidtis & Volpe, 1988; Van Lancker, Kreiman, & Cummings, 1989). Emotional experiencing occurs in large part in the RH, shown by studies too numerous to review here (e.g., Spence, Shapiro, & Zaidel, 1996; for review, see Van Lancker, 1997; McGilchrist, 2009, Chapter 1 ). Personally familiar phenomena, including the sense of familiarity for voices, faces, geographical locations, and persons are processed in the RH (Cutting, 1990; Kreiman & Sidtis, 2011, Chapter 6; Sperry, Zaidel, & Zaidel, 1979; Van Lancker, 1991, 2012). Figure 2 provides a schematic overview of these superior competences of the RH as contrasted with the LH. These properties—emotional experiencing, context-coupling, familiar (known) phenomena, holistic processing—provide a welcoming context for formulaic expressions.

Figure 2. Schema of cerebral lateralization compatible with the dual process model of language.

Schema of brain systems underlying the production of grammatical (LH) and formulaic language (RH & subcortical nuclei). Known functional characteristics in RH and subcortical systems are compatible with the features of formulaic expressions.

Greater understanding of the complex contributions of the basal ganglia to motor output behaviors has steadily emerged since the seminal paper of Marsden (1982). These structures initiate, execute, and monitor integrated motor gestures (Baev, 1997). Routines and habits are established there (Graybiel, 1998; Knowlton, Mangels, & Squire, 1996; Lieberman, 2001). A two-tier memory system has been described, broadly characterized as memories and habits (Mishkin & Petri, 1984), declarative and nondeclarative memory (Squire & Zola, 1996), referencing cortical and subcortical functions respectively. Involvement of the basal ganglia in FE production is compatible with these descriptions and with representation of these structures as supporting procedural memory (Ullman, 2004, 2008). It is a reasonable suggestion, supported by on-line and acoustic evidence, that formulaic expressions are produced as a coherent unit, representing a behavior that is well suited to significant participation of basal ganglia function in a cortical-subcortical network.

Stimulation during stereotaxic surgery of subcortical nuclei has elicited fixed utterances of the type we are describing here (Petrovici, 1980; Schaltenbrand, 1965). These comments have a relationship to observations in Gilles de la Tourette syndrome, where compulsive taboo utterances occur at some point during the disease (Singer, 1997; Van Lancker & Cummings, 1999). Gilles de la Tourette’s syndrome is believed to be due to basal ganglia disorder (Shapiro, Shapiro, Bruun, & Sweet, 1983).

Two Modes of Language Competence: Dual processing

The distinction between formulaic language and grammatical expression has been described using different approaches. Heine (2018) and his colleagues (Heine, Kuteva, & Kaltenböck, 2014) distinguish microgrammar (lexical items and rules forming propositional statements) and macrogrammar, or “theticals,” such as if you will, never mind, please, Watch out!, whoopee” that organize, comment on, evaluate, and direct the talk. In another approach, two tracks, called primary and collateral (Clark & Fox Tree, 2002) make a similar distinction of speech modes in actual use. A comparable duality from observations in aphasic speech is configured as referential and modalizing (Nespoulous, Code, Virbel, & Lecours, 1998). In similar perspectives contrasting formulaic and grammatical language, Erman & Warren (2000) propose that all speech consists of the idiom principle and the open choice principle, and Ellis (1983) stated that formulaic and novel modes are different approaches to expressing meaning.

These linguistic observations support a dual process model of language competence, based on the characteristics of cohesion, affective function, pragmatic practices, and knowledge basis in speakers (Bobrow & Bell, 1973; Heine et al., 2014; Lounsbury, 1963; Wray & Perkins, 2000). The neurolinguistics data indicate that propositional (grammatical or novel) and formulaic language are stored and processed differently in the brain: grammatical language is modulated by the LH, while formulaic language depends importantly on a RH-subcortical system (Van Lancker Sidtis, 2004, 2015).

Functional Imaging Studies using PET & fMRI: review & critique

The focus of this functional imaging review will be on the production rather than comprehension of non-literal expressions. Because of the prodigious language capacity of the left hemisphere, this system can process (i.e., comprehend) almost any type of linguistic material in most situations, but with diminished attributes (e.g., affect, social complexity, familiarity recognition, holistic processing) that are part of RH processing. Production is a better indicator of specialized brain function for formulaic language. Our review does not include examination of nonliteral language in general, usually focused on use of metaphor, which includes creative and poetic language. The focus on production in this review necessarily limits the body of relevant functional imaging studies (Eviatar & Just, 2006; Rapp, Luebe, Erb & Grodd, 2004; Bottini, Corcoran, Sterzi et al., 1994). Examining the functional imaging results for production of formulaic utterances also places the work in the context of the clinical evidence, which has consistently demonstrated reduction of FE expressions following damage to the RH and basal ganglia nuclei, along with increased incidence in LH damage.

There are several issues that contribute to the relatively small number of studies of formulaic language expression with functional imaging. Researchers have generally avoided overt speaking during functional magnetic resonance imaging (fMRI) because of concerns over movement artifact. Since the blood oxygen level dependent (BOLD) signal occurs seconds after the stimulating event, timing the separations of the expressive events or alternating the timing of the MR data acquisition (sparse sampling) have been used to mitigate the concerns about movement artifacts. Although positron emission tomography (PET) is less hostile to speech studies (quieter than MRI, less susceptible to movement artifact with longer imaging times), it has become less popular because it is more invasive, more expensive, and has longer temporal resolution.

A more serious empirical problem has been the inconsistency of results across both PET and fMRI studies, as well as inconsistencies between the large oeuvre of functional imaging results and clinical observations. There is evidence that both types of inconsistency -within the realm of imaging studies and between imaging and clinical findings--are strongly influenced by the way in which the data are processed, which usually involves task subtraction, often based on a “pure insertion” assumption (Kaan & Swaab, 2002). A PET study using spoken syllable production was specifically designed to evaluate the subtraction methodology. When an attempt was made to decompose the brain activity observed during the repetition of consonant-vowel syllables to patterns associated with phonation and articulatory movements, the results produced brain maps that had little relationships to the known functional anatomy of speech production (Sidtis, Strother, Anderson, & Rottenberg, 1999). Further, when a simple rest state was contrasted with a speech task as a control condition, a significant hemispheric asymmetry was observed, but the speech area in the RH was more “activated” than the homologous area in the left hemisphere (Sidtis, 2007; Sidtis, Strother, & Rottenberg, 2003). This and like reports using versions of subtraction methodology are at odds with over a century of clinical observations (Van Lancker Sidtis, 2007). One of the reasons for this is the observation that regional brain activity during the rest condition is influenced by the paired task condition for which it is supposed to act as a control, but the influence of task on its paired rest condition is not homogeneous across regions (Sidtis, Strother, & Rottenberg, 2004).

With these considerations in mind, the functional imaging results dealing with overt expression of examples of formulaic language will be considered. Using an early precursor of PET, Larsen, Skinhøj, and Lassen (1978) measured cerebral blood flow by injecting a radioactive tracer (¹³³Xe) into the internal carotid artery of the LH or RH. Left and right hemispheres were studied in different subjects who were undergoing neurodiagnostic procedures. Subjects were studied while at rest and again while producing “automatic speech” (counting repeatedly to 20 or reciting the days of the week). Subjects were instructed to produce words at the rate of one per second, inadvertently impacting the naturalness of the “automatic speech” activity, which has a fluent, rhythmic pattern of rate and prosody. Rest values were subtracted from speaking values, both of which were normalized using hemispheric average values. For the nine subjects who had the LH studied, there were significant task differences in two of the four frontal regions studied. For the nine subjects who had the RH studied, there were no group differences in these regions. However, considering individual subject data, 20 of the 36 LH frontal detectors showed significant increases during automatic speech, while 19 of the 36 RH frontal detectors showed significant increases during propositional speech._While this study attempted to address a clinically-driven question with an early form of functional imaging, there were no direct left-right comparisons possible, the normalization was not the same for the left and right data sets, task subtraction was employed, and the imposition of a timing constraint on the speech tasks likely reduced their “automatic” nature.

Recruiting healthy participants, Bookheimer, Zeffiro, Blaxton, Gaillard, and Theodore (2000) used oxygen-labeled water with PET to study tasks designated as automatic speech, including reciting the months of the year and the Pledge of Allegiance. Included in the study were also a syllable repetition task, an oral-motor task, and a rest state as contrast conditions. The data from both the serial-month and Pledge tasks were contrasted with data from the rest state. Of the 24 brain regions with significant increases in blood flow, 14 were in the LH while 10 were in the RH. As mentioned previously, contrasting rest and speech tasks in a similar PET study led to misidentified the RH speech area as being more active (i.e., having greater blood flow) that the homologous area in the LH (Sidtis et al., 2003; Sidtis, 2007). Further, in the Bookheimer et al. (2000) study, there was no consideration of the actual performance of subjects in the scanner, a factor that has been shown to resolve some of the discordance between imaging and clinical results (Sidtis et al., 2003; Sidtis, 2007, 2012).

In another study of automatic speech production using PET scanning, counting and reciting overlearned nursery rhymes were contrasted with spontaneous monologues (Blank, Scott, Murphy, Warburton, & Wise, 2002). All speech tasks activated various areas on the left hemisphere, including posterior supratemporal plane, lateral pars opercularis in the posterior inferior frontal gyrus and the anterior insula. Interpretation of these results is particularly difficult as the authors relied on multiple, complex and simple additions and subtractions of images obtained during different tasks.

PET was also used to study blood flow in healthy subjects who produced animal names, vocalized syllables, and, as an automatic speech task, counted from 1–10. Strict subtraction methodology was not utilized. Instead, using a procedure called partial least squares analysis (McIntosh, Bookstein, Haxby, & Grady, 1996), latent variables significantly associated with the tasks or combinations of tasks were identified. Blood flow images revealed three significant latent variables: one for naming and spoken syllables, with left anterior area predominating over right; a second latent variable was identified for naming in bilateral anterior areas, and a third, associated with counting, involved predominantly RH and subcortical sites. These results associated LH sites with naming and syllable production, while showing that counting is not strongly lateralized to the LH and, in addition, that counting recruited basal ganglia involvement (Van Lancker, McIntosh, & Grafton, 2003). These findings reflected clinical observations in persons with LH damage and aphasia: naming is very often impaired, while counting is almost always preserved even in the severest case of language disorder, as are other formulaic expressions known to the individual.

During a functional magnetic resonance imaging (fMRI) study using an action naming task with aphasic subjects, cries from participants were incidentally elicited (Postman-Caucheteux, 2010). Instead of correctly naming pictures depicting drowning, yelling, clapping, and so on, some subjects emitted formulaic expressions, such as help, yay, and hey. For these exclamations, right sided and subcortical structures showed an activation response that was not present when the correct action names were produced (Postman-Caucheteux et al., 2007). This essentially anecdoctal observation is deserving of future examination.

Findings using performance-based analysis

Although there are a limited number of functional imaging studies, they share the characteristic of suggesting largely bilateral brain activity during language expression. The observation of bilaterality is not restricted to studies of automatic speech or formulaic language in particular, however, but is characteristic of a wide range of functional imaging studies of language (Van Lancker Sidtis, 2007). The discordance between clinical and functional imaging data with respect to language lateralization led to the development of performance-based analysis (Sidtis et al., 2003; Sidtis, 2007, 2012; Sidtis, Van Lancker Sidtis, Dhawan, & Eidelberg, 2018). Rather than assuming that task contrasts, additions, and subtractions will reveal accurate functional maps of brain activity, performance-based analysis quantifies a behavior during scanning (e.g., speech rate) and determines if there is a combination of brain regions that significantly predicts that behavioral measure. This avoids the common assumptions that every increase in brain signal can be interpreted in the same way, that larger signal increases indicate greater functional relevance, and that decreases are irrelevant. These assumptions fail to identify clinically relevant brain regions (Sidtis et al., 2003; Sidtis 2007, 2012), and do not acknowledge the large role of inhibitory function in the brain.

Exploring PET data that initially showed bilateral activity in averaged group data during syllable repetition (reviewed above), subsequent performance-based analysis demonstrated that a linear combination of increased blood flow in the left inferior frontal region and decreased blood flow in the caudate nucleus was predictive of speech rate in normal speakers (Sidtis et al. 2003). While the original application of multiple linear regression in the performance-based analysis served as a discovery technique, the identification of a cortical-subcortical network in normal speakers became a testable hypothesis, which was confirmed in individuals with hereditary spino-cerebellar ataxia (Sidtis, Gomez, Naoum, Strother, & Rottenberg, 2006; Sidtis, Strother, Groshong, Rottenberg, & Gomez, 2010) and replicated in another independent group of normal speakers using a different scanner (Sidtis, Van Lancker Sidtis, et al. 2018). Unlike the bilateral patterns of blood flow found in group mean data, the predictive model is consistent with the data from over a century of clinical observations. From a methodological perspective, one of the fundamental aspects of performance-based analysis is the examination of how individual differences in patterns of brain activity may be related to individual differences in performance during the brain scan. It is a different approach to characterizing brain-behavior relationships than that employed in activation studies.

Performance-based analysis: Formulaic expressions

In addition to assessing cerebral blood flow using PET during the production of phonological and lexical items, Sidtis, Van Lancker Sidtis et al. (2018), analysis of monologues during PET scanning was conducted. Participants produced a 60 second monologue on a topic of their choice at two different times. FEs selected for study were conversational speech formulas, idioms, proverbs, expletives, sentence initials, discourse elements, and pause fillers. These were identified in the monologues by trained raters using methods developed previously (Van Lancker Sidtis & Rallon, 2004).

The monologues were recorded for acoustic and linguistic analyses. Word counts were conducted for each monologue and FEs were identified. The number of words in FEs was quantified as the proportion of FE words in the total number of words. Based on the previous study of normal speech production, performance-based analyses examined data from a number of regions extending from ventral to dorsal levels representing the left and right inferior frontal regions and the left and right caudate nuclei. Separate multiple linear regression analyses examined the repetition rates for phonological and lexical items (in syllables/sec) and the proportion of words in FEs. (Sidtis, Van Lancker Sidtis et al., 2018)

The results are presented in Figure 3. The model weights for the linear predictors of speech rate are in the left portion (A). As syllable and word production rates increased, blood flow increased in the left inferior frontal region and decreased in the right caudate nucleus. This pattern replicates the original results for normal speakers and is consistent with the clinical evidence for the left lateralization of propositional speech (Sidtis et al., 2003). In contrast, a complementary pattern of cortical-subcortical interaction (right portion, B) arose as the predictive model for the proportion of FEs in the monologues. As the proportion of FEs increased, blood flow increased in the right inferior frontal region and decreased in the left caudate nucleus. This cortical laterality is consistent with the effects of RH damage on the expression of FEs.

Figure 3. Results for conversational formulas in PET analysis.

The multiple linear regression model weights for regional cerebral blood flow in the inferior frontal regions and heads of the caudate nuclei in a group of normal speakers. The regression weights in the graph on the left (A) predict syllable production rates during the production of phonological and lexical items. The complementary regression weights in the graph on the right (B) are predictive of the proportion of words in FEs during the production of spontaneous monologues (Sidtis and Van Lancker Sidtis et al., 2018). Black-filled bars represent negative regression weights, gray bars represent positive regression weights.

Performance-based analysis: nonlexical pause fillers (Uh, um)

The monologues from the healthy native speakers of American English were further examined to investigate the status of pause fillers. As mentioned previously, these elements are included in the set of discourse elements, theticals, or formulaic expressions. Pause fillers (e.g., uh, um) have been traditionally described as non-lexical (Archer, Akjmer, & Wichmann, 2012), and they do appear to have features similar to those described for FEs. However, a previous linguistic analysis identified uh, um as bone fide lexical items (Clark & Fox Tree, 2002).

We addressed the question of whether the brain responses associated with pause fillers indicate that they are processed like lexical items (propositional language mode) or behave like FEs.

Methods

Subjects

Whole-brain cerebral blood flow (CBF) scans were obtained from 16 normal subjects. There were nine females and seven males with a mean age of 57 ± 10 years. Subjects were right-handed, native speakers of American English with normal speech. No subjects had a history of neurological or psychiatric disease and none were taking psychotropic medication. PET scans were performed at the Feinstein Research Institute of North Shore-Long Island Jewish Medical Center in accordance with the protocol approved by their Institutional Review Board. Speech studies were conducted in accordance with the protocol approved by the Nathan Kline Institute/Rockland Psychiatric Center Institutional Review Board. All subjects provided informed consents for both the speech and PET components of this study.

Positron Emission Tomography (PET) Imaging Procedures

Scanning sessions typically began at 8:00 AM so that participants could be consented, interviewed, and instructed in the procedures. Subjects were then positioned in the PET scanner (GE Advance Tomograph, General Electrics) and an intravenous line was placed in the subject’s left arm for H₂¹⁵O injection at approximately 10:00 am. Head positioning was achieved using a stereotactic head-holder with a 3D laser alignment used for stable and reproducible positioning. Lightweight headphones were attached to the head-holder to facilitate communication with the subject. A 10 min transmission scan was performed for attenuation correction followed by a 2D PET scan to establish the timing for the H₂¹⁵O injection. A series of whole-brain 3D PET scans was then performed with two scans dedicated to the performance of the monologues. Each monologue was produced for 60 s and scans were acquired using previously reported procedures (Sidtis et al., 2003; 2006; 2010; Sidtis, 2012; 2015). Blood flow was measured using a modified slow bolus injection of H₂¹⁵O using an automated injection system and image acquisition lasted approximately 2 min.

PET Image processing

Scans were reconstructed using 3D reprojection (3D RP) method, matrix dimensions 128 × 128 × 35, with voxel dimensions of 2.34 × 2.34 × 4.25 mm, with no smoothing applied. PET images were first aligned within subject and then spatially normalized to a standard space using the SPM99 software (SPM, London, UK http://www.fil.ion.ucl.ac.uk/spm/). Regions of interest used in previous PET-speech studies (Sidtis et al., 2003, 2006, 2010, 2012) extracted multiple regional CBF values from the ventral to dorsal extent of the head of the caudate, and regional values from the ventral to dorsal extent of the inferior frontal regions, bilaterally, using ScanVP image analysis software (Spetsieris, Dhawan, Takikawa, Margouleff, & Eidelberg, 1993). Irregular regions were used and adjusted on an individual basis to ensure capture of the target structure. A threshold was applied to each region so that the upper 10% of activity was captured to reduce partial volume errors and to minimize individual differences in anatomy. For each scan, a global CBF value was obtained using a whole-brain region of interest. This was used for normalization of regional brain values across subjects.

Speech samples

All monologues were generated by the speakers without control or direction by the examiners, with the request that they provide an account of something in their lives. Monologues consisted primarily of propositional (novel, grammatical, newly created) utterances, interspersed with unitary, formulaic expressions. The monologues used to extract dependent measures for the performance-based analysis were digitally recorded during scanning. Recordings were made using primary and secondary Marantz Professional digital recorders (PMD660) with boom mounted Audio-Technica AT3035 (primary) and AKG D5 (secondary) microphones. All recordings were made in .wav format at a 48k sampling rate. The total number of words and the numbers of occurrences of the pause filler (e.g., uh, um) in each monologue were computed to determine the percentage of total words in the monologues that were pause fillers.

Quantitative and Statistical Analysis

Incidence of pause fillers before formulaic or novel utterances was examined; this information could contribute to the role of these elements in utterance planning. For the performance-based analysis of the occurrence of pause fillers, regional CBF data were normalized for differences in global CBF values (Sidtis et al., 2003, 2006, 2010; Sidtis, 2012, 2015). The pause filler use data were entered into a stepwise multiple linear regression analysis (SPSS for PC version 7.5). together with the normalized CBF data from the left and right heads of the caudate nuclei and inferior frontal regions for each of the monologue scans. The performance-based analysis uses regression analysis to determine if there is a linear combination of regional CBF data that predicts a performance measure such as repetition rate or vocal stability (Sidtis, 2012, 2015). This statistical procedure assesses the contribution of each potential predictive region to establishing a significant linear relationship with the dependent variable. Variables are entered into a linear regression model, tested, and either retained or rejected. The following criteria were used for all regression analyses: probability of F to enter (0.05), probability of F to remove (0.10), and tolerance (0.01).

RESULTS

Monologues averaged 157.72 words (range 137–236, SD 8.31), with a mean of 12% consisting of words in formulaic expressions. The quantification of incidence before novel or formulaic expressions revealed large differences. There were significantly more pause fillers before novel utterances (mean = 5.5, S.D. = 4.10) than before formulaic utterances (mean = 1.1, S.D. = 1.6) [t(71) = −9.52; p < 0.001].

The performance-based analysis identified a pattern of CBF that was predictive of the relative use of pause fillers [F(2,29) = 7.95; p = 0.002]. The pattern was similar to that found for speech rate measured in productions of phonological and lexical items. The pattern consisted of a negative standardized beta weight (−0.53) for a right caudate nucleus region, and a positive standardized beta weight (0.41) for a left inferior frontal region. The pattern suggested that the brain was processing pause fillers with a network similar to that used for lexical items in the propositional mode Figure 4 (now at end of paper) and legend (paragraph 60).

Figure 4. PET results for pause fillers.

The multiple linear regression model weights for regional cerebral blood flow in the inferior frontal region and heads of the caudate nucleus in the same group of normal speakers whose regional brain activity was mapped for the repetition of phonological and lexical items (Figure 3A) and the proportion of words in FEs during the production of spontaneous monologues (Figure 3B). The cortical-subcortical interaction predictive of the use of pause-fillers was similar to that for the production of phonological and lexical items (Figure 4). Black-filled bars represent negative regression weights, gray bars represent positive regression weights.

Discussion and performance model

Our interest in this study has been to present a detailed description of formulaic language, revealing how, in its essential nature, formulaic language differs from propositional, grammatical language. Because observations of language use following neurologic injury made it clear that formulaic and language systems can be dissociated, our next goal was to review evidence that normal incidence of formulaic expressions involves a cerebral system different from the one subserving grammatical language. This has been done, first, through studies of persons with focal, unilateral lesions. It was observed that production in spontaneous speech of formulaic language relies less on the left, language hemisphere than does grammatical language. In contrast, idioms, proverbs, conversational speech formulas and other conventional, fixed expressions are modulated in large part by a cooperation between the cortical RH and subcortical nuclei. Clinical observations as well as experimental studies lead to the conclusion that FEs can be modulated outside of the well-known, clinically confirmed LH language component (Davis & Wada, 1978; Geschwind, 1970; Naeser et al., 2004; Postman-Caucheteux et al., 2007).

This model of language processing proposes a duality, with grammatical and formulaic language relying on different cerebral structures. The review of lesion and brain disorder studies was followed by an overview of the few brain imaging studies of speech production relating to FEs. While some intriguing results emerged, these studies fall short in consistency and reliability for methodological and other reasons. Unfortunately, the discordance between functional imaging results and clinical observations has not been restricted to the study of FEs, but has been characteristic of language studies in general (Van Lancker Sidtis, 2007).

We turned to an analytic approach that has had some success in demonstrating convergence between clinical and imaging results. Performance-based analysis of formulaic and propositional speech productions in association with PET scanning data was used to identify predictive patterns corresponding with these two modes. The patterns were complementary. Conversational speech formulas were associated with greater blood flow in the right hemisphere and reduced flow in the left caudate. This produced a remarkable fit to the lesion data. In contrast, the behaviors selected to be surrogates for propositional speech were significantly associated with the opposite pattern, replicating several previous studies (Sidtis, Van Lancker Sidtis et al., 2018).

For the nonlexical pause fillers (uh, um), traditionally considered within the formulaic repertory (Archer et al., 2012), a different picture emerged. First, the pause fillers, uh and um, occur five times more frequently before novel (newly created) utterances than before formulaic (fix, routinized) utterances; this can be said to reflect their role in speech production planning with respect to ideational content. This finding accords with linguistic analysis as well as with our finding that UH occurs much more often before propositional statements. Similar findings, suggesting that these kinds of pause fillers signal conceptual effort, have been reported (Fox Tree, 2002; Fraundorf & Watson; 2014). It might be inferred that pause fillers serve as a place holder during the process of retrieving the next propositional idea and imparting it with linguistic structure. Secondly, the performance-based analysis of productions of nonlexical pause fillers in the PET environment identified them as having the status of grammatical language, and relying on the LH for motor production. In contrast, it is proposed that FEs are recruited in the preverbal stage of speech output. (Figure 5, adapted from Kent, 2000). Evidence from faster reaction times and greater knowledge and familiarity reported in numerous experimental paradigms suggest that FEs are available at the very earliest time in the process of speech production.

Figure 5. Production model.

Proposed schema for the production of formulaic expressions (FEs) and pause fillers (uh,um). In this model, FEs appear during the preverbal stage, while pause fillers arise at the lexical node. Adapted from Kent, 2000

The characteristics of the brain systems modulating formulaic as contrasted with grammatical language are compatible with the proposed dual model of language that relies on different neurological networks that are well-integrated in normal language production. The RH network modulates personal relevance (familiar phenomena), emotional experiencing, context based meanings, and Gestalt configurations (Myers, 1998; Van Lancker, 1997). The basal ganglia are known to store and process routinized, overlearned motor gestures (Graybiel, 1998). These structures, in their functionality, are well-suited to the properties of FEs, and appear to modulate its production in speech as part of the RH system. In the dual processing language model, two distinctive modes of language competence exist: formulaic and grammatical. These have radically different intrinsic characteristics, rely on different cerebral cortical-subcortical networks, and the properties of the two modes fit the characteristics of the complementary cerebral systems.

Clinical studies provide evidence that the networks underlying both language modes are undoubtedly more complex (e.g., cerebellum, thalamus, multiple cortical areas). It has been suggested that in spite of the technical power of modern functional imaging systems, they likely identify only the “tip of the iceberg” of complex neurological systems (Sidtis, 2007, 2012a; Sidtis et al., 2006). The results from the performance-based studies reviewed above point toward the likelihood that cortical-subcortical interactions act as cerebral switching mechanisms that govern the effortless integration of these two modes in fluent speech, possibly mediated by interactions, including inhibitory ones, between the cerebral cortices and the caudate nuclei. The dual process model of language competence, featuring formulaic and novel modes, is based on a large range of evidence and has had wide acceptance in neurolinguistics research. Further research based on this model will contribute to our understanding of first and second language acquisition, and clinical rehabilitation of language disorders.

Highlights.

Formulaic language (unitary, stored utterances) has become a topic of interest in many of the language sciences.

This article offers a state-of-the-art overview of formulaic language.

Linguistic material developed for formulaic expressions provides a full description of formulaic expressions.

Experimental findings from healthy speakers support the theoretical descriptions.

Neurolinguistic findings from several clinical sources converge toward a model of brain function underlying production of formulaic expressions.

Functional imaging studies of production of formulaic expressions are few and the results using subtraction methodology are inconsistent

A PET study of formulaic language incidence in healthy speakers was conducted using the analytic technique of performance-based analysis.

A new, recent study shows how performance-based analysis can reveal details in language processing to lead to a consistent picture.

Overall, the findings support the dual process model of language, whereby grammatical, novel language is generated by the left hemisphere, while formulaic expressions may be produced by a brain network involving the right hemisphere and subcortical structures