Language Control After Phrasal Planning: Playing Whack-a-Mole with Language Switch Costs

Abstract

Spanish-English (Experiments 1–2) or Chinese-English (Experiment 3) bilinguals described arrays of moving pictures in English that began with a complex or a simple phrase (e.g., “[The shoe and the mesa/桌子] moved above the cloud” vs. “[The shoe] moved above the mesa/桌子 and the cloud”). Bilinguals were trained to name the second picture in English for half the objects (e.g., “table”) and Spanish/Chinese (e.g., “mesa”/“桌子”) the other half. In complex-initial sentences, production durations of “shoe” and “and” were longer on switch than nonswitch trials; in simple-initial sentences, in Experiments 1–2, speech rate was not affected by switching until “mesa” was produced, and in Experiment 3 not until “above” was produced. Thus, bilinguals paid language switch costs just before or just as they started to produce a phrase with a language switch in it, suggesting that bilinguals complete phrasal planning in the default language before switching to the nondefault language.

It is common to hear bilinguals switch back and forth between languages when conversing with other bilinguals. However, numerous studies have shown that switching languages is more costly than staying in one language, causing differences known as switch costs (Meuter & Allport, 1999). Robust switch costs have been found even for highly proficient bilinguals (Costa & Santesteban, 2004), when switching is predictable (Festman et al., 2010), and when bilinguals can voluntarily choose whether and when to switch (de Bruin et al., 2018; Gollan, Kleinman, et al., 2014; Gollan & Ferreira, 2009; but see Blanco-Elorrieta & Pylkkänen, 2017; Kleinman & Gollan, 2016). Bilinguals may pay the costs for controlling language switches at multiple processing levels, including lexical, phonological/phonetic, and at the whole language level (Declerck & Philipp, 2015b; Gollan, Schotter, et al., 2014). At the whole language level, bilinguals select one language to serve as the default language, leading most words and syntactic structures to be produced in that language, and to the observation of switch costs only for switches out of the default language, but not for switches back to the default language (Gollan & Goldrick, 2018). The Matrix Language Framework model makes similar assumptions; on this view selection of a matrix language leads that language to be preferred in bilingual speech while the other language may only be produced in embedded language islands (Myers-Scotton & Jake, 2009), roughly corresponding to the default and nondefault language respectively. The present study aimed to investigate the units of speech that default language selection operates on.

Language Control and Default Language Selection

According to the widely cited Inhibitory Control Model (ICM; Abutalebi & Green, 2007; Green, 1998), bilinguals produce speech in the intended target language by inhibiting the nontarget language. Early support for the ICM came from asymmetrical switch costs, whereby switching to the dominant language is more costly than switching to the nondominant language (Meuter & Allport, 1999; for reviews see Bobb & Wodniecka, 2013; Declerck & Philipp, 2015b). This asymmetry may reflect the need to overcome inhibition of the dominant language, which was inhibited strongly to allow the nondominant language to overcome the higher baseline activation of the dominant language. When producing isolated words in a mixed-language block, bilinguals may even inhibit the dominant language proactively and globally in anticipation of interference, to make two languages about equally accessible throughout the block (i.e., inhibitory control at the whole language level; see Declerck, 2020 for a review). With this control mechanism applied in mixed language blocks, the dominant language is produced more slowly than the nondominant language, a reversed language dominance effect (Christoffels et al., 2007; Costa & Santesteban, 2004; Declerck et al., 2013; Gollan & Ferreira, 2009; Kleinman & Gollan, 2018; Li & Gollan, 2018; Philipp et al., 2007; Philipp & Koch, 2009; Verhoef et al., 2009).¹

For default language selection, a different (but not mutually exclusive) possibility is that language control is achieved via boosted activation of the target language (e.g., Branzi et al., 2014, 2016; see also Yeung & Monsell, 2003, but see Gollan & Goldrick, 2018). Some recent evidence indeed suggests that the default language is selected by activation and not by inhibition. Li and Gollan (2021) investigated this question by contrasting cognate effects in bilingual picture naming with versus without sentence context. Cognates are translations that overlap in form (e.g., lemon-limón). In out-of-context picture-naming, cognates are named more quickly than noncognates (e.g., dog-perro), a result that is thought to reflect facilitation at the phonological level emanating from automatic flow of lexical activation from lexical representations in both languages to shared sounds (Costa et al., 2000). If sentence context leads to selection of one language as the default language through inhibition of the nontarget language, then cognate facilitation effects should be weaker in sentence context than they are in bare picture naming. To test this hypothesis, Spanish-English bilinguals named pictures out of context, or read aloud sentences with a single word replaced by a picture (i.e., to name a picture in sentence context, e.g., “The woman was scared by the [perro] that stood beside my uncle”; the word in brackets indicates the picture). In both cases, half of the pictures were cognates while the other half were noncognates, and participants were cued to use the same language as spoken just before the picture or to switch languages to name the pictures. Switch costs were larger in sentence context, suggesting that sentence context elicited a stronger default language selection than out-of-context picture naming. However, critically, cognates speeded picture naming times and reduced switch costs to the same extent in vs. out of sentence context, suggesting that default language selection involves activation of the default language, not inhibition of the nondefault language. If the nondefault language had been inhibited, cognate facilitation effects should have been weaker and cognate switch facilitation effects should have been larger in sentence context than in out of sentence context (for detailed explanation see Li & Gollan, 2021; also see Li & Gollan, 2018).² Thus, default language selection involves activation of word forms in the target language leading to larger switch costs in sentence context, but leaving the extent of nontarget language activation and cognate facilitation effects unchanged.

Default language selection likely also involves activation of syntactic structures in the target language. Supporting this view, switching out of the default language is more difficult on function words than content words, as function words carry more syntactic properties than content words (e.g., Fadlon et al., 2019; Gollan & Goldrick, 2018; Schotter et al., 2019). Declerck and Philipp (2015a) also showed evidence for the role of syntactic structure in controlling language switches. In that study, German-English bilinguals memorized a small set of sentences that consisted of five words and repeated each sentence four times in a row, producing each word in pace with an auditory signal, and switching languages on every other word, thus producing equal numbers of words in each language (e.g., on a sentence like “the boy runs very fast” bilinguals would produce “dieser Junge runs very schnell dieser boy runs sehr schnell the boy rennt sehr fast the Junge rennt very fast”). Despite frequent switching in this task, sentences with shared word order across languages revealed no switch costs. The absence of switch costs in this case could have reflected the fact that with shared word order, language switching did not violate default language selection, no matter which language was selected as the default language. In the same study, when word order was not shared across languages language switching slowed responses (e.g., on a sentence like “today you can go shopping” would be “today can you shopping go” in German word order, the sentence could follow word order of only one language, e.g., “heute kannst you shopping gehen”, is German word order). In this type of structure, the language that determined word order would likely be selected as the default language (even though equal numbers of words were produced in each language). Thus, half of the switches would have violated default language selection in this condition, which made switching particularly difficult.

Does the Scope of Planning Matter?

It is not known how default language selection and planning of language switches are coordinated. Connected speech production is planned on the fly; speakers do not plan entire sentences before speech onset. How far speakers plan in advance before speech onset is referred as the scope of planning. Before speech is initiated, words within the scope of planning have already been encoded and are held in working memory while other words in the same sentence have not been lexically encoded yet. Studies of monolingual speakers suggest they plan speech phrase by phrase (Martin et al., 2004, 2010; Martin & Freedman, 2001; Smith & Wheeldon, 1999; Wheeldon et al., 2013; also see Allum & Wheeldon, 2007; but see Brown-Schmidt & Konopka, 2008; Griffin, 2003; Griffin & Bock, 2000; Meyer et al., 1998). Smith and Wheeldon (1999) asked monolinguals to describe moving picture displays in sentences that began with either a complex noun phrase (e.g., “[The dog and the foot] move above the kite”) or a simple noun phrase (e.g., “[The dog] moves above the foot and the kite”). Speech onset latencies were longer for complex-initial sentences than for matched simple-initial sentences. Using the same paradigm, Martin et al. (2010) showed that this difference was not due to other potential confounds like retrieval fluency of the second content word in the sentence or visual grouping of pictures.

Several factors can modulate the scope of planning. For example, syntactic flexibility, or the availability of multiple syntactic alternatives has been shown to expand the scope of planning (van de Velde & Meyer, 2014) to the second post-verbal noun. Additionally, cross-language comparisons suggested that speakers of a syntactically flexible language (e.g., Russian) showed a broader scope of planning than those of a less flexible language (e.g., English; Myachykov et al., 2013). Message encoding difficulty is another factor, such that easier events may encourage speakers to plan more (Konopka & Meyer, 2014; van de Velde et al., 2014). Recent linguistic experience also plays a role, given that repetition of sentence structure can extend speakers’ scope of planning (Konopka, 2012). Nonlinguistic factors may also affect the scope of planning. For example, speakers tended to plan less with time pressure (Ferreira & Swets, 2002) or with faster speech rate (Wagner et al., 2010; Wheeldon et al., 2013). For bilinguals, the scope of planning may also depend on whether the target language is their dominant or nondominant language. While phrasal planning appears to apply to bilinguals’ dominant language, the scope of planning was longer in the nondominant language and was affected by speakers’ language experience and language proficiency (Gilbert et al., 2020; Konopka et al., 2018). However, no studies have examined the scope of planning in bilingual speech production with language switching, and little is known as to whether and how the scope of planning is coordinated with bilingual language control mechanisms.

In the present study, we aimed to examine how far in advance bilinguals plan language switches that violate default language selection in bilingual sentence production. To this end, we adopted the moving-picture description paradigm used by Smith and Wheeldon (1999) and included language switching either within or across phrases. To minimize lexical-level switch costs, we first trained bilinguals to produce particular words in a particular language. In addition to speech onset latency, we also measured the production duration of each word until switching occurs, following a procedure introduced by Momma et al. (2018; Momma & Ferreira, 2019) which makes it possible to pinpoint when in the course of sentence production bilingual speakers encounter difficulty – here, switch costs.

Data Availability

Stimuli and data from all three experiments are available on the Open Science Framework (https://osf.io/hkbe8/). The repository contains 1) full stimuli from Experiments 1–3, including experimental and filler sentences; 2) pictures that served as the 1^st, 2^nd, and 3^rd objects respectively in critical trials in Experiments 1–3, and 3) the data from Experiments 1–3 and the analysis codes.

Experiment 1

In Experiment 1, bilinguals described the movement of pictures in a 3-picture display in English. On some trials they were cued to switch languages to name any pictures shown in red (e.g., a red table). To isolate switch costs due to changes in default language selection, we first trained bilinguals to name half of the critical pictures in just one language and the other half in the other language. In a previous picture-naming study, language switches were cost-free when bilinguals were instructed to switch languages according to whichever language was more accessible for each given picture every time it repeated (Kleinman & Gollan, 2016). Thus, given that bilinguals were trained to name specific pictures in each language, so that they could always use the more accessible language when they had to produce a language switch in sentence context, switch costs due to lexical accessibility were minimized in the present study.

If bilinguals plan sentence production phrase by phrase, default language selection might also operate on this planning unit (instead of on whole sentences). If so, switches to the nondefault language that occur within the first phrase would need to be planned before speech onset, leading to switch costs in speech onset latency. Conversely, when bilinguals switch languages across phrases, no switch costs should be found in speech onset latency because the phrase that involves language switching would be planned after speech onset. Additionally, any language switches produced within the scope of planning (e.g., a phrase) may affect speech rate more than switches that happen beyond the scope of planning (e.g., between phrases). Switches in the following planning unit may be planned later than those in the current unit, so that the effects are not (at least) fully observed when bilinguals are producing words in the current unit. Alternatively, if default language selection is applied at the whole sentence level, then the time window where bilinguals pay switch costs and the extent of the costs should be not affected by the syntactic position of switch words, no matter which phrase includes the switch.

Participants

Twenty-four Spanish-English bilingual undergraduates (seven male, seventeen female) at the University of California, San Diego (UCSD) participated for course credit. Table 1 shows participant characteristics and Multilingual Naming Test scores in both languages (MINT; Gollan et al., 2012). All participants acquired Spanish as the first language and English as the second language, and all were English-dominant according to their MINT scores.

Table 1.

Means and standard deviations of participant characteristics

	Experiment 1 (Spanish-English)		Experiment 2 (Spanish-English)		Experiment 3 (Chinese-English)
Characteristic	M	SD	M	SD	M	SD
Age	20.0	1.5	20.3	2.4	19.9	1.3
Age of Acquisition of English	3.5	2.1	2.5	2.2	5.58	2.2
Age of Acquisition of Spanish/Chinese	0.6	0.8	0.2	0.4	0.1	0.4
Self-rated spoken English proficiencya	6.6	0.7	6.7	0.6	5.3	0.9
Self-rated spoken Spanish/Chinese proficiencya	5.8	1.0	6.0	0.9	7	0
Current percent of English use	80.0	10.6	80.3	14.2	57.3	18.7
Percent of English use during childhood	52.9	12.7	54.5	11.6	21.3	14.2
Primary caregiver English proficiencya	3.8	1.3	4.3	1.9	2.5	1.2
Primary caregiver Spanish/Chinese proficiencya	6.8	0.6	6.9	0.2	7	0
Secondary caregiver English proficiencya	3.2	1.8	3.8	2.0	2.0	0.9
Secondary caregiver Spanish/Chinese proficiencya	6.8	0.6	6.9	0.3	7	0
Years lived in Spanish/Chinese-speaking country	2.1	4.7	2.3	4.5	16.2	4.1
MINT score in Englishb	62.5	3.2	61.2	2.4	51.1	5.0
MINT score in Spanish/Chineseb	47.9	9.7	46.2	8.4	60.3	1.7

Note. None the characteristics showed significant difference between any two experiments (ps > .10) except the age of acquisition of Spanish (p = .039). However, Spanish was the first language of all participants, so that this difference might just be a result of participants’ different definition of “acquired since born”.

^aProficiency-level self-ratings were obtained using a scale from 1 (almost none) to 7 (like a native speaker).

^bThe maximum possible score is 68 on the Multilingual Naming Test (MINT; Gollan et al., 2012).

Materials & Design

Following Smith and Wheeldon (1999), we selected pictures to create displays that consisted of three pictures in each. First, we selected eight critical line drawings that were easy to name in both English and Spanish and had noncognate names (candle-vela, dog-perro, envelope-sobre, key-llave, king-rey, pencil-lápiz, strawberry-fresa, table-mesa) and divided them into two groups of four. For each participant, one group always appeared in black and was to be named in English, while the other group appeared in red and was to be named in Spanish. Target language color of each picture was counterbalanced across participants, and English and Spanish words within each group of 4 were matched for length in average number of syllables. The critical switch or no-switch (i.e., red or black) pictures always appeared as the 2^nd object of each 3-picture display.

We selected another forty-eight pictures with noncognate names that were easy to name in English to be the noncritical pictures in each display (24 served as the first object, and the other 24 served as the third object; see the appendix for the full lists). We created 96 critical displays with these critical and noncritical pictures divided into four blocks. The order of the four blocks rotated across participants. The three pictures in each display were always semantically unrelated (e.g., “shoe”, “table”, “cloud”). Therefore, in each block we had six critical sentence trials in each of the four conditions: (a) complex-initial switch (the sentence began with a complex noun phrase and the 2^nd object was red, i.e., needed to be named in Spanish); (b) complex-initial stay (the sentence began with a complex noun phrase and the 2^nd object was black i.e., needed to be named in English); (c) simple-initial switch (the sentence began with a simple noun phrase and the 2^nd object was red, i.e., needed to be named in Spanish); and (d) simple-initial stay (the sentence began with a simple noun phrase and the 2^nd object was black, i.e., needed to be named in English). See Figure 1 for sample displays. Sample sentences in these four conditions were:

• acomplex-initial switch: The shoe and the mesa moved above the cloud.
• bcomplex-initial stay: The shoe and the table moved above the cloud.
• csimple-initial switch: The shoe moved above the mesa and the cloud.
• dsimple-initial stay: The shoe moved above the table and the cloud.

Figure 1.

Examples of depictions for complex-initial and simple-initial sentences in Experiment 1, with or without a switch. Panel As show the initial status while Panels Bs and Cs show the status after pictures moved.

Third, to increase the variation of the syntactic constructions and to minimize inter-trial priming, 32 additional pictures that were easy to name in English were selected as filler pictures, which would appear in filler sentences only. On filler trials all the objects moved in one direction, eliciting utterances as

• efillers: They all moved up/down/left/right.

Following Martin et al. (2010), each block contained 32 filler sentences, eight in each direction of movement.

Procedure

First, bilinguals were instructed to name all the noncritical and filler pictures in English, and any response that was inconsistent with the target name was corrected (which rarely occurred). Next, bilinguals were instructed to name all the critical pictures in English or Spanish according to each picture’s color. They were trained to name each of these pictures six times in pseudorandomized orders to increase accessibility of each target picture in its intended target language based on the color cue. Third, in a demonstration session, the experimenter produced sentences for moving-picture displays (4 in critical sentence structures and 2 fillers in which all three pictures moved to the same direction) and instructed participants to produce sentences in the same syntactic structures, for both critical and filler sentences. Fourth, participants practiced on their own for 20 displays (16 critical trials and 4 fillers). In both the demonstration and practice sessions, the second object of each critical trial was always one of the critical pictures, while only filler pictures were used as the 1^st and 3^rd pictures in all displays.

Each display started with a 500 ms fixation cross (“+”) accompanied with a 100 ms click sound, followed by the moving-picture display. The pictures moved immediately once they appeared on the screen. The movement took 1.5 seconds, and all the pictures disappeared 500 ms after participants began their utterances, to encourage participants to conceptually encode the display before beginning to speak (Smith & Wheeldon, 1999, Martin et al., 2010).

Results

A proficient Spanish-English bilingual coded each trial for accuracy. The audio files and the transcription were aligned using a text-to-speech automatic forced alignment technique (the Penn Phonetics Lab Forced Aligner (p2fa); Yuan & Lieberman, 2008). Afterwards, errors on alignment were corrected using Praat (Boersma & Weenink, 2019). For each experimental sentence, speech onset latency was calculated as the interval between the click sound onset and the onset of speech minus 500 ms.³ Following Momma and Ferreira (2019), production durations of each word until the 2^nd noun were measured from the onset of the target word until the onset of the following word. Therefore, the interval between the target word and its following word was also counted towards the target word production duration. Trials with an error, disfluencies/stutters, audible nonspeech noise before the utterance onset, speech onset of more than 3 seconds, or if the previous sentence was not completed before the beginning of the current trial were excluded from analysis (10.1% of critical trials). In addition, onset latencies or durations more than three standard deviations from each participant’s mean across conditions were excluded from the analysis. In total, 88.8% of the data were analyzed, a percentage similar to that in Momma and Ferreira (2019) which used the same technique.

Analyses of speech onset latencies and production durations of each word until the switch words were carried out in R, an open source programming environment for statistical computing (R Core Team, 2013) with the lme4 package (Bates et al., 2015) for linear mixed effects modeling (LMM). For each sentence, analyses of error rates were carried out in R using general linear mixed effects modeling (GLMM). In all the analyses, contrast-coded fixed effects included sentence type (complex-initial vs. simple-initial), trial type (switch vs. stay), and their interaction; subjects and target picture names were entered as random intercepts with related random slopes. Exceptions included the word “and” in the complex-initial sentences, and “moved” and “above/below” in the simple-initial sentences, given that these words were unique in their sentence types. For these words, only trial type was included as the fixed effect. In each condition, half of the sentences used “moved above” and the other half used “moved below”. We collapsed the production durations of “above” and “below”. The significance of each fixed effect was assessed via likelihood ratio tests (Barr et al., 2013).

In the critical sentences, participants produced errors on 8.6% (SD=4.8%) of trials. Analysis of error responses showed no significant differences (ps > .13) except that participants produced marginally more errors in simple-initial sentences than in complex-initial sentences (M = 10.0% vs. 7.5%; β = .36, 95% CI = [−.05, .77], χ² (1) = 2.96, p = .085; see Table 2 for means in different conditions). The most common errors were disfluencies which included stutters, fillers, false-starts and self-corrections (28.3%; e.g., saying the uh… hat), followed by within-language lexical errors (27.8%; e.g., saying chair instead of table or saying went instead of moved), incorrect structure (9.6%; e.g., saying the bowl moved above… instead of the bowl and the key moved above…), incorrect direction (8.6%; saying moved above instead of moved below), missing function words (8.6%; e.g., saying the boy and envelope instead of the boy and the envelope), incorrect language (8.6%; saying pencil instead of lapiz or saying el instead of the), incomplete sentences (7.6%; e.g., saying the arrow moved… then stopped), and phonemic errors (1.0%; e.g., saying moy instead of boy).

Table 2.

Mean error rates (standard errors in parentheses) of sentence production in Experiments 1 to 3

	Experiment 1		Experiment 2		Experiment 3
	Stay	Switch	Stay	Switch	Stay	Switch
Complex-initial	6.7% (1.0%)	8.3% (1.2%)	5.9% (0.9%)	6.4% (0.9%)	10.2% (1.3%)	7.8% (1.1%)
Simple-initial	8.0% (1.1%)	12.0% (1.4%)	7.1% (1.0%)	8.1% (1.0%)	8.7% (1.2%)	10.2% (1.3%)

Speech Onset Latency

Figure 2 shows the subject means for speech onset latency and production duration of each word in each condition, and Figure 3 shows subject-mean differences between switch and stay trials. Table 3 shows the corresponding statistics. Bilinguals initiated speech more slowly when producing complex-initial than simple-initial sentences (M = 1553 ms vs. 1500 ms; p < .001). Surprisingly, bilinguals began speaking more quickly on sentences that involved a language switch on the second noun relative to sentences with no switch (i.e., when all the pictures were black; M = 1503 ms vs. 1550 ms; p < .001). The interaction between sentence type and trial type was not significant (p = .783) – switch benefits were equal in size and significant in the two sentence types (ps < .01).

Figure 2.

Subject means of speech onset latencies and production durations of words until the 2^nd object for each condition in Experiment 1 (error bars refer to 95% confidence intervals). Panel A is for the complex-initial sentences, Panel B is for the simple-initial sentences. Given that speech onset latencies were much longer than production durations of words, their scales were different in this figure.

Figure 3.

Subject means of difference score between switch and stay trials for each sentence type in Experiment 1 (error bars refer to 95% confidence intervals). Positive values refer to switch costs, whereas negative values refer to switch benefits. Noun2 with a circle is the region where switching occurred, and grey areas refer to the regions that showed evidence of switch costs. Panel A is for the complex-initial sentences, Panel B is for the simple-initial sentences.

Table 3.

Structure and results of speech onset latency and production duration analyses in Experiments 1 and 2 (Spanish-English Bilinguals)

		Experiment 1 (n=24)				Experiment 2 (n=30)
Region	Fixed Effects	β (ms)	95% CI (ms)	χ2 (1)	P	β (ms)	95% CI (ms)	χ2(1)	p
Speech Onset Latency	Sentence Type	56	[35, 76]	28.3	< 0.001	51	[35, 68]	37.37	< 0.001
	Trial Type	−51	[−78, −25]	14.2	< 0.001	−34	[−49, −20]	20.85	< 0.001
	Sentence Type * Trial Type	−6	[−49, 37]	0.08	0.783	−9	[−48, 29]	0.23	0.633
The1	Sentence Type	−3	[−7, 1]	2.35	0.126	0	[−4, 4]	0	0.994
	Trial Type	2	[−3, 7]	0.76	0.385	1	[−3, 6]	0.74	0.390
	Sentence Type * Trial Type	−1	[−9, 6]	0.16	0.691	−5	[−11, 1]	2.71	0.105
Noun1 (shoe)	Sentence Type	3	[−10, 16]	0	0.960	2	[−12, 16]	1.74	0.187
	Trial Type	12	[4, 19]	5.95	0.015	19	[14, 24]	37.7	< 0.001
	Sentence Type * Trial Type	26	[13, 38]	15.4	< 0.001	31	[21, 41]	35.43	< 0.001
	Noun1 (complex-initial)					Noun1 (complex-initial)
	Trial Type	25	[14, 35]	21.4	< 0.001	35	[27, 43]	76.69	< 0.001
	Noun1 (simple-initial)					Noun1 (simple-initial)
	Trial Type	−1	[−10, 8]	0.06	0.800	4	[−3, 10]	1.4	0.237
and (complex-initial)	Trial Type	18	[9, 29]	9.98	0.002	18	[9, 27]	14.72	< 0.001
moved (simple-initial)	Trial Type	0	[−11, 12]	0.01	0.946	0	[−9, 9]	4.58	0.995
above/below (simple-initial)	Trial Type	−4	[−24, 16]	0.16	0.688	10	[−7, 27]	1.32	0.251
the2	Sentence Type	−27	[−34, −20]	63.2	< .001	−17	[−22, −12]	44.31	< 0.001
	Trial Type	−11	[−35, 13]	0.75	0.385	−15	[−42, 12]	1.32	0.251
	Sentence Type * Trial Type	−9	[−19, 3]	1.66	0.129	−3	[−11, 5]	0.55	0.457
Noun2 (mesa/table)	Sentence Type	−44	[−57, −33]	57.3	< .001	−41	[−55, −27]	31.94	< 0.001
	Trial Type	17	[−64, 98]	0.21	0.645	26	[−55, 107]	0.33	0.566
	Sentence Type * Trial Type	−21	[−38, −3]	5.45	0.020	−24	[−46, −2]	4.61	0.032
	Noun2 (complex-initial)					Noun2 (complex-initial)
	Trial Type	7	[−76, 89]	0.02	0.877	14	[−68, 95]	14	0.740
	Noun2 (simple-initial)					Noun2 (simple-initial)
	Trial Type	27	[−54, 109]	0.44	0.508	38	[−44, 120]	0.83	0.364

Word Production durations

The earliest sign of switch costs was found on the first noun, henceforth “Noun1” (e.g., “shoe”). Bilinguals were slower to produce Noun1 (“shoe”) when they had to switch on the second noun, or Noun2 (e.g., “mesa”/ “table”), than when they did not switch on Noun2, M = 332 ms vs. 319 ms; p = .015). However, this was driven by complex-initial sentences (M = 340 ms vs. 315 ms; p < .001), while simple-initial sentences exhibited no difference between switch and stay trials (M = 323 ms vs. 323 ms; p = .80), a significant interaction between sentence type and trial type (p < .001). The main effect of sentence type was not significant (M = 327 ms vs. 324 ms for complex-initial vs. simple-initial sentences; p = .96). In complex-initial sentences, switch costs also spilled over to the following conjunction word “and”. Bilinguals also took longer to finish producing “and” on switch than on stay trials (M = 146 ms vs. 128 ms; p = .002).

Bilinguals took longer to finish producing the determiner the2 before Noun2 (“mesa”/ “table”) in simple-initial than complex-initial sentences (M = 135 ms vs. 108 ms; p < .001) but there were no switch costs on the2 (ps > .13; see Table 3).

Interestingly, in simple-initial sentences, bilinguals did not show any sign of switch costs until Noun2 (“mesa”/ “table”). Bilinguals took longer to finish producing Noun2 (“mesa”/ “table”) in simple-initial than complex-initial sentences (M = 403 ms vs. 359 ms, p < .001), and more importantly, a significant interaction between sentence type and trial type (p = .020), suggests that production durations of Noun2 (“mesa”/ “table”) were slower on switch vs. stay trials (M = 391 ms vs. 371 ms; p = .645) more in simple-initial (M = 419 ms vs. 387 ms; p = .508) than in complex-initial sentences (M = 364 ms vs. 355 ms; p = .877).

No significant condition effects were found on any other words (ps > .12).

Discussion

In Experiment 1, bilinguals initiated speech faster for simple-initial than complex-initial sentences, revealing the phrase to be the speech planning unit in bilingual sentence production (as shown in previous work for monolingual sentence production; e.g., Martin et al., 2010; Smith & Wheeldon, 1999). More importantly, we found some evidence of switch costs on production duration in both sentence types, which occurred earlier in complex-initial than simple-initial sentences. It seemed that switch costs were always paid when bilinguals produced the first noun of the phrase that contained a switch (i.e., “shoe” in “[The shoe and the mesa] moved above the cloud” vs. “mesa” in “The shoe moved above [the mesa and the cloud]”). However, an unexpected result was that bilinguals initiated speech faster on switch than on stay trials in both sentence types, showing switch benefits rather than switch costs in speech onset latency. These switch benefits are surprising if default language selection is planned phrase by phrase because, especially for simple-initial sentences, the language switch would not have been planned before speech onset.

However, before discussing the implications of our results at length, we conducted a second experiment to rule out the possibility that effects in Experiment 1 were due to the perceptual difference between switch vs. stay trials. That is, in Experiment 1 participants always saw two black pictures and a red picture on switch trials but three black pictures on stay trials. The red picture stood out on switch trials which might have facilitated processing of the second picture in the display. This in turn might have speeded speech onset latency leading participants to have to slow down later to make up for the initial fast start. If so, this perceptual difference might have contaminated language switch effects leading us to observe a spurious facilitation effect on speech onsets on switch trials. Therefore, in Experiment 2 we focused on the color differences and asked if the results could be replicated if the second picture stands out to the same extent on switch vs. stay trials.

Experiment 2

In Experiment 2, target pictures on both switch and stay trials were colored. Specifically, the second picture was either red or green (i.e., always different from the other two pictures in the same display), color-language correspondence was counterbalanced across participants, and bilinguals were trained to respond in English to one color, and in Spanish to the other.

Method

Participants

Thirty-two Spanish-English bilingual undergraduates (six male, twenty-six female) at UCSD who did not participate Experiment 1 were recruited for course credit. Table 1 shows participant characteristics and MINT scores in both languages (Gollan et al., 2012). As in Experiment 1, all participants were English-dominant but acquired Spanish first.

Materials, Design & Procedure

The materials, design, and procedure were the same as in Experiment 1, except that the 2^nd picture was never black. Instead, four pictures were red while the other four were green. Both the color-picture and the color-language correspondence were counterbalanced across participants. Namely, for half of the participants, red pictures were named in Spanish, while for the other half green pictures were named in Spanish. See Figure 4 for sample displays. For half of the participants, sample sentences in these four conditions were:

• acomplex-initial switch: The shoe and the mesa moved above the cloud.
• bcomplex-initial stay: The shoe and the table moved above the cloud.
• csimple-initial switch: The shoe moved above the mesa and the cloud.
• dsimple-initial stay: The shoe moved above the table and the cloud.

The language-color correspondence was reversed for the other half of the participants, namely:

• ecomplex-initial stay: The shoe and the table moved above the cloud.
• fcomplex-initial switch: The shoe and the mesa moved above the cloud.
• gsimple-initial stay: The shoe moved above the table and the cloud.
• hsimple-initial switch: The shoe moved above the mesa and the cloud.

Figure 4.

Examples of depictions for complex-initial and simple-initial sentences in Experiment 2, with or without a switch. Panel As show the initial status while Panels Bs and Cs show the status after pictures moved. Note that this illustration applied for half of the participants; for the other half, red pictures elicited stay trials and green pictures elicited switch trials.

Results

Two participants were removed due to high error rates on critical trials (> 20%). Following the same data cleaning procedure as described in Experiment 1, 91.2% of the remaining 30 participants’ data were analyzed. The procedures for data analyses were also the same as in Experiment 1, and the analyses of error rates (M = 6.9%) did not show any significant differences (ps > .23), (see Table 2 for condition means). Of the errors produced, the most common type were within-language lexical errors (34.0%), followed by disfluencies (18.6%), incorrect language (12.4%), incorrect direction (11.3%), missing function words (10.3%), incomplete sentences (9.3%), incorrect structure (2.6%), and phonemic errors (1.5%). Figure 5 shows mean by-subject speech onset latencies and production durations in each condition, and Figure 6 shows the differences between switch and stay trials. Table 3 shows the corresponding statistics.

Figure 5.

Subject means of speech onset latencies and production durations of words until the 2^nd object for each condition in Experiment 2 (error bars refer to 95% confidence intervals). Panel A is for the complex-initial sentences, Panel B is for the simple-initial sentences. Again, given that speech onset latencies were much longer than production durations of words, their scales were different in this figure.

Figure 6.

Subject means of difference score between switch and stay trials for each sentence type in Experiment 2 (error bars refer to 95% confidence intervals). Positive values refer to switch costs, whereas negative values refer to switch benefits. Noun2 with a circle is the region where switching occurred, and grey areas refer to the regions that showed evidence of switch costs. Panel A is for the complex-initial sentences, Panel B is for the simple-initial sentences.

Speech Onset Latency

As in Experiment 1, bilinguals initiated their speech more slowly when producing complex-initial than simple-initial sentences (M = 1389 ms vs. 1337 ms; p < .001), and began speaking faster on switch than stay trials (M = 1345 ms vs. 1381 ms; p < .001). The interaction between sentence type and trial type was not significant (p = .633) – switch benefits were equal in size and significant in both sentence types (ps < .01).

Word Production durations

Consistent with Experiment 1, the earliest evidence of switch costs was found on Noun1 (“shoe”). Bilinguals were slower to produce Noun1 (“shoe”) when they had to switch on Noun2 (“mesa”) than when they did not switch on Noun2 (“table”) (M = 333 ms vs. 315 ms; p < .001). Again, this effect was driven by complex-initial sentences (M = 341 ms vs. 309 ms; p < .001), while simple-initial sentences exhibited no difference between switch and stay trials (M = 324 ms vs. 321 ms; p = .237), a significant interaction between sentence type and trial type (p < .001). The main effect of sentence type was not significant (M = 325 ms vs. 323 ms for complex-initial vs. simple-initial sentences; p = .187). In complex-initial sentences, switch costs also spilled over to the following conjunction word “and” — bilinguals took longer to finish producing “and” on switch than on stay trials (M = 152 ms vs. 135 ms; p < .001).

Bilinguals took longer to finish producing the determiner the2 in simple-initial than complex-initial sentences (M = 127 ms vs. 110 ms; p < .001) but there were no effects of trial type (switching costs or benefits) on the2 (ps > .25).

In simple-initial sentences, bilinguals did not show any sign of switch costs until Noun2 (“mesa”/ “table”), that is, Noun1 (“shoe”) and the following “moved above/below” exhibited no switch costs (see Table 3). However, bilinguals took longer to finish producing Noun2 (“mesa”/ “table”) in simple-initial than in complex-initial sentences (M = 396 ms vs. 354 ms; p < .001), and again, there was a significant interaction between sentence type and trial type (p = .032), as switches slowed the duration of Noun2 (“mesa”/“table”) (M = 390 ms vs. 360 ms in switch vs. stay trials; p = .566) more in simple-initial (M = 418 ms vs. 375 ms; p = .364) than in complex-initial sentences (M = 364 ms vs. 348 ms; p = .740). In a further analysis on Noun2 (“mesa”/ “table”), we combined the data in Experiments 1 and 2 to increase power and found the same results. That is, there was a significant interaction between sentence type and trial type, p = .014), while the production duration was only numerically but not significantly longer on switch than stay trials in both simple-initial (M = 418 ms vs. 380 ms; p = .422) and complex-initial sentences (M = 364 ms vs. 349 ms; p = .803).

None of the other words exhibited any significant effects of trial type, sentence type, or their interaction (ps ≥ .10).

Discussion

The results of Experiment 2 replicated the results of Experiment 1, even with no salient perceptual difference between switch and stay trials and with a greater number of bilinguals than were tested in Experiment 1. In both experiments we found switch benefits on speech onset latencies in both complex and simple-initial sentences, switch costs in production durations of the first noun and the following conjunction word in complex-initial sentences, and no evidence of switch costs in simple-initial sentences until the switch word itself. The consistency of the patterns across experiments suggests the results reflect mechanisms of speech production and planning, and not an artifact of processing color cues specific to one condition.

In both experiments, the evidence of switch costs in simple-initial sentences came from the significant interaction between sentence type and trial type on Noun2 (“mesa”/ “table”). However, the trial type effect itself did not reach significance, even after we combined the data of two experiments. Before discussing this issue further, we conducted a third experiment on a different group of bilinguals – Chinese-English bilinguals (to be specific, Mandarin-English bilinguals) who are readily accessible at UCSD, to investigate the generalizability of the above reported results to different types of bilinguals.

Experiment 3

Method

Participants

Twenty-four Chinese-English bilingual undergraduates (four males, twenty females) at UCSD were recruited for course credit. Table 1 shows participant characteristics and MINT scores in both languages (Gollan et al., 2012). Different from Experiments 1 and 2, in which all bilinguals were English-dominant but had learned Spanish before English, in Experiment 3, bilinguals were Chinese-dominant and had learned Chinese before English.

Materials, Design & Procedure

The materials, design, and procedure were the same as in Experiment 1, except that the 2^nd picture was either named in English (e.g., “table”) or Chinese (e.g., “桌子”/“zhuo1zi”⁴). This means that in Experiment 3 the default language was still English, but participants switched from the nondominant language to the dominant one on the switch word, while participants in the previous two experiments switched from the dominant language to the nondominant one.

Results

Following the same data cleaning procedure as described in Experiments 1 and 2, 85.1% of the data were analyzed. The procedures for data analyses were also same as in Experiments 1 and 2, and the analyses of error rates (M = 9.2%) did not show any significant differences (ps > .67), except a marginally significant interaction between sentence type and trial type (β = −0.67, SE β = 0.38, χ²= 3.12, p = .077). Nevertheless, the trial type effect was not significant in either type of sentence (ps > .18) (see Table 2 for condition means). Of the errors produced, the most common types were within-language lexical errors (34.0%), followed by disfluencies (18.6%), incorrect language (12.4%), incorrect direction (11.3%), missing function words (10.3%), incomplete sentences (9.3%), incorrect structure (2.6%), and phonemic errors (1.5%). Figure 7 shows mean by-subject speech onset latencies and production durations in each condition, and Figure 8 shows the difference between switch and stay trials. Table 4 shows the corresponding statistics.

Figure 7.

Subject means of speech onset latencies and production durations of words until the 2^nd object for each condition in Experiment 3 (error bars refer to 95% confidence intervals). Panel A is for the complex-initial sentences, Panel B is for the simple-initial sentences. Again, given that speech onset latencies were much longer than production durations of words, their scales were different in this figure.

Figure 8.

Subject means of difference score between switch and stay trials for each sentence type in Experiment 3 (error bars refer to 95% confidence intervals). Positive values refer to switch costs, whereas negative values refer to switch benefits. Grey areas refer to the regions that showed evidence of switch costs. Panel A is for the complex-initial sentences, Panel B is for the simple-initial sentences.

Table 4.

Structure and results of speech onset latency and production duration analyses in Experiment 3 (Chinese-English Bilinguals; n=24)

Region	Fixed Effects	β (ms)	95% CI (ms)	χ2 (1)	p
Speech Onset Latency	Sentence Type	27	[2, 52]	4.43	0.035
	Trial Type	−52	[−74, −30]	19.93	< .001
	Sentence Type * Trial Type	−22	[−55, 10]	1.86	0.18
The1	Sentence Type	−1	[−5, 2]	0.74	0.389
	Trial Type	−3	[−8, 1]	1.35	0.245
	Sentence Type * Trial Type	−3	[−10, 5]	0.53	0.467
Noun1 (shoe)	Sentence Type	−16	[−27, −5]	15.92	< .001
	Trial Type	17	[12, 24]	22.60	< .001
	Sentence Type * Trial Type	28	[16, 39]	22.77	< .001
	Noun1 (complex-initial)
	Trial Type	32	[23, 40]	48.98	< .001
	Noun1 (simple-initial)
	Trial Type	4	[−3, 11]	1.14	0.286
and (complex-initial)	Trial Type	12	[4, 20]	6.64	0.009
moved (simple-initial)	Trial Type	4	[−5, 13]	0.72	0.397
above/below (simple-initial)	Trial Type	12	[2, 23]	4.55	0.033
the2	Sentence Type	−17	[−25, −8]	15.49	< .001
	Trial Type	9	[−14, 24]	0.57	0.448
	Sentence Type * Trial Type	−23	[−36, −11]	13.40	< .001
	the2(complex-initial)
	Trial Type	−3	[−15, 10]	0.177	0.675
	the2 (simple-initial)
	Trial Type	21	[4, 37]	6.08	0.014

Speech Onset Latency

As in Experiments 1 and 2, bilinguals initiated their speech more slowly when producing complex-initial than simple-initial sentences (M = 1457 ms vs. 1432 ms; p = .035) and began speaking faster on switch than stay trials (M = 1473 ms vs. 1417 ms; p < .001). The interaction between sentence type and trial type was not significant (p = .180) – switch benefits were equal in size and significant in both sentence types (ps < .001).

Word Production durations

Consistent with Experiment 1, the earliest evidence of switch costs was found on Noun1 (“shoe”). Bilinguals were slower to produce Noun1 (“shoe”) when they switched on Noun2 (“桌子”/“zhuo1zi”) than when they did not switch on Noun2 (“table”) (M = 358 ms vs. 338 ms; p < .001). Again, this effect was driven by complex-initial sentences (M = 357 ms vs. 322 ms; p < .001), while simple-initial sentences exhibited no difference between switch and stay trials (M = 358 ms vs. 352 ms; p = .286), a significant interaction between sentence type and trial type (p < .001). The main effect of sentence type was also significant (M = 340 ms vs. 355 ms for complex-initial vs. simple-initial sentences; p < .001). In complex-initial sentences, switch costs also spilled over to the following conjunction word “and” — bilinguals took longer to finish producing “and” on switch than on stay trials (M = 167 ms vs. 155 ms; p = .009).

In simple-initial sentences, bilinguals did not show a significant difference between switch and stay trials on the word “moved” (M = 342 ms vs. 338 ms; p = .397), but showed significant switch costs on the word “above” (M = 378 ms vs. 366 ms; p = .033) and the following determiner the2 (M = 149 ms vs. 128 ms; p = .014). In contrast, no switch costs were found on the2 in complex-initial sentences (M = 124 ms vs. 120 ms; p = .675), a significant interaction between sentence type and trial type on the2 (p < .001).

Discussion

Though Experiment 3 tested bilinguals who knew a different language combination, acquired a second language later in life, spoke Chinese instead of English as the dominant language and were also immersed in the nondominant instead of in the dominant language, all differences relative to Experiments 1–2, Experiment 3 largely replicated the results of Experiments 1–2. First, we found switch benefits on speech onset latencies in both types of sentences. Second, in complex-initial sentences, switch costs were found in production durations of the first noun and the following conjunction word. A small difference that emerged in Experiment 3 was that in simple-initial sentences, switch costs appeared on the word “above” and also on the word that followed i.e., “the” — the determiner before the switch word, which was earlier relative to Experiments 1 and 2 (where switch costs appeared on the switch word itself, i.e., “table”/ “mesa”).

General Discussion

The present study investigated when bilinguals pay the costs of switching out of the default language in connected speech, and how the scope of planning in connected speech is coordinated with bilingual language control mechanisms. Bilinguals were cued to switch out of English, the default language, based on previous training that made the names of switch words more accessible at the lexical level in the language of the switch (i.e., Spanish in Experiments 1 and 2, Chinese in Experiment 3). Language switches always occurred on the second noun in the display either within the first noun phrase in complex-initial sentences or at the beginning of the second noun phrase in simple-initial sentences. Replicating previous results reported in studies of monolingual sentence production, bilinguals took longer to initiate complex-initial sentences than simple-initial sentences, revealing the phrase as a critical planning unit in bilingual sentence production, both with and without language switching. Of greatest interest, bilinguals paid switch costs earlier in complex-initial than simple-initial sentences, so that switch costs were paid consistently at the beginning of whichever phrase included a language switch in both sentence types. Additionally, switching out of the default language did not elicit costs in speech onset latency in either type of sentence – instead, and most surprisingly, bilinguals started speaking faster when they had to switch languages than when they did not.

Phrasal Planning Precedes Payment for Language Switches

In all three experiments, we consistently found longer production durations on Noun1 (“shoe”) and the conjunction that followed it in complex-initial sentences on switch vs. stay trials, robust switch costs that were paid three words before the switches were produced (e.g., “The shoe and the mesa…”). By the time the determiner preceding the switch word was produced, switch costs had already been paid. In contrast, in Experiments 1–2, in simple-initial sentences no switch costs were found until the switch word itself was produced (e.g., “The shoe moved above the mesa…”), while in Experiment 3, switch costs were paid two words before the switch word and were still robust on the determiner preceding the switch word (e.g., “The shoe moved above the 桌子…”). Thus, across Experiments 1–3 switch costs were paid later in simple-initial than in complex-initial sentences. Importantly, we had trained bilinguals to name pictures just before their participation in the critical task, so that switch words were more accessible in the nondefault language. However, we still found significant language switch costs in spoken word durations in both types of sentences. Given that previous work showed that lexical accessibility-driven language switches were cost-free in isolated word production (Kleinman & Gollan, 2016), and the fact that lexical-level switch costs could not possibly explain why bilinguals paid the costs at different points in different sentence types, costs on word durations found in the present study should reflect the cost of having to violate default language selection in connected speech production. On this view, default language selection operates phrase by phrase, the scope of speech planning, and the cost of violating default language selection is therefore paid on or right before the word that begins the phrase. On this interpretation, bilinguals plan phrases before planning switches out of the default language. Furthermore, this implies that default language selection is planned phrase-by-phrase, consistent with phrasal planning scope. It is worth pointing out that the determiner “the” (but not a noun) was the first word of each noun phrase, but no costs were found on this word (except in simple-initial sentences in Experiment 3; see below). This could be because participants had to say “the” before every noun and may have treated “the + noun” as an integral unit in the present study. Alternatively, it may be difficult to detect significant differences across conditions on such a short high frequency word.

In simple-initial sentences, switch costs were paid earlier in Experiment 3 than Experiments 1–2, and these costs were still robust on the preceding determiner. This small difference between the experiments could be for multiple reasons. First, in Experiment 3 bilinguals switched from the nondominant language to the dominant language, whereas in the other two experiments they switched from the dominant to the nondominant language. Typically, switching from nondominant to dominant (as in Experiment 3) is more costly than switching from dominant to nondominant (as in Experiments 1 and 2), both in isolated word and in sentence production (e.g., Gollan, Schotter, et al., 2014; Li & Gollan, 2021; Meuter & Allport, 1999). As a result, switch costs were paid earlier in the situation of the more difficult switch – before the critical phrase was produced. This difference did not affect complex-initial sentences, probably because the critical phrase was already the very first phrase of the sentence. Second, the word order of “moved above the table and the tie” stays the same in Spanish, but should be changed to “moved to the table and the tie above” in Chinese. Given the strong grammatical constraints of default language selection (e.g., Gollan & Goldrick, 2016, 2018), this word order difference might encourage Chinese-English bilinguals to plan and pay switch costs earlier than Spanish-English bilinguals. This difference should not affect complex-initial sentences, as switching occurred before the verb phrase. Future research is needed to convincingly determine what causes differences like those observed here between Experiments 1–2 versus Experiment 3. However, in either case, the difference does not affect our conclusion that language switch costs were modulated by phrasal planning, as we consistently observed different patterns in complex-initial and simple-initial sentences.

In Experiments 1 and 2, we concluded that switch costs were paid at the second noun in simple-initial sentences based only on the significant interaction between trial type and sentence type, even though switch costs (i.e., the pairwise differences between switch and stay words) were not significant in simple-initial sentences on Noun2. It is unlikely that these differences were nonsignificant because of inherent limitations in our paradigm for detecting cross-phrase switch costs, given that in Experiment 3 we found significant switch costs on the two words that precede the target word in simple-initial sentences. An important consideration is likely that the compared words on switch versus stay trials for these switches were different (e.g., in Experiments 1–2 the switch word was always in Spanish while the nonswitch/stay word was in English, “mesa” versus “table”). Ideally, switch costs should involve comparison of the same word in different conditions; comparison of different words will add noise to true switch costs, especially when as in here, costs are assessed with duration measures. Thus, the interaction we observed between sentence type (simple- vs. complex-initial sentences) and trial type (switched vs. stay target words) gives us a reasonable basis for concluding that switch costs were indeed paid later during sentence production in all three experiments.

Note that in our study, bilinguals were trained to use the more accessible language, to facilitate switches on the specific lexical items. When bilinguals voluntarily choose which language to use, they may pay switch costs earlier than the phrase that includes a switch. In Sarkis and Montag (2020), Spanish-English bilinguals first named a set of pictures in their choice of either English or Spanish, and pictures named more often in English were identified as being more accessible in English. In a following task, the same objects were presented in pairs (e.g., a fork and a ball). Bilinguals were prompted to describe the location of one of them (e.g., the fork) in a Spanish sentence (e.g., “El tenedor está debajo de la pelota”; meaning “The fork is below the ball”). When language switching was allowed, greater lexical accessibility in English (e.g., “ball” is more accessible than “pelota”) led to higher switching rates (e.g., “El tenedor está debajo de la ball”), and longer production duration for utterances before switching occurred (e.g., “El tenedor está debajo de la”). According to the results of Experiments 1–2 in the present study which were also conducted with Spanish-English bilinguals, it seems that bilinguals should pay switch costs on the very last word “ball” when switching occurred. However, they paid costs earlier than that, although Sarkis and Montag (2020) did not report the exact words that showed switch costs. One possibility for these earlier switch costs is that in that study, bilinguals always switched from the nondominant language to the dominant language (like Experiment 3 in the present study). A more interesting possibility is that with both options (to switch or not), bilinguals might pay the costs earlier because they need to choose between using the more accessible language vs. violating default language selection, a question that can be explored in future research. If this interpretation is correct, cross-phrase language switch costs are likely to be paid earlier in voluntary switching regardless of the switch direction. Consistently, in spontaneous speech, switch costs were also typically paid before switch words were produced. For example, Fricke et al (2016) divided the Bangor Miami Corpus into a series of “utterances” that consist of one main clause in each and measured the average syllable production duration of each utterance. In the corpus, bilinguals may switch in both directions, and the results showed significantly longer syllable production duration in utterances before language switching occurred in general.

Switch Benefits on Speech Onset Latency

An unexpected result was that speech onset latencies exhibited robust switching benefits in both complex and simple-initial sentences. In Experiment 2 we excluded the possibility that this was caused by color differences for switch versus nonswitch trials. Another possibility is that the planning scope is narrowed in difficult situations (e.g., time pressure, Ferreira & Swets, 2002; more difficult event codability, van de Velde et al., 2014, high cognitive load, Wagner et al., 2010), and thus is narrowed due to switching in the present study. However, this seems unlikely as well, given that there was no room to narrow the planning scope in simple-initial sentences (i.e., just a single noun). If the planning scope of complex-initial sentences were narrowed, there should have been an interaction between trial type and sentence type and there should not have been switch benefits in simple-initial sentences (which we found in all experiments in the present study). Speculatively, switch benefits might have reflected a response of increased cognitive effort when switching was required. As speakers were aware that pictures would not stay on the screen indefinitely and also knew whether switching would be needed or not once pictures were presented, they might consciously or unconsciously have exerted more effort at the beginning when switching is required, trying to manage the switch word as soon as possible. Monolingual speakers have also showed similar benefits with higher working memory load in previous work (Ivanova & Ferreira, 2019). However, speeding up first might have led to later slow down, thus affecting production durations of later words. Importantly, this interpretation does not affect our account of switch costs and the level of processing at which default language selection operates, as this could not explain why the position where bilinguals started to slow down differed across sentence types.

Lastly, in the present study, bilinguals produced sentences primarily in English in all three experiments, and consistently showed evidence of switch benefits and phrasal planning despite the fact that English was the dominant language in the first two experiments while it was the nondominant language in Experiment 3. This result suggests that how bilinguals plan their utterances before speech onset is not affected by language dominance. However, other studies with different methods reached different conclusions, indicating that the scope of planning might in some circumstances be affected by language dominance — specifically that there is a larger scope of planning when bilinguals speak in the nondominant language (Gilbert et al., 2020; Konopka et al., 2018). This inconsistency might be due to muliple differences across studies, such as the sentence structures of the materials or bilinguals’ language backgrounds (e.g. in Experiments 1–2 bilinguals were heritage speakers whose dominant language was their second language, and in Experiment 3 bilinguals were immersed in the nondominant language, neither of which was true in studies by Gilbert et al. and Konopka et al.). Regardless of the exact reasons for this apparent discrepancy across studies, the present study reveals an orthogonal conclusion – i.e., that phrasal planning occurs before planning of switches to the nondefault language, and that this seems to hold whether the dominant or the nondominant language serves as the default language.

In conclusion, the present study showed how default language selection is coordinated with phrasal planning in bilingual sentence production. Three experiments with two very different bilingual groups consistently showed that phrasal planning occurs before planning of language switches. As such, when and how language switch costs are paid seems to be affected by the structure of the to-be-produced sentences and the syntactic position where the switch occurs. The fact that the differences were minor between Experiments 1 and 2 on the one hand and Experiment 3 on the other suggests that switching direction (to the dominant versus to the nondominant languages), and/or linguistic features of the two languages (or some other difference between Spanish-English and Chinese-English bilinguals tested herein) might also affect switching patterns, but the larger implication in terms of what is planned when appears to hold quite broadly – that is, phrasal planning in the default language occurs before the costs to switch out of the default language are paid. Additionally, the presence or absence of language switch costs in a particular processing measure may present only a small part of the picture involved in bilingual language control. Switches that appear to be “cost free” in speech onset latencies or in neuroimaging data (Blanco-Elorrieta & Pylkkänen, 2017; Zhu et al., 2022) at the point of the switch itself may in fact be costly in production durations or in different words in the utterance. The time window where bilinguals pay switch costs may depend on the syntactic position where language switching occurs (e.g., within or across phrases), but they do pay the costs somewhere in the speech. This paints of picture of bilingual speech production as a game of whack-a-mole⁵ where choices the speaker makes in one part of the utterance might have consequences in earlier or later parts (Gullifer & Titone, 2019) and reveals the danger of narrowly focusing on a single measure or production of single words which likely misses important aspects of what bilinguals do when they speak and why.

Appendix

Pictures served as the first object in Experiments 1–3

apple, arrow, basket, bird, book, bowl, boy, butterfly, cage, carrot, cow, dress, fish, ghost, girl, hammer, hand, hat, mirror, moon, orange, shoe, tree, umbrella

Pictures served as the third object in Experiments 1–3

bear, bell, belt, cake, car, chair, cheese, chicken, cloud, door, egg, fork, fox, glasses, kite, knife, leaf, ring, saw, shirt, skirt, spoon, tie, window