Research on bilingualism as discovery science

Abstract

An important aim of research on bilingualism is to understand how the brain adapts to the demands of using more than one language. In this paper, we argue that pursuing such an aim entails valuing our research as a discovery process that acts on variety. Prescriptions about sample size and methodology, rightly aimed at establishing a sound basis for generalization, should be understood as being in the service of science as a discovery process. We propose and illustrate by drawing from previous and contemporary examples within brain and cognitive sciences, that this necessitates exploring the neural bases of bilingual phenotypes: the adaptive variety induced through the interplay of biology and culture. We identify the conceptual and methodological prerequisites for such exploration and briefly allude to the publication practices that afford it as a community practice and to the risk of allowing methodological prescriptions, rather than discovery, to dominate the research endeavor.

“We have to remember that what we observe is not nature in itself, but nature exposed to our method of questioning.” Werner Heisenberg (1958)

1. Introduction

Research on bilingualism generates debate on the neural bases of language that address fundamental questions about language learning (e.g., the role of critical periods), the specificity of language networks (e. g., the nature of any modularity) and their control (e.g., the domain-generality of such control). More recently, specific aspects of the field, namely the putative cognitive and neural consequences (often framed in the form of advantages) of bilingualism, have become a hotspot for controversy tied to the replication crisis in psychology. The critique of this research appears to be broad, addressing issues of power and sample size (e.g., Brysbaert, 2020; Nichols, Wild, Stojanoski, Battista, & Owen, 2020), failures to replicate (e.g., Paap & Greenberg, 2013), noise in samples and methods (e.g., García-Pentón, Fernández García, Costello, Duñabeitia, & Carreiras, 2016a, 2016b; Valian, 2015), and publication bias (e.g., de Bruin, Treccani, & Della Sala, 2015a; but see Bialystok, Kroll, Green, MacWhinney, & Craik, 2015), suggesting that the effects of bilingualism on cognitive and brain functioning are the result of questionable research practices. Consequently, several prescribed remedies, such as large samples (Brysbaert, 2020) and uniform¹ experimental procedures (García-Pentón et al., 2016a, 2016b), have been marketed as solutions (see also Szucs & Ioannidis, 2020 for an example involving neuroscience more generally). However, such critiques and remedies, though well intended, often fail to place discussions in the broader context of science and its function throughout history. This raises the question of how the implementation of compulsory prescriptions would come to affect research on bilingualism more generally.

In this paper, we argue that the remedies and prescriptions put forward are deceptively simple and place us on a misleading path as they are based on a mischaracterization of the fundamentals of the scientific endeavor. While this paper is geared toward discussing current issues in research on bilingualism, we necessarily draw from the history of science to make the argument self-evident. Our position is that both large samples and conventionalized methods are important, but their role needs to be understood in the context of science as a discovery process, in which research findings are generated through interrelated iterations of exploration and falsification, which in turn lead to new insights and allow for the formulation of new questions. Fundamental to this process is the generation of variety² that permits incremental advance. The generation of variety serves two purposes: to identify reliable signals in the noise of our observations and to allow the formulation of effective theories and constructs about our world. Hypotheses, for instance, that the shape of the head is correlated with psychological traits (Simpson, 2005) or that bilingualism negatively impacts intelligence (Peal & Lambert, 1962), are discarded along the way. Constraints on the exploration of variety, such as those imposed by prescriptive remedies (e.g., keeping experimental designs as simple as possible), hinder the discovery process and so it is imperative in our view to ensure that methodological injunctions and publication practices are understood within the context of science as a discovery process.

The remainder of the paper is organized as follows: in the next section (Why a prescriptive science is problematic), we provide a critique on both practical and conceptual grounds of the rationale for power and uniformity prescriptions. Our line of argument then leads us to consider the implications for research practice in bilingualism (Articulating the research enterprise of bilingualism), where we emphasize the value of rich characterization of the sample, practices that enable the assessment of interactions rather than main effects on their own, and the application of sensitive tools. The implication is that without appropriate characterization, and without research practices and tools that lead to effective signal extraction, replication and large samples may be void of scientific interest. In both sections, we illustrate the manifestation of science as a discovery process with a range of past and contemporary examples drawn from research on bilingualism as well as from other fields. We necessarily draw on a range of examples, including those outside bilingualism, because these points are not unique to research on bilingualism; rather, they reflect a healthy and productive scientific enterprise. We do not argue against the importance of replication, the analytic value of Big Data, nor the application of sensitive and conventionalized research tools. Rather, we suggest that the application of method should be grounded in science as a discovery process.

2. Why a prescriptive science is problematic

We proposed above that prescriptions to remedy poor research practice fail to adequately acknowledge science as a discovery process. Curiously, in applying these prescriptions to research on bilingualism, the analogy invoked is bringing an image into focus: just as glasses improve blurry vision, larger samples have been claimed to increase the resolution of data (Brysbaert, 2020). Similarly, methodological uniformity transforms haziness into a well-defined picture (García-Pentón et al., 2016a, 2016b). Such analogies, rhetorically persuasive perhaps, are misleading. There is a sensible motivation to establish the stability of effect sizes for a given class of data (see Lorca-Puls et al., 2018 for actual rather than simulated data in the context of the relation between brain damage and speech articulation), but it is our ability to identify a reliable signal that is key, not sample size or uniformity per se. Below we comment on four points to illustrate why power and uniformity prescriptions are insufficient for effective signal extraction.

2.1. Ambiguity is independent of power

Studies of individuals who speak two or more languages have demonstrated a range of consequences for cognition (see Bialystok, 2017 for a review) but controversy surrounds some of these effects. Many large sample studies have yielded null results (Antón et al., 2014; Dick et al., 2019; Duñabeitia et al., 2014; Kalamala, Szewczyk, Chuderski, Senderecka, & Wodniecka, 2020; Nichols et al., 2020; Paap, Anders-Jefferson, Mason, Alvarado, & Zimiga, 2018) and other meta-analyses report inconsistency (Anderson, Hawrylewicz, & Grundy, 2020; Donnelly, Brooks, & Homer, 2019; Grundy & Timmer, 2017; Lehtonen et al., 2018; Mukadam, Sommerlad, & Livingston, 2017; Schroeder, 2018; Sulpizio, Del Maschio, Fedeli, & Abutalebi, 2020). On the face of it, such reports have called putative bilingualism effects into question. However, if statistical power were indeed the solution to ambiguity, then we would expect greater consistency across studies with large samples. Problematically, from a naive prescriptive approach, other large sample studies do report effects of bilingualism (Bak, Nissan, Allerhand, & Deary, 2014; Hartanto, Toh, & Yang, 2018; Santillán & Khurana, 2017). Are these latter studies like “black swans” reducing our belief in the generalization that “all swans are white”?

We need not take the current impasse at face value. Consider a contemporary example. The COVID-19 pandemic made it urgent for scientific communities to address a critical question: does the human body develop long-term immunity to the virus? While some large-scale studies suggest that it does (e.g., Iyer et al., 2020; Wu et al., 2020), other large-scale studies show that the effects are limited (e.g., Liu, Wu, Tao, Zeng, & Zhou, 2020; Pollán et al., 2020). Curiously, it is the collection of single-case patients with reinfection (e.g., Tillett et al., 2020) that initially became more decisive in addressing this question. The point here is that, without an understanding of the boundary conditions of an effect, power, in the form of large sample studies, does not, on its own, improve our ability to extract a signal. In fact, as reinfection cases suggest, and as we further illustrate below, small-n studies that exploit the features of the sample can be more informative than studies with poorly characterized large samples, which are bound to increase noise in our signals.³

2.2. Discovery and the power of the small

The history of science suggests that we should recognize the value of the small sample to increase signal.⁴ Research on bilingualism, as well as cognitive and neuroscience research more generally, also attests to this point. Consider a fundamental question: How does the brain adapt to input deprivation? One claim is that brain specialization is determined by input senses. Vetter et al. (2020) examined brain activity in the primary visual cortex in healthy blindfolded (n = 10) and congenitally blind (n = 5) individuals while listening to natural sounds. Using multivariate pattern analysis, they found that the blindfolded participants activated the primary visual cortex in response to the sounds despite not having access to visual input in the moment. Remarkably, the same pattern of activation was observed in the congenitally blind group, suggesting that it is not sensorial input per se, but rather, the tasks performed by a brain region, that shape brain specialization. It is not sample size, but signal quality that is key here.

Neuropsychological data, typically based on a small number of cases, have been instrumental for our understanding of memory systems⁵, but have also been critical to the emergence of research on language control in bilinguals, as they establish the face validity of the distinction between language networks and their control (Green & Kroll, 2019). For example, S.J., a Friulian-Italian speaker (Fabbro, Skrap, & Aglioti, 2000), had intact clausal processing for speech comprehension and speech production in both languages, combined with an inability to avoid switching inappropriately in a conversation (e.g., into Friulian when speaking to an Italian-only speaker). More complex control problems reveal dissociations between speech production in one language and translation into it, as exemplified in the alternate antagonism and paradoxical translation of two bilingual patients (Paradis, Goldblum, & Abidi, 1982), modelled narratively in Green (1986) and neurocomputationally in Noor, Friston, Ekert, Price, and Green (2020). Such cases pave the way for neuroimaging research on the nature of recovery in bilingual aphasia in which we can ask, for example, whether recovery depends on perilesional activation or the use of a previously inhibited alternative network.

A final example exploits the presence of bilingualism in two different modalities –speech and sign language. Hearing bimodal bilinguals are a small population of speakers. They are typically either children of deaf adults or sign interpreters. Bimodal bilinguals are able to do something that is impossible to do with two spoken languages, namely speaking one language while simultaneously signing another (i.e., code-blending). Because of this feature, bimodal bilingualism provides a unique opportunity to test claims about how the bilingual’s languages are controlled. Initial naturalistic production data from two children (Petitto et al., 2001) and 11 adults (Emmorey, Borinstein, Thompson, & Gollan, 2008) showed that bimodal bilinguals strongly prefer code-blending over switching between sign and speech, suggesting that combining the two languages is relatively free of control demands. More recent work using magnetoencephalography (Blanco-Elorrieta, Emmorey, & Pylkkänen, 2018; see also Emmorey, Li, Petrich, & Gollan, 2020 for converging behavioral evidence) confirmed this finding, but also showed that increased cognitive effort is required when bimodal bilinguals switch out of a code-blend to either language alone, suggesting that it is the disengagement of one language to switch into the other that requires active control.

The point is that, so long the data are adequately characterized (see ‘A rich characterization of the sample and an identification of boundary conditions’ subsection) and measures are sensitive (see ‘Realizing signal extraction’ subsection), a small sample, even a single case, that exploits the special properties of a particular population allow for effective signal extraction that can generate new observations and move the field forward. This is not to say that large-sample studies cannot be equally informative, or that these findings should not be replicated, but that the force of the evidence is not based on statistical power alone.

2.3. Discovery acts on variety

Science acts as a discovery process with Darwinian-like properties, except it also possesses a time-binding property in which earlier ideas and methods can be recruited at a later point in time.⁶ Just as natural selection depends on biological diversification to ensure the continuation of evolution, science relies on variety to ensure incremental improvements in our signal-extraction abilities. Progress, in the form of new discoveries and insights, is made by the gradual accumulation of patterns that emerge over distinct data and methods, a process that William Whewell referred to as consilience (Laudan, 1971).

In some cases, consilience is relatively straightforward: Converging evidence is obtained from variations of a method (see Green & Abutalebi, 2015 for an example of left caudate involvement in language control). For instance, in examining the question of whether bilingualism changes the engagement of control processes, Wu and Thierry (2013) found in a group of Welsh-English bilinguals a modulation of Flanker performance by experimentally inducing a shift in the language context. Using a novel paradigm in which Flanker was interleaved with words from Welsh or English (single-language context) or both languages (dual-language context), they showed that exposure to words in a dual-language context led to greater electrophysiological efficiency in Flanker performance. Since then, several studies have also reported electrophysiological Flanker modulations using variations of the paradigm in different bilingual populations (Bosma & Pablos, 2020; Jiao, Grundy, Liu, & Chen, 2020; Jiao, Liu, de Bruin, & Chen, 2020), suggesting that the effect reflects a more general feature of bilingualism. This is an important discovery not only because it shows how control processes adapt to the language context, but also because it makes a more general point that the relative involvement of control processes on a particular task will depend on the control state of an individual at a particular time (see Hsu, Kuchinsky, & Novick, 2020; Salig, Valdés Kroff, Robert, & Novick, 2021 for an elaboration of this argument).

In other cases, consilience requires us to bring together evidence from different methods and populations. Consider the claim that language processing is determined by factors unique to the language system. Converging evidence from neuroscience suggests that domain-general processes also play a vital role. For instance, in studies examining monolingual brain activity across a variety of linguistic and non-linguistic conflict-related tasks (Hsu, Jaeggi, & Novick, 2017), colocalization and functional connectivity analyses reveal that, although activation of the Multiple Demands system varies across tasks, engagement of the left inferior frontal gyrus is constant across tasks while also co-activating with other task-specific networks. Research on bilingualism, too, attests to this idea (KK Nair, Rayner, Siyambalapitiya, & Biedermann, 2021). In proficient bilinguals, brain potentials reveal that the ability to recover from prediction errors during L2 sentence reading is mediated by individual differences in control ability, but this effect depends on L1 verbal fluency (Zirnstein, van Hell, & Kroll, 2018). The interaction between control and fluency suggests that successful L2 prediction may depend on language-related processes that are partially overlapping with more domain-general control processes. It is the coordination, not the presence or absence, of particular processes or brain regions that is relevant (see also Bialystok, 2011; Morales, Gómez-Ariza, & Bajo, 2013).

What makes these ideas (i.e., that control processes are state dependent, or that language draws from both domain-general and language-specific resources) compelling is not the ability to replicate a finding using the same method ad infinitum. Rather, it is the fact that we can identify converging patterns despite the use of different tools, procedures, and populations with varying sample sizes, all of which might seem to work against us but may in fact improve identification (i.e., signal quality). Such insights allow us to ask new and more useful questions. That is why deep insights about language and the brain emerge through the application of variety.

2.4. Focus is meaningless without context

Analogies invoking focus of an image via large samples and uniformity minimize the conceptual basis on which an observation is made and are fundamentally misleading for a simple reason. We only know the significance of increased focus because we already know the picture (i. e., the conceptual ground). Experience generates the conceptual ground for our everyday lives: we learn to recognize different objects and entities through our ability to interact with them over time. But the conceptual ground for the processes and causal mechanisms underlying the brain and behavior are typically unknown. Science as a discovery process fundamentally concerns the identification of effective theories and constructs of those unknowns. Such theories are based on our justified true beliefs given the evidence and are an intersubjective agreement about that evidence. Theories contest for that agreement. They are necessarily an intersubjective agreement because our senses and scientific tools do not provide immediate access to the physical world⁷, as the introductory quote by German physicist Werner Heisenberg suggests. Thus, we come to know the significance of increased focus not by power or uniformity, but by generating and exploring the conceptual ground, as Fig. 1 gently illustrates. The problem lies not in the pursuit of statistical power or conventionalized tools per se, but in assuming that increasing power or achieving uniformity will generate a picture. As the Blind Men and an Elephant parable suggests, a thousand blind people inspecting separate parts of an elephant will yield an enormous effect size for their one bit but yield zilch for totality!

Fig. 1.

a (left) and b (right). We formulate our hypotheses in the context of discovery – a possible picture of the world (a). Bringing an event into focus without context (i.e., testing a hypothesis without a conceptual ground) is meaningless (b).

3. Articulating the research enterprise of bilingualism

The key question is how we generate the conceptual ground for effective theories and constructs on research on bilingualism and its consequences. We do so by recognizing that language and the brain are byproducts of evolutionary and ecological processes. Such recognition is a generator of the expertise and intuitions for researchers and can play an important role in recognizing the significance of a chance observation or novel finding just as experience furnishes the hunches of everyday life (Bowers, Regehr, Balthazard, & Parker, 1990). We propose two key factors. First, pre-existing evolutionary older systems are coordinated and the use of these systems exapts⁸ mechanisms for the control of language and action more generally (Stout & Chaminade, 2012). Bilingual speakers necessarily must select and control the language of use (e. g., Green, 1986, 1998; see Pliatsikas & Luk, 2016 for a review of data on the overlap). Second, and critical for research on bilingualism, there is a need to characterize the socio-cultural niche in which speakers act. To this end, we emphasize the need for research practices and tools that provide a rich characterization of the participant sample, and ultimately envision a research enterprise focused on the identification of bilingual phenotypes. Under this notion, interactions become of high relevance, and questions eliciting binary outcomes (e.g., is there a bilingual advantage?) become inadequate as they mask the richness of the science. Below we consider each of these points more carefully.

3.1. A rich characterization of the sample and an identification of boundary conditions

The interactional contexts of language use constrain which languages can be used and how they can be used. Characterizing speakers in terms of their habitual community practices, as well as their trajectory through particular contexts, is essential if we are to understand adaptive change⁹. We refer to such characterization as bilingual phenotyping (Adamou, 2010; Beatty-Martínez & Dussias, 2017; Beatty-Martínez et al., 2020; Poplack, 1987). Such phenotyping helps determine the boundary conditions for any adaptive effect because we might predict an effect for one phenotype but not for another (Bak, 2016). On the grounds of degeneracy (see footnote 9), we can ask what kinds of cognitive and neural changes might be expected given the demands of particular contexts on language and the control processes supporting it (DeLuca, Segaert, Mazaheri, & Krott, 2020; Green & Abutalebi, 2013). We propose that a plausible answer to this question will require the application of ethnographic practices in brain and cognitive sciences (see Billig, 2020; Torres Cacoullos & Travis, 2018 for illustrations in psychology and variationist linguistics, respectively) tied to multi-lab collaborations (Leivada, Westergaard, Duñabeitia, & Rothman, 2020). We illustrate with a comparison of two Spanish-speaking locations, San Juan, Puerto Rico, and Granada, Spain, to highlight three key aspects of rich characterization using ethnography.

First, we need to be aware of the diachronic processes that have shaped the culture and history of a community. Granada is located in the community of Andalusia, officially considered one of several monolingual autonomous communities of Spain. Despite having a longstanding influence of Arabic culture, Andalusia has historically perpetuated ideologies tied to monolingualism, especially throughout the 20th century under the Francoist regime (see Lorenzo & Moore, 2009). As a result, foreign-language prevalence has remained lower in Andalusia relative to the rest of Spain and Europe (de Educación, 2012), creating fewer opportunities for other languages such as English to influence everyday language use. By comparison, Puerto Rico is the byproduct of a rich colonial history spanning across five centuries until the present (see Guzzardo Tamargo, Loureiro-Rodríguez, Acar, & Vélez Avilés, 2018). Although Spanish had been the established language following four centuries of Spanish colonial rule, the island became a US territory after the Spanish-American war at the end of the 19th century and continues to this day to be a non-incorporated territory of the US. Unlike Granada, the history of Puerto Rico created conditions in which American culture would become highly influential for the already established Hispanic culture, especially in the metropolitan area of San Juan.

The observation and description of current community practices in a well-defined speech community¹⁰ (Labov, 2001) leads to the second feature of rich characterization. Determining how bilinguals’ languages are habitually used (e.g., whether a speaker has extensive experience codeswitching or not) is important, but doing so requires an understanding of how the global environment of a community supports those practices. For instance, Spanish-English codeswitching is often a prominent form of communication among proficient bilinguals in San Juan whereas speakers from Granada tend to use their languages separately (see Beatty-Martínez & Dussias, 2017; Beatty-Martínez et al., 2020; Beatty-Martínez, Navarro-Torres, & Dussias, 2020 for a more comprehensive characterization). But in terms of interactional demands, the key difference between these two communities is not the frequency of switching behavior per se¹¹; rather, it is the fact that, given the history and culture of each location, one context (i.e., San Juan) enables speakers to use both languages more openly and opportunistically with little-to-no interactional costs, whereas in the other context (i.e., Granada), there is a strong tendency to expect the use of Spanish (the L1) most of the time, creating constraints as to when speakers expect the use of the L2. This is not to say that codeswitching is not critical to understand how bilinguals control their languages (see Adler et al., 2020; Green, 2018; Hofweber et al., 2019 for how control processes may be engaged during codeswitching), but that the relative involvement of control processes during different kinds of speech acts may also depend on the demands imposed by the global environment. Characterizing speakers in terms of their habitual community practices is vital to understand such dynamics.

The final feature of rich characterization relates to changes in an individual’s trajectory of experiences. While some bilinguals live in homogeneous communities where the language dynamics are relatively stable over the lifespan, other bilinguals undergo radical shifts in language use at particular time points (see Kubota, Chevalier, & Sorace, 2020; Pallier et al., 2003 for examples involving international returnees and adoptees). To illustrate, speakers may initially grow up in a home environment where a minority language (e.g., Spanish in the US) is used, but can then become educated and socialized in the majority language of the community (e.g., English in the US) during childhood. As such, some bilinguals (a.k.a., heritage speakers and indigenous-speaking bilinguals) may grow up and become educated in a context where the L1 is the majority language but then shift to an environment where the L2 becomes the dominant language (e.g., Garraffa, Beveridge, & Sorace, 2015; Garraffa, Obregon, & Sorace, 2017; Bonfieni, Branigan, Pickering, & Sorace, 2019; Polinsky & Scontras, 2020). A similar case is observed with young adults seeking higher education who relocate to a new environment (e.g., a foreign country) with a different predominant language (Beatty-Martínez, Bruni, Bajo, & Dussias, 2020). These shifts in language immersion status are likely to generate unique adaptive brain responses. Indeed, emerging evidence suggests that heritage speakers’ initial minority-language experience has long-term consequences for language processing in the majority language (Bice & Kroll, 2021) and that contexts with high linguistic diversity (Gullifer et al., 2018) or L1- to-L2-immersion shifts (Beatty-Martínez et al., 2020) may trigger a novel adaptation of control processes in the form of proactive control engagement (see also Blanco-Elorrieta & Pylkkänen, 2018 for a similar observation regarding dual-language contexts).

Although we still know little about the boundary conditions of these effects, the point is that in a main-effect group analysis, different groups of speakers would be assumed to represent the same underlying population of bilinguals (see Weyman, Shake, & Redifer, 2020, for an illustration), despite having remarkably different community practices and/or individual trajectories that become evident through rich characterization. In making such an assumption, we may miss critical information that can change our conclusions. Hence, research on bilingualism¹² is likely to benefit more from small sample studies with rich characterization.

3.2. Beyond main effects and binary oppositions

Under a traditional lens in research on bilingualism, idiosyncratic patterns are typically discarded as random noise and complexity is equated with complication. Despite several notable critiques to this approach (Baum & Titone, 2014; Fricke, Zirnstein, Navarro-Torres, & Kroll, 2019; Luk & Bialystok, 2013; Tanner, McLaughlin, Herschensohn, & Osterhout, 2013), binary classifications and group comparisons, together with recommendations to keep experimental designs as simple as possible (Brysbaert, 2020), continue to dominate much of research on bilingualism, forcing discussions into a binary opposition not unlike those that have recently characterized the consequences of bilingualism (Nichols et al., 2020; c.f. Leivada et al., 2020), as well as psychological research more generally (Newell, 1973). But given the degenerate nature of biological and cognitive systems, solely main effect ‘yes’ or ‘no’ questions are unhelpful. As Bronfenbrenner (1977, p. 518) noted, “in ecological research, the principal main effects are likely to be interactions.” To illustrate this point, we return to the study by Zirnstein et al. (2018; see ‘Discovery acts on variety’ subsection).

Following a main-effect analysis, Zirnstein and colleagues found different electrophysiological responses for L1 and L2 speakers to predictions errors during sentence reading: Only L1 speakers showed reliable electrophysiological costs when encountering semantically unexpected words. At first glance, it would be tempting to conclude that L2 speakers were unable to generate predictions, consistent with previous claims (e.g., Martin et al., 2013; Grüter, Rohde, & Schafer, 2017). But instead of asking whether L2 speakers can generate predictions (a ‘yes’ or ‘no’ question), one can ask about the cognitive processes that enable prediction in the first place. Upon examining individual differences in control and verbal fluency in both groups, Zirnstein and colleagues identified a more complex, but also more insightful, picture. First, both L1 and L2 speakers recruited control processes to recover from prediction errors (i.e., increased control ability related to reduced prediction costs). But for L2 speakers, as mentioned previously, there was an interaction between control and L1 fluency, such that increased L1 fluency related to larger prediction costs in the L2. This suggests that L2 speakers had to overcome the challenge of regulating the L1 in order to engage prediction mechanisms in ways comparable to L1 speakers. More critically, the interaction reveals that the absence of an electrophysiological response in the L2 group stemmed from an aggregate of bilingual phenotypes with different configurations of control and regulatory engagement. If we had only asked whether L2 speakers can generate predictions, we might have come to a different conclusion (see also Pulido, 2021; Tanner & van Hell, 2014 for illustrations with adult L2 learners and monolinguals, respectively).

The point is that a simple main-effects approach focused on attaining large samples or replication-via-uniformity would disregard the fact that the form of language and cognitive engagement varies across individuals. Arguably, there may be some important main effects, but the way to identify them is by first seeking out meaningful interactions that are informed by a rich characterization (see Rohrer & Arslan, 2021 for a discussion on the application of interactions). As a discovery process, science benefits from relatively open-ended questions such as “how do these regions in the temporal lobe dissociate during different tasks?” or “what are the possible range of phenotypes that can emerge in this community?”. As Calhoun and Bandettini (2020) point out, such questions cast an effective net in making sense of large amounts of data¹³.

3.3. Realizing signal extraction

Rich characterization, as well as the framing of our questions, is vital for effective signal extraction, but just as important is determining task and test sensitivity (e.g., for a given sample, to what extent do we expect a non-verbal task to tap executive processes used in language control so that any putative adaptive response of language experience could be realized?). We comment on four aspects. First, task sensitivity might require a revised conception of the task construct that it is designed to tap, such as using within-subject paradigms that allow us to induce different control states and track how they are engaged during language processing (e.g., Adler et al., 2020; Hsu et al., 2020; Navarro-Torres, Garcia, Chidambaram, & Kroll, 2020; Salig et al., 2021), as opposed to exclusively relying on aggregate executive function measures (e.g., Stroop effects) that likely mask degenerate patterns. As such, the insensitivity or the appropriateness of the test to tap into control processes engaged in language use limits their relevance for exploring any putative wider effects on non-verbal control tasks, as acknowledged in recent papers reporting data based on more richly characterized large samples (Gullifer & Titone, 2020a; Kalamala et al., 2020; Kheder & Kaan, 2021).

Second, deepening theoretical understanding also requires that we understand the totality of performance for which we need to consider data from a number of modalities – some of which may be more sensitive to the effects of interest than others. For instance, brain measures may better capture some aspects of early L2 learning than overt behavioral responses (e.g., Bice & Kroll, 2015, 2019; Kurkela, Hämäläinen, Leppänen, Shu, & Astikainen, 2019; McLaughlin, Osterhout, & Kim, 2004), although in other cases, both brain and behavior converge (see Li, Legault, & Litcofsky, 2014 for a review on imaging studies). Multi-lab collaborations can be an effective way to explore the boundary conditions of such issues for a given task and sample. We note, however, that the goal should not be replication per se –using paradigms that are simple enough to easily reproduce—but to see collaborations in ways that are designed to exploit variation across different labs and different locations (Leivada et al., 2020).

Third, it is important that we use measures that are reasonably commensurate with the questions being asked. For instance, with respect to determining how bilinguals’ languages are habitually used, self-reported data can be informative (e.g., Gullifer & Titone, 2020b) but likely insufficient in the absence of conversational data that correspond to the vernacular of the speech community (Labov, 1984) or that reflect engagement of different attentional/control states when bilinguals shift between different modes of communication (Green, 2019).¹⁴ The application of Network Science (Tiv, Gullifer, Feng, & Titone, 2020) and Information Theory (Gullifer & Titone, 2020b; Feldman, Srinivasan, Fernandes, & Shaikh, 2021) practices can also be of high value regarding effective phenotyping as they can help us establish correspondence between individual differences in language experience and the extent to which those trajectories reflect (or deviate from) more general community practices.

Finally, more sensitive data analysis practices are likely to be more revealing of individual differences and degenerate patterns. For instance, using ex-Gaussian distributions (Sundh, Collsiöö, Millroth, & Juslin, 2021; von Bastian, Blais, Brewer, Gyurkovics, Hedge, Kalamala, & Wiemers, 2020), delta plots (Morales et al., 2013), or Bayesian mixture models (Ferrigno, Cheyette, Piantadosi, & Cantlon, 2020) to infer the possible range of strategies in a given task, rather than simply averaging effects for a condition. Individual differences also allow us to construct generative models of behavior and neuroplasticity (see Parr, Rees, & Friston, 2018 for an example in neuropsychology) which can be used to computationally model, say, neuroplastic effects of different interactional contexts given a set of behavioral profiles. Further, within a large sample, there may be different phenotypes and we need to be able to explore and characterize these using data-driven techniques such as multivariate statistics (e.g., cluster and/or factor analyses; Hartanto & Yang, 2020; Rodriguez-Fornells, Krämer, Lorenzo-Seva, Festman, & Münte, 2012) if the data are sufficiently rich to detect different profiles. However, although individual differences offer an opportunity for effective phenotyping, they potentially involve the same risks as those observed with main-effect practices in the absence of a rich characterization tied to well-defined speech communities.

We consider it likely that all four aspects are pertinent to advance. Further, in some cases, and to reinforce our earlier point (see ‘Discovery and power of the small’ subsection), only small samples may be feasible and yield decisive evidence. For example, localization of phonemic restoration effects in the auditory cortices is best achieved through the high signal-to-noise ratio afforded by electrocorticography arrays implants for clinical purposes (see Leonard, Baud, Sjerps, & Chang, 2016), and so establishing convergence in bilingual speakers in two languages may sometimes require small samples with rich characterization (see “A rich characterization of the sample and an identification of boundary conditions” subsection). In short, replication, conventionalized tools, and large samples have value, but their role in the discovery process in research on bilingualism hinges on conceptual, experimental, and analytic advance.

4. Concluding remarks

In this paper, we have emphasized the community-value of incremental contributions via science as a discovery process against the enforcement of prescribed remedies, such as pre-determined sample sizes and/or methods, because we trust in the basic integrity of participants in the enterprise of research on bilingualism and ultimately in the self-correcting dynamic of science itself.

From the point of view of ensuring variety on which the quasi-Darwinian process of science can act, we require the publication of possibilities (e.g., sensitive tasks geared to testing specific processes, rich characterization to identify phenotypic variation, and non-binary questions that enable the exploration of interactions) that may or may not lead to deeper understanding. Replication and reproducibility efforts, important as they are to the scientific enterprise, need to be in service of such aims to advance. As the National Academies of Sciences, Engineering, and Medicine recently acknowledged: “The goal of science is not to compare or replicate [studies], but to understand the overall effect of a group of studies and the body of knowledge that emerges from them” (The National Academies of Sciences, Engineering, and Medicine, 2019, as cited in Miceli, 2019, p. 14). Like the brain, bilingualism is complex, and we are far from having a complete understanding of the boundary conditions of previously reported findings for replicability to be fruitful on its own. And while some have proposed that such understanding lies in the data itself (de Bruin, Treccani, & Della Sala, 2015b), we make the point that the answers ultimately lie in the characterization (i.e., the intersubjective agreement) of the data.

Finally, in establishing the need to view science as a discovery process, we wish to return to the question raised in the introduction of how compulsory prescriptions would come to affect research on bilingualism, as well as psychological and neuroscience research more generally. If we choose to allow prescriptions to dominate the scientific enterprise, then we must ask how they will come to shape not only the environment in which research is currently being conducted, but also how they will shape the minds of young and early-career researchers, and ultimately, whether we are willing to live with the consequences of those choices. Thus, in articulating the research enterprise of bilingualism, we hope to contribute to the establishment of a viable future research enterprise more generally.