Measuring Bilingualism: The Quest for a “Bilingualism Quotient”

Abstract

The study of bilingualism has a history that extends from deciphering ancient multilingual texts to mapping the structure of the multilingual brain. The language experiences of individual bilinguals are equally diverse and characterized by unique contexts of acquisition and use that can shape not only sociocultural identity, but also cognitive and neural function. Perhaps unsurprisingly, this variability in scholarly perspectives and language experiences has given rise to a range of methods for defining bilingualism. The goal of this paper is to initiate a conversation about the utility of a more unified approach to how we think about, study, and measure bilingualism. Using concrete case studies, we illustrate the value of enhancing communication and streamlining terminology across researchers with different methodologies within questions, different questions within domains, and different domains within scientific inquiry. We specifically consider the utility and feasibility of a Bilingualism Quotient (BQ) construct, discuss the idea of a BQ relative to the well-established Intelligence Quotient (IQ), and include recommendations for next steps. We conclude that though the variability in language backgrounds and approaches to defining bilingualism presents significant challenges, concerted efforts to systematize and synthesize research across the field may enable the construction of a valid and generalizable index of multilingual experience.

1. Measuring Bilingualism

1a. Introduction

“How bilingual are you?” Anyone who speaks multiple languages is likely to have been questioned about their experiences and abilities - questions such as how, when, and why different languages were acquired. Each of these queries can be deceptively difficult to answer, but attempting to quantify “how bilingual” an individual is (especially relative to others) is particularly challenging due to the many forms that bilingualism can take. Consider, for instance, a native English speaker who began learning Spanish in college, who has little exposure to native Spanish speakers, but has attained high levels of proficiency reading and writing due to continued independent study and engagement with Spanish literature. Now consider someone exposed to both English and Spanish from birth, who frequently code-switches within the home, but has a limited Spanish vocabulary due to the fact that English is the primary language everywhere else (see Figure 1). These two bilinguals differ in their age and manner of acquisition, proficiency, language use, language switching, and language identity - but which of them would be considered “more” bilingual? Though the answer may not be of special importance to any given individual, the ability to quantify and compare levels of bilingualism is critical for those conducting research with bilingual populations. The goal of this article is to pose the question of whether the essence of bilingualism can or should be captured using a single quotient, and if not, how we can work towards gaining a more complete characterization of language background. To this end, we provide an overview of existing methods and challenges for bilingualism researchers, and call for greater communication and consensus as a way to move forward.

Figure 1.

Illustration of the variability in language profiles between two bilinguals.

1b. Quantifying Bilingualism: Self-Reports and Standardized Tests

To address the need for reliable measures of bilingualism, the Language Experience and Proficiency Questionnaire, or LEAP-Q (Marian, Blumenfeld, & Kaushanskaya, 2007) was developed and made freely available online for the increasingly linguistically diverse scientific landscape. Ten years later, the authors published a summary describing its use over a decade of research (Kaushanskaya, Blumenfeld, & Marian, 2019). The LEAP-Q has now been cited over 1,200 times, used in hundreds of studies worldwide, adapted for use with children, and translated into 26 languages, including Arabic, French, Mandarin, Russian, and Turkish (see Figure 2). This self-report survey, which is available as a paper and pencil assessment, as well as electronically, includes measures of language proficiency, age of acquisition (AoA), quantity and type of language exposure, cultural identification, and more.

Figure 2.

The Language Experience and Proficiency Questionnaire (LEAP-Q; Marian, Blumenfeld, & Kaushanskaya, 2007) is available for use with children and adults, can be administered as an online or paper/pencil survey, and has been translated into 26 languages. The LEAP-Q is freely available for download at http://bilingualism.northwestern.edu/leapq/.

Other labs around the world have developed similar self-report questionnaires to quantify bilingualism. One commonly used tool is the Language History Questionnaire (Li, Sepanski, & Zhao, 2006), which is well-suited for online research due to its user-friendly web interface. Another option is the Language Background Questionnaire (Sabourin et al., 2016), which includes a short and long version, allowing researchers to flexibly utilize measures of varying depth and detail depending on the population and topic of interest. The recently updated Language and Social Background Questionnaire (Anderson, Mak, Chahi, & Bialystok, 2018; Luk & Bialystok, 2013) includes detailed questions regarding language use in specific contexts, such as browsing the internet or healthcare services, and with specific individuals, such as with neighbors or grandparents. In addition to assessing how often different languages are used in different contexts, efforts have been made to quantify the extent to which individuals switch between different languages within and across contexts. For instance, Gullifer and Titone (2018) devised a method for calculating a continuous measure of “social diversity of language use” (available in the languageEntropy R package) based on what they describe as “language entropy” - that is, how predictably a particular language will be used within a given context. Other self-report questionnaires have been developed to obtain more fine-grained measures of language switching behavior. For instance, the Bilingual Switching Questionnaire (Rodriguez-Fornells et al., 2012) can be used to quantify how often individuals switch into L1 versus L2, how often switches are triggered by particular situations or topics, as well as how often switches occur unintentionally. There are also validated tools specifically designed for use with children that rely on parental reports, such as the Language Exposure Assessment Tool (DeAnda et al., 2016) and the Alberta Language and Development Questionnaire (Paradis, Emmerzael, & Duncan, 2010), as well as surveys which include both parental and teacher assessments of children’s language background and proficiency (e.g., Gutiérrez-Clellen & Kreiter, 2003).

One advantage of self-report measures is the ability to efficiently tap into a number of complex constructs spanning a range of dimensions relevant to bilingual status. However, there are several limitations, including response biases and differences in how respondents interpret questions and scales (Tomoschuk, Ferreira, & Gollan, 2018). While self-reported measures of proficiency often show robust correlations with objective measures (Ross, 1998; Marian et al., 2007), the validity of self-reports can vary considerably across skills (e.g., reading vs writing; Ross, 1998), measures (e.g., specific tasks vs. general abilities; Brantmeier et al., 2012), proficiency levels (Delgado et al., 1999), ages (Bedore et al., 2012), and raters (e.g., parents vs. teachers; Bedore et al., 2011). Therefore, the prevailing recommendation within the field has been to utilize a combination of self-report questionnaires and standardized tests in order to gain a more accurate and comprehensive picture of bilingual ability and experience. Some of the most frequently utilized standardized tests include picture naming and expressive vocabulary tasks, such as the Expressive Vocabulary Test (Williams, 1997), Multilingual Naming Test (Gollan et al., 2012) and Boston Naming Test (Kaplan, Goodglass, & Weintraub, 1978), as well as tests of receptive vocabulary, such as the LexTALE (Lemhöfer, & Broersma, 2012) and Peabody Picture Vocabulary Test (Dunn & Dunn, 1997). In recent years, researchers have also taken advantage of newer technologies to obtain real-time measures of language behavior, for example by sampling speech throughout the day using tools like Electronically Activated Recorders (Mehl et al., 2001), Language Environment Analysis (Gilkerson & Richards, 2008), and Ecological Momentary Assessment (Jylkkä, Soveri, Laine, & Lehtonen, 2019), which can record language-use and prompt participants to report language behavior at specified times via smartphone applications. Despite the increasing availability of useful tools for describing and quantifying bilingualism, a common question that often arises is how researchers can obtain a single score to capture an individual’s level of bilingualism.

1c. The Quest for a Quotient

The quest for a “bilingualism quotient” is reminiscent of scientific endeavors to find a quantifiable measure of intelligence. As with bilingualism, which has been, and continues to be operationalized using highly heterogeneous methods, the study of intelligence has been approached from numerous perspectives, ranging from the early and now-defunct use of craniometry to measure skulls to the standardized cognitive measures seen today (see Figure 3). A critical development in the quantification of intelligence was the introduction of the Binet-Simon intelligence test (Binet-Simon, 1905), which assessed abilities such as logical reasoning and picture naming. These measures were then used to derive a single value representing an individual’s “mental age” relative to what would be expected based on one’s chronological age, and in contemporary versions, relative to one’s age cohort. The later adapted Stanford-Binet test (1916) remains a popular tool and has provided a model for how complex traits and abilities could plausibly be distilled down to a single quotient.

Figure 3.

Example item from a cognitive ability test typically used to measure an individual’s IQ (based on Raven’s Progressive Matrices; Raven, 1936; Wikimedia Commons / CC-BY-SA-3.0).

And yet, Binet himself has stressed the limitations of such measures and the “difficulty in expressing all the oscillations of thought in a simple, brutal number” (Binet, 1900, p. 119–120).

While IQ tests have undoubtedly served important functions, researchers, educators, and clinicians have increasingly begun to question whether a single number is sufficient to capture the many facets and nuances of intelligence (Sternberg, Grigorenko, & Bundy, 2001; Stuebing et al., 2002). This has given rise to theories centered on the notion of multiple intelligences, such as Sternberg’s (1985) triarchic theory of intelligence, which posits the existence of multiple subcomponents that are broadly grouped under analytical, creative, and practical intelligence, as well as Gardner’s (1983) theory of multiple intelligences, which further specifies different domains of intelligence including Musical, Bodily-Kinesthetic, Inter- and Intrapersonal, Linguistic, Logical-Mathematical, and Spatial intelligence. Intelligence is therefore a complex construct that has warranted extensive research and inspired countless articles and books - some of which, like The Bell Curve (Herrnstein & Murray, 1996), introduce problematic ideas such as the existence of “cognitive elites” and group differences in intelligence. Reducing a multifaceted construct down to a single number can therefore be not only challenging, but even dangerous, as biased assessments can be used and interpreted in nefarious ways. In short, though measures that output a single index of intelligence may have practical value, they are likely to, at best, mask the nuances of human intellectual ability, and at worst, result in unwarranted characterizations of individuals whose intelligence deviates from the models that are imposed upon them.

The search for a Bilingualism Quotient presents a similar dilemma, as bilingualism emerges from a complex and interactive set of dimensions that can take many different forms. Attempts to quantify an individual as more or less bilingual using a single quotient are therefore meaningless without specifying a particular dimension of interest. Unfortunately, even if we restrict our analyses to a specific variable (such as AoA or proficiency), it is not possible to fully disregard other factors, as the variable of interest is typically related to others, making it difficult to draw causal inferences and generalize across different types of bilinguals. While it may not be reasonable to expect a simple solution to a complicated problem, we propose that enhancing communication and consensus is a critical step towards addressing a number of challenges relevant to quantifying and understanding bilingualism.

We begin by considering the importance of communication across researchers using different methods to study the same question, illustrating it through a discussion of the effects of bilingualism on executive function. Next, we highlight the need for cross-talk when asking different questions within the same domain, with foreign versus heritage language learning as the test case. Finally, we discuss the value of greater conceptual integration across different domains of scientific inquiry, using linguistic experience versus musical experience as an example (see Figure 4).

Figure 4.

An illustration of the communication hierarchy across methodologies within questions (e.g., effects of bilingualism on executive function), questions within domains (e.g., foreign versus heritage language learning), and domains within scientific inquiry (e.g., consequences of linguistic experience versus music experience).

2. Enhancing Communication and Consensus

2a. Across Methods

A prominent issue within bilingualism research is the heterogeneity of methods and measures used to characterize bilinguals and monolinguals, even among those investigating similar questions. While few researchers are likely to disagree that bilingualism is a continuum rather than a category, it is still common practice to broadly group individuals as being either bilingual or monolingual. In addition to losing potentially valuable information by masking variability within each group, this practice forces researchers to impose an artificial boundary based on criteria that are idiosyncratic across studies (see Bedore et al., 2012; Sabourin et al., 2016 for discussions). Combined with the fact that populations differ dramatically in language history and experience, this heterogeneity in how bilingualism is operationalized can make it difficult to interpret variable findings across different studies. Though such variability can be a significant challenge, it is possible to take advantage of these differences to better understand the consequences of multilingual experience so long as relevant information about participants is measured in ways that allow for comparisons across studies. In order to foster cross-laboratory communication and consensus, it is necessary to provide detailed descriptions of the populations tested following a consistent approach.

One notable example of problems that arise from lack of standardization can be observed in the so-called “bilingual advantage controversy” in executive function. While many studies have provided evidence suggesting that multilingual experience can confer advantages for domain-general cognitive functions (e.g., Abutalebi et al., 2011; Blumenfeld & Marian, 2014; see Bialystok, 2009 for a review), these effects are not consistently replicated, leading some to suggest that no such advantages exist (e.g., Dick et al., 2019; Paap & Greenberg, 2013). As noted by many in the field, however, it is difficult to interpret these inconsistent results due to great variability in how both executive function (Hilchey & Klein, 2011; Valian, 2014) and bilingualism (Luk & Bialystok, 2013; Marian, 2018) are defined and measured across studies. Factors extraneous to language experience, such as SES and education, can mask or masquerade as effects of bilingualism when comparing groups that are likely to differ in ways other than bilingual status. Even when these factors are well-controlled, relying on a single dimension of bilingualism such as AoA or proficiency to explain variance in executive function can lead to misleading conclusions, as these variables are likely to be correlated with other potentially unmeasured components of language experience (Kaushanskaya & Prior, 2015; see also DeLuca et al., 2019a,b for examples of how different forms of bilingual experience affect neural structures).

Though it may not be possible to account for all relevant variables, it is possible to narrow down the relevance of particular features of bilingualism through the use of comprehensive language assessments and analyses that are more sensitive to interactions among variables. We can use the LEAP-Q to illustrate this. As noted, the LEAP-Q assesses a broad range of variables relevant to language learning and use, including age of acquisition, immigration experience, language exposure, and proficiency in each known language. In order to validate the measure, self-reported proficiency was compared to scores on objective, standardized tests, revealing a high degree of correspondence. We then utilized factor analysis to identify three orthogonal factors, which were identified as L1 competence (e.g., proficiency and preference for L1), late L2 learning (e.g., age of L2 acquisition and years in an L2 country), and L2 competence (e.g., proficiency and preference for L2). These types of analyses are useful for determining whether measures capture variables of interest, as well as to reveal the underlying associations between variables. It is critical, however, to keep in mind that the relationships between language variables will vary across populations and contexts – factors extracted from a particular population should not be assumed to reflect a universal characterization of bilingualism. Rather, researchers should collect detailed information on their populations and determine the relationship among variables for particular cases, including the relationship between language experience and executive function (see Iverson & Miller, 2017 and Miller & Rothman, 2019 for similar approaches to studying language acquisition, attrition, and maintenance). Other analyses, such as mixed effects modeling (e.g., De Cat, Gusnanto, & Serratrice, 2018; Linck & Cunnings, 2015), can be used to estimate the contributions of specific aspects of bilingual experience while accounting for variance explained by participant and item-related factors. There are also methods such as propensity-matching, whereby pairs of participants are matched across multiple attributes (e.g., SES, musical ability) to reduce the impact of confounding variables (e.g., Hartanto & Yang, 2019). In this respect, longitudinal investigations can be particularly informative, as the effects of bilingualism can be observed over time while controlling for individual differences (Hope, 2015).

These types of approaches can help us make strides towards identifying the aspects of bilingual experience that affect domain-general cognitive abilities, but a number of limitations remain as a result of heterogeneous participant populations and research methods. As noted, relationships among linguistic and non-linguistic variables may not generalize from one population to another (Adesope et al., 2010). Even the relationship between self-reported and objective measures can vary across populations (Edele et al., 2015). The considerable variance in how different factors relate to each other illustrates that there is no one-size-fits-all method or tool that can be universally applied to all bilinguals. Until there is greater consensus on how individual differences should be assessed, it will be challenging to determine whether inconsistent results across studies emerge from differences in populations or measures. It is therefore necessary to develop methods to equate indices derived from different assessments and actively strive to build agreement around basic, but important, procedural and conceptual tenets. In a recent commentary, Leivada et al. (2020) identify procedural characteristics that could help facilitate comparisons across studies investigating the consequences of bilingualism. These include obtaining measures of individual differences and language characteristics, as well as the use of registered reports and appropriate control groups. The authors propose that multi-lab teams could systematically examine the impact of particular factors by intentionally holding key variables, such as the target language group, constant across research teams.

Though the precise constellation of relevant variables will continue to vary across participants and researchers, establishing consensus around terminology and best practices, including the types of information that should be reported in publications, will facilitate cross-laboratory comparisons and collaborations. It is important to note, however, that we are not advocating for a single static set of variables to be examined and reported. Such a rigid approach could contribute to reductionist characterizations of bilingual experience and impede scientific discovery (for example, of new relevant variables). Rather, a potential solution could be to cultivate norms around a minimum set of attributes that should be reported to facilitate comparisons across studies, akin to variables such as age and gender, which are commonly reported regardless of the research question. By enhancing communication and consensus across laboratories investigating similar phenomena, we can move away from piecemeal investigations to gain a broader, more complete understanding of the consequences of multilingual experience.

2b. Across Questions

In addition to establishing consensus when studying the same phenomenon, our understanding of bilingualism can benefit from enhancing cross-talk between researchers asking different questions. As it is, even seemingly similar spheres of investigation often suffer from lack of integration across their respective literatures (Kondo-Brown, 2005). There is often limited consideration of work outside the narrow scope of a research agenda, even in cases when the same types of tasks are utilized. As a result, we potentially miss out on opportunities to learn from relevant, but distinct programs of research that may have already generated a wealth of information that could help us develop more sensitive, reliable, and comprehensive measures of individual differences. Researchers within areas of study come to acquire particular biases and ways of thinking about a question. We can capitalize on cross-area variability in approaches and perspectives to reveal blind spots within our own areas, as well as identify common mechanisms and potential moderators.

Take for instance research examining foreign versus heritage language learning. In both areas, researchers often seek to identify individual differences in language background that predict language proficiency and maintenance. However, in addition to limited standardization within each sub-field (Norris & Ortega, 2000; Porte, 2012), the two areas often differ in the types of variables that are examined. While foreign language studies commonly look at factors such as AoA, language exposure, and domain-general cognitive abilities (e.g., Birdsong, 2006; DeKeyser, 2005), research with heritage learners is far more likely to consider the impact of sociocultural identity, as well as personal and community attitudes towards particular languages (Oh & Au, 2005; Potowski, 2004). Given that these variables can impact language learning, it would be valuable to consider their potential influence in both sub-fields. For instance, language identity has been associated with motivation and investment in a language for both heritage and foreign language learners, which can in turn promote language acquisition and maintenance (Norton, 2000; Potowski, 2004). Similarly, factors such as the quality and quantity of formal training can have a significant impact on language abilities, not only among foreign language learners, but for heritage speakers as well (Kupisch & Rothman, 2018; Putnam, Kupisch, y Cabo, 2018).

It is equally appropriate to determine the validity of adopting measures designed for different types of populations. It is not uncommon for heritage language researchers to utilize proficiency tests designed for foreign language learners, which could potentially result in misleading characterizations of language ability (Montrul & Perpinan, 2011; Potowski, Parada, & Morgan-Short, 2012). For instance, Montrul and Perpinan (2011) suggest that written production tests designed for foreign language learners may underestimate grammatical ability among heritage learners because the latter often relies more on implicit knowledge, which may not be captured by this type of task. The issue of utilizing possibly inappropriate diagnostic tools is reminiscent of problems that emerge when language assessments designed for monolingual populations are adopted for bilingual learners, leading to potentially inaccurate characterizations of language differences as language disorders (Gutiérrez-Clellen & Peña, 2001). Even seemingly straight-forward variables can be difficult to operationalize when dealing with heterogeneous populations (de Bruin, 2019). For instance, as noted by Unsworth and Blom (2010), age of first exposure may be relatively simple to determine when there is a clear-cut age of immigration, but there are often complicating factors, such as if an individual’s immersion in a given language community is interrupted. It may therefore be necessary to acquire different or additional information depending on the population. Determining the appropriateness of particular tests for different populations will not only allow researchers to develop better assessments, but also facilitate comparisons of different language groups. This could have important practical implications, such as for language education programs (e.g., Kondo-Brown, 2003), as well as theoretical value, such as for investigating critical periods (e.g., Benmamoun, Montrul, & Polinsky, 2013; Judy, Putnam, & Rothman, 2018).

While some progress has been made to identify commonalities and differences across foreign and heritage language learners, more research is needed to systematically evaluate how reliably assessments designed for particular populations can be generalized to other groups (Hulstijn, 2012), as well as to develop guidelines for how to adapt and validate questionnaires for new populations (Beaton et al., 2000). There is also a need for greater cross-talk between researchers, educators, and clinicians so that professionals in applied domains can benefit from empirical research, and research questions and methods can be informed by bilinguals in the real world. In sum, greater integration across areas of bilingualism research can not only expand our current understanding of bilingualism, but also facilitate the development of more comprehensive and valid tools as we all move forward.

2c. Across Domains

So far, we have considered how greater communication across laboratories and areas of bilingualism research could improve measures of language background and clarify the consequences of bilingual experience. Isolating the impact of language experience, however, requires consideration of how individuals differ in ways other than language background. Different forms of experience - such as music, physical exercise, and video-game playing, for example - can have an impact on both linguistic and non-linguistic abilities, which, if not accounted for, can be confounded with the effects of bilingualism. Effects of language and other forms of experience can also emerge from shared mechanisms, which could facilitate the development of theoretical frameworks and justifications for examining specific variables of interest.

Take for instance, the often overlapping consequences of language and musical experience for executive function and working memory (Janus, Lee, Moreno, & Bialystok, 2016; Parbery-Clark et al., 2009), as well as auditory processing and perception (Krizman et al., 2014; Ressel et al., 2012). Language and musical ability are not necessarily independent. There is evidence that processing music engages neural regions commonly associated with the language-network (Maess et al., 2001), and ERP components reveal similar processing of musical and linguistic syntactic anomalies (Patel, 2003). Simultaneous presentation of incongruent musical and linguistic pitch contours can modulate the N2b component among speakers of tonal languages (Sadakata, Weidema, & Honing, 2020), suggesting that listening to music has the potential to actively interfere with concurrent language processing. Longer-term musical training has also been found to influence language learning (Zhao & Kuhl, 2016), often resulting in beneficial effects, such as for speech perception. Language experience can influence musical ability, for instance by increasing sensitivity to pitch contrasts (Liu & Kager, 2017).

Compared to other forms of experience, however, the relationship between language and musical abilities has become a relatively fertile area of research, and on several occasions, researchers have directly compared individuals with bilingual and/or musical experience. To illustrate, Schroeder et al. (2016) investigated the independent and additive effects of bilingual and musical experience on executive control using the non-linguistic, non-musical Simon task. Critically, they compared four groups of individuals - bilinguals, musicians, bilingual musicians, and non-bilingual non-musician controls (see Table 1). The authors observed that both bilingualism and musicianship improved cognitive control, but that having both types of experience did not confer any additional advantages. These types of comparisons across domains can help inform us about the potential mechanisms underlying the respective effects and help dissociate their relative contributions. There have also been longitudinal studies employing language and music training paradigms (e.g., Bak et al., 2016; Janus et al., 2016), which are valuable for determining causal relationships.

Table 1.

Mean interference effects (incongruent - neutral RT) and participant demographics across groups in a study examining the effects of bilingual experience and music experience on cognitive control.

	Bilinguals (N = 43)	Musicians (N = 42)	Bilingual Musicians (N = 69)	Control (N = 65)
Interference (ms; Simon)*	14.47 (2.58)	17.16 (2.66)	16.71 (2.34)	28.68 (2.58)
Age (years)*	22.30 (4.05)	22.21 (3.42)	20.60 (2.86)	22.88 (4.03)
IQ (WASI)*	109.16 (8.59)	114.74 (9.92)	113.55 (9.56)	111.67 (11.39)
Digit Span (CTOPP)*	16.12 (3.00)	17.69 (1.54)	17.47 (2.76)	17.55 (2.46)

Note. Values in parentheses represent standard errors of mean differences for Interference and standard deviations for Age, IQ, and Digit Span. Asterisks indicate significant group differences at p < .05. Adapted from Schroeder et al., 2016.

In many cases, however, it remains difficult to draw causal inferences as it is not always feasible to directly manipulate musical and language experience (see Schellenberg, 2019). It can also be difficult to equate levels of language versus musical experience, which can diminish the ability to interpret cases in which effects are found for one and not the other. There has additionally been relatively little consideration of other sources of individual variability, such as physical exercise or video-game playing (Valian, 2015), both of which can have similar consequences to those of music and language (Bavelier, 2012; Colcombe & Kramer, 2003). All this, to say that bilingualism researchers would benefit from greater consideration of measures assessing other sources of individual differences, both to control confounding variables and to provide insight into underlying mechanisms - systematically examining how and when different forms of experience have common or distinct consequences for cognitive and neural functioning. These insights can, in turn, enable the development of more precise theoretical frameworks of bilingualism and cognition.

3. A Bilingualism Quotient?

Greater communication and consensus, both within and across disciplines, can help us converge upon the processes underlying the impact of bilingualism and the power of experience more generally. Identifying how different components of bilingualism relate to each other, as well as how they systematically vary across groups and contexts, could also help us develop tools that are suitable for a wider range of purposes - potentially even culminating in the elusive BQ.

3a. What Do IQ and BQ Measure?

We began by drawing the parallel to the field of intelligence research, and in particular, the development of IQ as a seemingly tenable model for how one could distill complex constructs down to a single number. The concept of IQ is predicated on the idea that there is a general mental capacity - often identified as the G factor - that influences other, more specific cognitive abilities (Spearman, 1904). IQ is thus derived by extracting the shared variance among a battery of tests (e.g., verbal fluency, mental rotation), with the logic that the latent factor common to all of them indexes general intelligence. Though bilingualism is similarly associated with multiple, often correlated attributes, many of them characterize language history or context, rather than language ability per se (see Figure 6 for an example). This is in contrast to the subcomponents of intelligence, which uniformly pertain to cognitive functions that could reasonably be expected to emerge from a common underlying capacity, such as G. In other words, while the G factor is thought to be the cause of associations among different components of intelligence, a B factor is more likely to be an emergent property of existing relationships (e.g., between age of acquisition and proficiency). This distinction is meaningful, as an effect of BQ could tell us that some aspect of bilingualism is relevant to a variable of interest, but not necessarily what that aspect is.

Figure 6.

Schematic of relations among sample variables relevant to intelligence (left) and bilingualism (right).

Both intelligence and bilingualism have also been notoriously difficult to operationalize due to their multidimensional nature and associated outcomes. Historically, intelligence has been considered an ability, and yet IQ is highly correlated with standardized tests of achievement (Frey & Detterman, 2004) and academic success (Deary et al., 2007). Bilingualism is often considered an achievement, but impacts a host of abilities, ranging from speech discrimination (Garcia et al., 2018) to language learning (Kaushanskaya & Marian, 2009). Both constructs also share reciprocal relationships to environmental variables such as education and SES (Neisser et al., 1996; Peng et al., 2019; Rindermann et al., 2010; Wyche et al., 2019) - further blurring the lines between assessing ability and achievement and raising fundamental questions about what, exactly, intelligence and bilingualism measure.

3b. How would BQ be Computed?

Beyond determining what a BQ is meant to capture, a notable challenge will be to decide what it means to be more or less bilingual. Variability in some, if not all, relevant attributes (e.g., manner of acquisition and use) are more naturally conceived of as qualitative differences that do not necessarily index “degree” of bilingualism. Though it is possible to operationalize most variables along a continuum (e.g, more or less immersion), calculating a single quotient almost necessarily requires a value judgment. One approach could be to make such determinations on a semantic basis, such as by classifying greater frequency, duration, and depth of bilingual experience as “more” bilingual. We could also empirically identify the characteristics that explain variability in other domains, such as neural function. These classifications could then serve as the basis for a BQ, which could in turn be used to explore other consequences and antecedents of bilingualism. The danger of this method, however, would be in the potentially tautological use of a BQ to investigate its relationship to the very functions or variables used to construct it.

A potentially simpler option may be to restrict BQ to cognitive abilities, which are easier to conceive along a continuum. Using large datasets, we may be able to identify a general index of language ability that supports specific functions (e.g., oral production, literacy). Such a metric could additionally inform the results of earlier factor analyses (Marian, 2007) indicating that different facets of language proficiency (e.g., speaking, writing, reading) are associated with different aspects of language history (e.g., AoA, L2 exposure, L1 duration, respectively; see Figure 7). One of the main challenges will be to determine whether there are, in fact, metrics that have strong predictive validity across domains and populations - and if not, whether there are systematic moderators that can be used to “customize” the formula.

Figure 7.

Results of a factor analysis on measures of the LEAP-Q. Reproduced from Marian et al. (2007; Table 2).

Developing a single, generalizable index of bilingualism could make a significant contribution to the field by facilitating direct comparisons across individuals, as well as across laboratories. Such a tool does, however, have the limitation of obscuring the many cognitive, affective, social, and environmental dimensions of bilingualism that do not directly characterize language ability. Assuming it is possible to use a more diverse set of variables as metrics of bilingualism, the BQ would ideally maximize flexibility over the choice of particular measures, while still maintaining an overall level of consistency.

For instance, we could start by identifying four to six general categories corresponding to key components of bilingual experience. These categories would be consistent and agreed upon by researchers in the field and may include, for example, language ability, history of use, acquisition, and possibly aspects like identity or cultural affiliation or others. Each of these categories would include multiple measures within it. The language ability category, for example, could include abilities to comprehend spoken language, to speak, to read, to write, and perhaps other abilities as well, as agreed upon in the field. Individual researchers could then have the freedom to select specific sub-measures from each category that are most relevant for their research question and population under study. For example, from the acquisition category, for some research questions age of acquisition may be the most relevant measure, whereas for other research questions the manner of acquisition (e.g., classroom or heritage) may be more relevant. Similarly, from the history of use category, for some research questions current use may be most relevant, whereas for other research questions cumulative historical use may be more relevant. The exact number of measures (one, two, more) from each category that is to be used towards computing the BQ would be determined during the course of BQ development. Responses could then be weighted and aggregated to obtain a single quotient. The BQ will thus be a composite score across the agreed-upon general categories, where the score for each general category is obtained based on the sub-dimensions from that category that are most relevant to the specific researcher. This will provide the advantage of flexibility for individual researchers to choose the variables most relevant for their topic from each category while at the same time maintaining uniformity by indexing all agreed-upon general categories within the BQ construct.

The advantage of including flexible options would be that the same categories could be used for a wide range of purposes, and hopefully, result in widespread adoption of a common construct. Constraining the represented categories, on the other hand, would help facilitate cross-laboratory comparisons.

3c. Variability Across Time and Groups

In the final part of this section, we consider how we might increase the validity of a BQ by addressing two complicating factors - variability across time and variability across groups.

Intelligence research shows that IQ is relatively stable over one’s lifetime, and the genetic versus environmental contributions to IQ have been studied extensively. In fact, it is this stability over one’s lifetime that made IQ scores so attractive historically - to the military, to potential employers, and to academic institutions. Such stability cannot be expected, and in fact is directly inconsistent with the dynamic nature of bilingualism. Even cognitive abilities such as language proficiency are expected to fluctuate over time as a bilingual learns or forgets a language (for example due to immigration, international adoption, or other changes in language use). Though it may not be possible to predict future circumstances, test items could be designed to assess levels of language proficiency, exposure, use, etc. at different periods in the past (see Figure 10 for an example). Responses could then be aggregated to calculate a single index, used to classify individuals based on their developmental trajectory, or analyzed individually to determine the impact of particular variables at specific time points.

Figure 10.

Example of a developmental self-report item assessing relative language exposure over time for a 30 year-old English-Spanish bilingual.

Extending IQ’s cohort-based approach, a BQ could characterize an individual’s level of bilingualism relative to others who are matched on key demographic variables such as socioeconomic status and age. Unlike IQ, however, which is generally calculated based on a static set of criteria, a BQ would ideally reflect the dynamic nature of bilingualism through a more flexible system of measurement and scoring. For instance, factors like degree of language exposure and proficiency could be differentially weighted depending on demographic and language background variables. Using age as an example, assessments of young children would likely privilege factors such as language exposure over language proficiency, while measures of linguistic competence may be weighted more heavily for older children (see Byers-Heinlein et al., 2019). We could rely on the data to determine the weights for a given study. Factor loadings could be determined by the intercorrelations among variables for a particular population, so that the variables that share the most variance with all others would be weighted the most. This approach is similar to the one behind IQ, such that the BQ would represent the latent aspect of bilingualism that is common across factors. In addition to allowing for “customized” factor loadings, this approach should still enable comparisons across studies, as each participant’s score would represent a relative ranking within their group based on attributes that were most important for that population. It may even be possible to additionally include ways to calculate a score in relation to an established standard.

4. Conclusions: Towards a Unified Approach to Measuring Bilingualism

In summary, we have outlined some potential approaches, challenges, and points of discussion for formulating a tool that could be used to calculate a BQ (see Table 2). We caution, however, that a Bilingualism Quotient, while providing a useful heuristic, is subject to the same constraints and limitations as its cousin the Intelligence Quotient on which it was modeled. Just like a single IQ number can not capture the nuanced cognitive capacities of every mind, especially for neurodiverse populations, a single BQ number will only provide an approximate estimation of a bilingual’s linguistic profile and will have limited predictive ability for specific language acts and individuals. Most proximally, it will be necessary to generate agreement around the form and function of the quotient, possibly through flexible methods of administration and scoring while enabling cross-laboratory comparisons. Advocating for how (or whether) a BQ should be formulated, however, would be antithetical to our ultimate intention, which is to begin a conversation (rather than provide definitive answers) so that solutions can be found through scientific discourse and research in the field.

Table 2.

Summary of key challenges, points of discussion, and recommendations.

Challenges

• BQ would need to account for heterogenous variables related to language ability, history, and context.

• Language exposure, use, and ability change over time, and the relationships among variables vary across populations.

• Some language background variables capture qualitative differences that do not clearly index a relative “degree” of bilingualism.

Discussion Questions

• What would a BQ represent?

• Is it possible to operationalize all relevant factors along a continuum of bilingualism?

• What criteria and principles could be used to establish a bilingual continuum?

Recommendations

• Use of large, diverse datasets and advanced statistical methods to select and weight attributes.

• Implement flexible methods of administering, weighting, and scoring (e.g., rely on several agreed-upon categories with multiple specific attributes within each to select among).

• Variable standards of comparison based on demographic characteristics to determine relative bilingualism.

• Include mechanisms for assessing change over time.

• Establish norms and procedures for reporting and depositing data.

In recent years, there has been significant progress in how society and individuals view bilingualism - these changes in attitudes and perceptions are in no small part due to the findings that have been brought to light by researchers who study bilingualism. How we understand bilingualism has also continued to evolve - for instance, a theme that has increasingly emerged is the importance of considering bilingualism as a continuum rather than a category (see Luk & Bialystok, 2013). Operationalizing bilingualism as a category can mask important variability within each group and impede comparisons across studies, as the boundaries between “monolingual” and “bilingual” are often idiosyncratically and arbitrarily drawn. Even in cases when categorization is necessary, it will be important to rely on standards and guidelines that have been agreed-upon by researchers within the field. One silver lining that has emerged from the controversies surrounding bilingualism is that they have inspired researchers to reconsider the types of questions that we should be asking - not merely whether bilinguals differ from monolinguals, but rather, how, why, and when language experience alters aspects of cognitive and neural function (see Baum & Titone, 2014; Tabori, Mech, & Atagi, 2018).

While the inherent variability of language experience can pose significant challenges, it can also provide useful insights so long as we are able to organize the information. Enhancing communication and consensus is an important step towards this goal, but how do we do it? Just as we can look to other fields to shed light on potential obstacles, we may find useful solutions by examining how other spheres of scientific investigation have resolved similar problems. For instance, scientists in many fields, including chemistry, biology, health, and food science, often have professional organizations, such as the US Food and Drug Administration (USDA) and the Clinical and Laboratory Standards Institute (CLSI), which help establish guidelines and build consensus. The CLSI offers online workshops, webinars, and certificate programs for the purposes of enhancing the quality and standards of health-care related research. Their website also provides information regarding their consensus process, which begins with the identification of problems and projects by experts and community members, followed by the recruitment of panels and councils to develop plans, disseminate relevant information, and obtain comments and votes on preliminary drafts, and ultimately results in the publication of research standards and guidelines (clsi.org). Groups to develop standards have been established in areas ranging from consumer product development to microbiology research. So, might this be possible for bilingualism?

In fact, there are already initiatives in place, such as Bilingualism Matters (bilingualism-matters.ppls.ed.ac.uk), that have the express purpose of bridging bilingualism researchers with each other and members of the public. This rapidly expanding community of researchers has established branches all over the world. At its inception, Bilingualism Matters was primarily established to help connect bilingualism researchers with community members to enable parents and educators to make better, more informed evidence-based decisions about bilingualism and language learning. As it continues to expand and evolve, Bilingualism Matters and other organizations like it may be able to connect bilingualism researchers with each other and provide a centralized structure through which we can begin to establish initiatives for enhancing communication and consensus.

An increasing number of such collaborative endeavors have already begun to emerge. The Quantifying Bilingual Experience (Q-BEx) project is currently underway to develop measures of cumulative and current language experience, with the objective of enabling cross-laboratory comparisons (see https://q-bex.org/). In collaboration with an international team of researchers, speech-language pathologists, and educators, project lead Cécile De Cat aims to create a user-friendly online tool, which will be available in 13 different languages and with varying levels of detail and length. Along similar lines, the Psycholinguistics of Language Representation (PoLaR) lab has initiated projects to identify variables that predict language performance among heritage speakers, which could help inform the development of novel measures (see https://site.uit.no/polar/funded-projects/).

Another critical piece of the puzzle is the establishment of websites and applications to consolidate and disseminate information, including surveys, assessment tools, data, and analysis scripts. A number of useful resources are already available, such as the Open Science Framework (osf.io) and GitHub (github.com) for uploading data and materials, PsychFileDrawer (psychfiledrawer.org) for reporting published and unpublished findings, and university- and laboratory-run websites, such as by the National Heritage Language Resource Center (nhlrc.ucla.edu), for consolidating assessment tools and making them easily accessible. In addition to continuing to encourage the use and development of these types of platforms, a key objective moving forward should be to establish best-practices for what and where information should be reported and deposited, with the ultimate goal of having a centralized and user-friendly repository of tools, procedures, and data. Concerted efforts to build on existing structures for the purposes of improving, standardizing, and consolidating measures could help us move towards enhancing consistency in how we talk about, study, and measure bilingualism.

In other words, in order to gain a unified and comprehensive understanding of multilingualism, we need to start speaking the same language.