Abstract
The importance of input factors for bilingual children’s vocabulary development was investigated. Forty-seven Argentine, 42 South Korean, 51 European American, 29 Latino immigrant, 26 Japanese immigrant, and 35 Korean immigrant mothers completed checklists of their 20-month-old children’s productive vocabularies. Bilingual children’s vocabulary sizes in each language separately were consistently smaller than their monolingual peers but only Latino bilingual children had smaller total vocabularies than monolingual children. Bilingual children’s vocabulary sizes were similar to each other. Maternal acculturation predicted the amount of input in each language, which then predicted children’s vocabulary size in each language. Maternal acculturation also predicted children’s English-language vocabulary size directly.
Researchers agree that typically developing human infants are born with an amazing biological capacity to learn language. What is in dispute, however, is how experience and biology influence rates of language acquisition (Tomasello, 2006). Some theories of language acquisition suggest that minimal input is necessary for children to learn language (Lenneberg, 1967) or that input plays a minor role in language acquisition (Chomsky, 1980), particularly with respect to grammatical development. However, other theories give a central role to input in children’s language acquisition (MacWhinney, 2004) by asserting that variations in input are responsible for variability in children’s language acquisition (Gathercole & Hoff, 2007) and vocabulary development in particular (Hoff et al., 2012; Huttenlocher, Haight, Bryk, Seltzer, & Lyons, 1991). The task of vocabulary learning for children acquiring a first language involves learning the label (word) that members of a linguistic community have assigned to a concept; such learning relies on input from the environment (Bates, O’Connell, & Shore, 1987). This study examined the extent to which differences and variability in input influence children’s language acquisition by comparing several samples of bilingual children to their monolingual peers and by examining whether relative input predicts language acquisition.
Productive vocabulary is developmentally important: vocabulary in toddlerhood relates to other aspects of language acquisition (i.e., phonology, semantics, grammar, and pragmatics) and subsequent literacy skills (Bates & Goodman, 2001). Indeed, vocabulary size at 20 months is the best predictor of grammatical development (MLU) at 28 months among monolingual children learning English (Bates & Goodman, 2001) and Spanish (Thal, Jackson-Maldonado, & Acosta, 2000), and bilingual Spanish-English learners (aged 20- and 30-months, Conboy & Thal, 2006; aged 22- and 25-months, Parra, Hoff, & Core, 2011). Despite the developmental importance of toddlers’ productive vocabulary, and despite the fact that cultural norms vary regarding linguistic input to children (Bornstein et al., 1992), relatively few studies have examined young children’s vocabulary development from a cross-cultural perspective (Bornstein, Cote, et al., 2004), and fewer still have examined bilingual children’s vocabulary development from the same vantage.
Input and Group Differences in Vocabulary Development
Bilingual children offer a particularly relevant test case for the influence of input because, by definition, monolingual children receive 100% of their input in a single language, and bilingual children receive some fraction of their input in each of two languages (Bialystok, 2001). Thus, if the relative amount of input matters to vocabulary development as we hypothesize, then bilingual children’s vocabulary size in each of their languages should be smaller than their monolingual peers’. Insofar as the total amount of linguistic input that children experience does not differ between monolinguals and bilinguals, we expected no differences between monolingual and bilingual children’s total vocabulary size. Empirically, some research and review articles have reported that Spanish-English bilingual children’s vocabulary is slower to develop than for monolingual children, at least in the early years (Bialystok, 2001; Nicoladis, 2008), but other research has found that this is not the case when children’s total vocabulary size (both languages combined) is considered (Pearson, Fernandez, & Oller, 1993). For example, Hoff et al. (2012) compared high-SES Spanish-English bilingual children aged 22 – 30 months to high-SES monolingual English-speakers. They found that bilingual children had lower productive English-language vocabularies compared to their English-speaking monolingual peers but there were no differences in vocabulary size when total vocabulary size (both languages combined) was compared. The current study compares bilingual children to monolingual English speakers in the United States (the country of destination for their immigrant mothers) and consistent with Rescorla’s (2004) recommendations, in this study two groups of bilingual children (Spanish-English and Korean-English) were also compared to monolingual speakers of their mothers’ heritage language, a rarity in the research literature. Additionally, per Bialystok’s (2001) recommendation, we assessed bilingual toddlers’ vocabulary development by comparing bilingual children to each other, which is also novel.
Input and Individual Differences in Vocabulary Development
In addition to group differences based on input, if input matters to vocabulary acquisition there should also be individual differences in children’s vocabulary acquisition as a result of differential input. Indeed, the amount of input monolingual children receive in a given language is positively related to their vocabulary size in that language (e.g., Huttenlocher et al., 1991), and this same association has been found among children being reared bilingually from birth. For example, Pearson, Fernández, Lewedeg, and Oller (1997) found a positive relation between the percentage of words a child was reported to produce on Spanish and English MCDIs and estimates of language input in a sample of 25 simultaneous bilingual Spanish-English children. Patterson (2002) reported positive relations between children’s vocabulary size in English and their overall exposure to English and the amount of bookreading in English that they experienced, and she reported similar positive relations for Spanish. Marchman, Martinez-Sussman, and Dale (2004) reported that Spanish-English bilingual children’s English vocabulary size was positively correlated with parental estimates of the proportion of English input that the child received, and their Spanish-language vocabulary size was negatively correlated with estimates of English input. Place and Hoff (2011) and Hoff et al. (2012) found that Spanish-English bilingual children’s English-language input was positively related to their vocabulary size in English and negatively related to their vocabulary size in Spanish. The current study seeks to replicate these findings in a high=SES sample of Latino children from South America and extend them to Japanese-English and Korean-English bilingual children. By examining three groups of bilingual toddlers we were able to investigate the broader generality versus language-specific nature of individual differences in vocabulary development among bilingual toddlers, which has not been done before in a single study using a uniform methodology.
Determinants of Input
Because language is one of the primary ways that parents socialize their children into cultural norms for social interaction, culture itself can fundamentally shape children’s language input (Schieffelin & Ochs, 1986). As mentioned, previous studies have reported direct relations between linguistic input and children’s language-specific vocabulary size (for Spanish-English bilinguals); additionally, studies have reported relations between cultural factors, specifically acculturation, and bilingual children’s language development. For example, Boyce, Gillam, Innocenti, Cook, and Ortiz (2013) found that maternal factors (including maternal acculturation) and the home language/literacy environment each predicted Spanish-English bilingual children’s total vocabulary size at 24 months (see also Hammer, Miccio, & Rodriguez, 2004). However instead of exploring direct relations between mothers’ acculturation and children’s language learning as previous researchers have done, we hypothesized that mothers’ acculturation level would influence their childrearing decisions with respect to the language environment that they create for their children (input), which in turn would influence children’s language development. More specifically, we hypothesized that immigrant mothers who were more acculturated to American society would expose their children to more English, which would in turn lead their children to have larger English-language vocabularies.
We also studied heritage language maintenance in children because it is important for many immigrant families (Bialystok, 2001). Again the limited literature on heritage language maintenance has found links between parents’ cultural beliefs and children’s language acquisition, which we believe are mediated by input in the heritage language. Eilers, Pearson, and Cobo-Lewis (2006) reported that parents’ desire to pass on Spanish to their children and thereby maintain their cultural heritage influenced Spanish-English bilingual children’s heritage language maintenance. In some of the few studies of Japanese-English and Korean-English bilingual children, the important role of parents’ cultural beliefs in children’s Japanese- and Korean-language proficiency or loss has been noted (Kamada, 1997; Song, 2010). We hypothesized that mothers who were less psychologically acculturated to American society would expose their children to more of their heritage language, resulting in larger heritage-language vocabularies for children.
Method
Participants
Altogether 230 mothers of 20-month-old children from six cultural groups participated: Argentines in Buenos Aires, South Koreans in Seoul, and four samples from one large metropolitan area in the United States: European Americans, Latina immigrants (primarily from Argentina, Colombia, and Peru; empirically there were no differences within this group on any dependent variables), Japanese immigrants, and Korean immigrants. All children were firstborn, full term, healthy at birth and at the time of the study, and none had diagnosed hearing or developmental problems. The majority of children (96%) were living in two-parent households in which parents were married. Seventy-six percent of children lived in nuclear families. Families were recruited from hospital birth notifications, medical group patient lists, newspaper birth announcements, newspaper advertisements, and mass mailings. Participants in this study were part of a larger longitudinal study of parenting and children’s development.
Argentine, Korean, and European American children were monolingual and learning the dominant language of their community (they responded “no” to the question “Is the baby bilingual?”) and “yes” to “Is Spanish/Korean/English [respectively] the child’s first language?” on the demographic questionnaire). Monolingual samples were chosen to match the bilingual samples demographically (Table 1 contains sample sizes and demographic information). Immigrant mothers in this study met the following criteria: they self-identified as South American, Japanese American, or Korean American, were first generation (i.e., they were born in Spanish-speaking South American countries, Japan, or South Korea, respectively), and their children were simultaneously learning two languages from birth, meaning that children’s exposure to their less-frequently-heard language constituted at least 10% of their total linguistic input based on caregiver report (see Hoff et al., 2012, for precedent and validity). Thus, these samples constitute a particular segment of the bilingual population.
Table 1.
Demographic Characteristics of the Families
| Cultural group | Argentine (n = 47) | Latino immigrant (n = 29) | Japanese immigrant (n = 26) | Korean immigrant (n = 35) | South Korean (n = 42) | European American U.S. (n = 51) |
|---|---|---|---|---|---|---|
| Child | ||||||
| Age (mo)a | 20.60 (0.36) | 20.39 (0.67) | 20.06 (0.50) | 20.47 (0.52) | 20.43 (0.72) | 20.03 (0.14) |
| Gender (female:male)b | 20:27 | 15:14 | 13:13 | 15:20 | 20:22 | 20:31 |
| Mother | ||||||
| Age at report (yr)c | 28.32 (5.72) | 33.53 (4.98) | 33.44 (4.17) | 31.94 (3.37) | 30.01 (2.96) | 32.65 (2.43) |
| Educationd | 5.21 (1.94) | 5.96 (0.84) | 5.62 (0.80) | 6.31 (0.83) | 5.57 (1.02) | 6.33 (0.48) |
| Hours of work/weeke | 14.64 (17.96) | 17.13 (19.18) | 10.02 (14.15) | 20.88 (19.91) | 16.17 (24.58) | 23.99 (21.84) |
| Hours of childcare per weekf | 17.72 (18.83) | 15.24 (20.31) | 14.74 (18.46) | 27.48 (26.07) | -- | 30.53 (23.85) |
| SDSg | 15.58 (4.83) | 21.86 (4.46) | 17.65 (4.15) | 17.71 (6.66) | 15.74 (3.89) | 15.55 (4.56) |
| Exposure to Englishh | -- | .44 (.23) | .44 (.23) | .44 (.27) | -- | -- |
| Acculturationi | -- | 2.22 (0.26) | 2.05 (0.27) | 2.42 (0.63) | -- | -- |
| Age at immigrationj | -- | 21.43 (9.90) | 27.05 (3.96) | 16.54 (9.58) | -- | -- |
| Years in U.S.k | -- | 12.85 (7.77) | 6.77 (3.35) | 13.63 (10.64) | -- | -- |
Note: All values denote M (SD) except for gender. Some sample sizes are smaller due to missing data. ANOVAs were followed by Tamhane’s T2 tests in which immigrant families were compared to each other, to European Americans, and to their culture-of-origin.
F(5, 224) = 8.91, p < .001; Korean American children were older than Japanese American and European American children.
χ2 (5, N = 230) = 1.77, ns.
F(5, 224) = 10.48, p < .001; Latina immigrant mothers were older than Argentine mothers in South America.
Hollingshead (1975) 7-point scale where 1 = less than 7th grade, 2 = completed grades 7 – 9, 3 = completed 10th or 11th grade, 4 = high school graduate or GED, 5 = some college or completed specialized training, 6 = standard college or university graduate, 7 = completed graduate degree. European American mothers completed more years of schooling than Japanese immigrant mothers, Korean immigrant mothers completed more years of schooling than Japanese immigrant or South Korean mothers, F(5, 223) = 6.79, p < .001.
Includes homemaker mothers; F(5, 216) = 2.12, ns.
F(4, 175) = 4.14, p < .01; European American toddlers spent more time in childcare than Latino or Japanese American children; data were not collected for the S. Korean sample.
Social Desirability Scale (scores range from 0 to 33), F(5, 206) = 7.95, p < .001; Latina immigrant mothers scored higher on social desirability than Japanese immigrant, Argentine, and European American mothers.
F(2, 87) = 0.001, ns.
Korean immigrant mothers were more acculturated than Japanese immigrants (5 point scale), F (2, 79) = 4.60, p < .05.
Japanese immigrant mothers were older than Latina immigrant and Korean immigrant mothers when they moved to the U.S, F (2, 80) = 10.05, p < .001.
Latina immigrant and Korean immigrant mothers had lived in the U.S. longer than Japanese immigrant mothers, F (2, 80) = 5.02, p < .01.
Procedure
Participants were treated in accordance with ethical standards defined by the American Psychological Association and BLIND’s IRB. When their children were 5 months old, all mothers completed a demographic questionnaire and immigrant mothers completed an acculturation scale, and 15 months later all mothers completed an updated sociodemographic questionnaire, a social desirability measure, and a vocabulary checklist for their children -- either the Early Language Inventory (ELI) or the MacArthur Communicative Development Inventories (CDI). Immigrant mothers also completed a questionnaire about their child’s language environment, specifically they answered a series of questions that required the mothers to reflect on their children’s language environment; they listed the language(s) spoken to and in the presence of the child by significant adults in the child’s life (i.e., parents, all caregivers) and by the child him/herself and estimated the amount of time each language was spoken. After answering these questions mothers were asked, “Which language(s) does your child hear (experience)?” In response to this question mothers listed the language(s) that their child heard/experienced and the percentage of time for each (totaling 100%); this procedure yielded our measure of English and heritage-language input (Table 1). All bilingual participants listed only two languages (the target languages).
Following procedures outlined by van de Vijver and Leung (1997), we sought ‘adapted equivalence’ of measures (i.e., culturally appropriate and cross-linguistically valid). Specifically, the original English-language questionnaires were forward-translated into Spanish, Japanese, and Korean and then back-translated by bilingual, bicultural Argentine, Japanese, and South Korean natives, respectively, using standard translation techniques (Brislin, 1986; Peña, 2007). Experienced professionals in developmental science and native to one of the countries in this study then added or substituted culturally appropriate words to the ELI (BLIND). All instruments were then checked for preservation of meaning and cultural appropriateness by professional psychologists from each cultural context. Last, pilot testing was undertaken to ensure ethnographic validity of the measures.
Measures
Early Language Inventory (ELI; Bates et al., 1984)
This maternal report measure of children’s productive vocabulary presents mothers with 645 English words divided into 19 categories (based on adult parts of speech) and asks them to check off the words that their child said (produced) spontaneously. Although the ELI allows mothers to write in any additional words that their child says, additional words were not counted. The ELI is a representative index of children’s productive vocabulary, not an exhaustive list of their productive vocabulary. The ELI used in this study was a precursor to both the revised ELI (Fenson, Thal, & Bates, 1990) and the CDI (Fenson et al., 1993), and it correlates positively with observational and standardized measures of children’s language use (Bornstein & Haynes, 1998) and has demonstrated construct validity (Dale, Bates, Reznick, & Morisset, 1989). The English, Spanish (669 words), and Korean (571 words) ELIs used in this study have been used in published research (BLIND).
MacArthur-Bates Communicative Development Inventories (CDI; Fenson et al., 1993) are comparable to the ELI because they are maternal report measures of children’s productive vocabulary, and are representative indexes rather than exhaustive lists of the words children produce. For the English version, mothers were presented with 680 words divided into 22 categories (based on adult parts of speech) and were asked to check off the words that their child said (produced) spontaneously (additional words mothers wrote in were not counted). The Korean CDI contained 506 words. The English and Korean CDIs that we used are found on the MacArthur-Bates CDI website: http://www.sci.sdsu.edu/cdi/adaptations_ol.htm. CDIs were used in this study only for the Korean American sample and not the other samples because data collection for the other samples predated wide availability of CDIs in those languages. Maternal report data have been used in cross-linguistic comparisons of children’s productive language (e.g., Bornstein, Cote, et al., 2004) and are reliable for bilingual children (Vagh, Pan, & Mancilla-Martinez, 2009). No comparable vocabulary data were gathered in Japan thereby preventing a heritage-language comparison for the Japanese American sample.
Following Pearson (1998), for bilingual children vocabulary size in English was computed by summing all words the mother checked on the English checklist, and vocabulary size in the heritage language was computed by summing all words mothers checked on the heritage-language checklist (Spanish, Japanese, or Korean). Some vocabularies share phonological doublets, words that have the same pronunciation and meaning in two languages. Phonological doublets were counted once as “English” words and again as “Spanish, Japanese, or Korean” words.
From these checklists we also computed total vocabulary size, defined as the total number of words that a child produced in both languages combined. Just as Pearson et al. (1993) and Pearson (1998) did, for bilingual children we computed total vocabulary as all the words checked on the English vocabulary checklist plus all the words checked on the heritage-language vocabulary checklist (i.e., Spanish, Japanese, or Korean) minus phonological doublets (so that they were not counted twice). We counted vocabulary in this way rather than using total conceptual vocabulary (Conboy & Thal, 2006; Pearson, 1998) because we wanted to know the total number of different words (labels) each child has in his or her productive vocabulary, and we wanted to allow bilingual children the most liberal counting of vocabulary size.
We used raw scores in contrast to ‘word opportunity scores’ (i.e., the total number of words the child is reported to say divided by the proportion of all possible words on the checklist/s). Word opportunity scores are often used in research when the number of word opportunities differs across checklists (as ours does); however, we used raw scores because it allowed the most generous counting of vocabulary size for bilingual children (their mothers completed two checklists therefore they could potentially produce nearly twice as many words as monolingual children).
Marlowe-Crowne Social Desirability Scale (SDS; Crowne & Marlowe, 1960)
This is a 33-item instrument that assesses a person’s tendency to answer questions in a socially desirable way; we used it in this study to check for reporting bias on maternal reports of children’s vocabulary. The SDS is one of the most widely used instruments to assess social desirability, and it has been used extensively across cultures (e.g., Bornstein et al., 2015). The SDS has demonstrated internal consistency (α = .88), test–retest reliability (r = .89), and construct validity (r = .35 with a personality measure; Crowne & Marlowe, 1960).
South American Acculturation Scale (SAAS), Japanese American Acculturation Scale (JAAS), and Korean American Acculturation Scales (KAAS-1) (BLIND)
These 21-item measures cover topics such as language, identity, friendship, behavior, generation, and attitudes and are identical except for the ethnicity term. Items include: “How do you identify yourself?” and “Do you participate in South American/Japanese/Korean occasions, holidays, traditions, etc.?” Participants rated each item on the SAAS scale from 1 (South American) to 5 (U.S. American), ratings on the JAAS ranged from 1 (Japanese) to 5 (U.S. American), and items on the KAAS ranged from 1 (Korean) to 5 (U.S. American). Acculturation level was calculated by taking the mean of mothers’ ratings on all 21 items. These acculturation scales had high internal reliability (α ≥ .91) and good construct validity in the larger samples from which the current samples were drawn, i.e., significant negative relations to mothers’ age at immigration, Pearson’s 1-tailed correlations were r(30) = −.68, p < .001, for Latina immigrant mothers, r(23) = −.38, p = .03, for Japanese immigrant mothers, and r(61) = −.80, p < .001, for Korean immigrant mothers. Mothers’ acculturation (measured with an acculturation scale, age at immigration, and number of years living in the United States) were investigated as predictors of children’s language ability.
Results
Prior to analyses, dependent variables were screened to see whether they met the criteria for parametric statistical testing; if not, they were transformed. Vocabulary size in each language separately and total vocabulary size were cube-root transformed; transformed data were used in analyses; however, untransformed data not controlled for covariates (M, SD) are presented below and in Tables to enhance comprehensibility and comparison with published data.
All variables in Table 1 were screened as potential covariates in the ANOVAs; to be used as a covariate, the variable had to correlate significantly (p < .05) with the DV and have a significant effect in analysis. When covariates were used, they are indicated below. Because gender differences have sometimes been found in young children’s vocabulary sizes (e.g., Bornstein, Hahn, & Haynes, 2004; Fenson et al., 1994; Huttenlocher et al., 1991), gender was used as a factor in all ANOVAs. Gender differences were not the focus of this study and thus were noted only when they interacted with Linguistic Community. Pairwise comparisons using t-tests with Bonferroni’s correction were performed to decompose significant main effects and interactions in AN(C)OVAs.
Input and Group Differences in Vocabulary Development
English-language vocabulary size
An ANCOVA with two between-subjects factors (Linguistic Community, Gender) one DV (total English vocabulary size) and one covariate (mothers’ education level) indicated main effects of Linguistic Community, F(3, 128) = 17.52, p < .001, η2p = .29, where monolingual European American children’s English vocabularies were significantly larger than bilingual children’s English vocabularies for each bilingual group (Table 2); no significant differences among bilingual children were found.
Table 2.
Vocabulary Size by Cultural Group and Language
| Argentine (Spanish) | Bilingual Latino American | Bilingual Japanese American | Bilingual Korean American | South Korean (Korean) | European American (English) | |
|---|---|---|---|---|---|---|
| Total Productive Vocabulary Size1: | ||||||
| M (SD) | 156.77 (99.43) | 99.17 (103.41) | 68.73 (96.03) | 117.57 (130.07) | 80.79 (64.67) | 148.33 (114.74) |
| Minimum | 1 | 6 | 2 | 13 | 3 | 4 |
| Maximum | 402 | 363 | 429 | 590 | 253 | 487 |
| Bilingual Children’s Vocabulary Size in English | ||||||
| M (SD) | -- | 39.29 (46.45) | 41.19 (76.06) | 66.23 (80.71) | -- | -- |
| Bilingual Children’s Vocabulary Size in their Heritage Language2 | ||||||
| M (SD) | -- | 66.31 (90.36) | 40.38 (46.21) | 64.03 (91.97) | -- | -- |
Both languages combined for bilingual children.
Heritage language is Spanish, Japanese, or Korean.
Heritage-language vocabulary size
An ANCOVA with two between-subjects factors (Linguistic Community, Gender), one DV (total Spanish vocabulary size), and one covariate (mother’s age) indicated significant main effects of Linguistic Community, F(1, 70) = 15.61, p < .001, η2p = .18; monolingual children living in South America had significantly larger Spanish vocabularies than bilingual Latino children in the United States (Table 2).
An ANCOVA with two between-subjects factors (Linguistic Community, Gender), one DV (total Korean vocabulary size), and one covariate (child’s age), revealed a significant main effect of Linguistic Community, F(1, 71) = 5.05, p = .028, η2p = .07; monolingual children living in South Korea had significantly larger Korean-language vocabularies than bilingual Korean American children (Table 2).
An ANOVA with two between-subjects factors (Linguistic Community, Gender) and one DV (total vocabulary size in heritage language) found no differences among bilingual groups in heritage-language vocabulary size, F(2, 84) = 0.99, ns, η2p = .02 (Table 2).
Thus, when bilingual children’s languages are considered separately, their English and heritage-language vocabularies are smaller than their monolingual peers’ and not different from each other.
Total vocabulary size
ANOVAs with two between-subjects factors (Linguistic Community, Gender) were performed for each cultural comparison to investigate whether bilingual children’s total vocabularies differed from their monolingual peers; covariates were controlled as necessary and are mentioned when used. Bilingual Latino children’s total vocabularies (both languages combined) were significantly smaller than those of either monolingual Spanish-speaking Argentine or English-speaking European American children: Linguistic Community main effect, F(2, 121) = 4.74, p = .01, η2p = .07 (Table 2).
A comparison of bilingual Japanese American children with English-speaking European American children in the United States (controlling mothers’ education level and social desirability scale score) showed a main effect of Linguistic Community, F(1, 67) = 9.34, p = .003, η2p = .12, and also a Linguistic Community x Gender interaction, F(1, 67) = 5.70, p = .02, η2p = .08. Specifically, although bilingual Japanese American children’s total vocabulary size was significantly smaller than monolingual English-speaking European American children, decomposition of the interaction effect showed that this was due to Japanese girls’ vocabulary size (M =70.69, SD = 118.18) being significantly smaller than European American girls’ vocabulary size (M = 205.10, SD = 113.01), F(1, 67) = 13.12, p = .001, η2p = .16.
Bilingual Korean American children’s total vocabularies did not differ in size from their monolingual peers; the significant main effect of Linguistic Community was the result of a significant difference between monolingual Korean and European American children, F(2, 122) = 6.71, p = .002, η2p = .10 (Table 2).
Input and Individual Differences in Vocabulary Development
As an initial step toward investigating factors that predict bilingual children’s vocabulary size, a series of two-tailed Pearson correlations between children’s vocabulary size in each language and cultural and input factors was performed. Generally mothers’ acculturation level when her infant was 5 months old and the percentage of time children were exposed to English were positively related to children’s English-language vocabulary size at 20 months (Table 3). The general trend in Table 4 is mothers’ acculturation level was positively related to the percentage of time children were exposed to English. Partial correlations were performed controlling sociodemographic covariates that correlated significantly (p < .05) with acculturation, percentage language exposure, or vocabulary size; the direction, magnitude, and significance of the correlations in Tables 3 and 4 did not change; therefore, covariates were not used in the path analyses below.
Table 3.
Correlations between Child Vocabulary Size and Acculturation and Input
| Mothers’ acculturation level | Number of years mothers lived in the United States | Mothers’ age at immigration | Percentage of time child is exposed to English | |
|---|---|---|---|---|
| English Vocabulary Size | ||||
| Latino (n = 29) | .37+ | .10 | .03 | .45* |
| Japanese American (n = 26) | .41+ | .29 | −.19 | .39* |
| Korean American (n = 35) | .29+ | .19 | −.19 | .26 |
| All bilingual groups combined | .36*** | .19+ | −.18 | .34*** |
| Heritage-Language Vocabulary Size | ||||
| Latino (n = 29) | .21 | .26 | −.19 | −.42* |
| Japanese American (n = 26) | .11 | .22 | .02 | −.03 |
| Korean American (n = 35) | −.38* | −.26 | .24 | −.44** |
| All bilingual groups combined | −.11 | −.01 | −.00 | −.33*** |
Note. Sample sizes vary due to missing data. Data are correlation coefficients.
p < .10,
p ≤ .05,
p ≤ .01,
p ≤ .001
Table 4.
Correlations between Mothers’ Acculturation Level and Language Exposure
| Cultural group | Correlation between Mothers’ Acculturation Level and Percentage of Time Child is Exposed to English |
|---|---|
| Latino immigrant (n = 25) | .16 |
| Japanese immigrant (n = 23) | .48* |
| Korean immigrant (n = 34) | .46** |
| All 3 immigrant groups combined | .36*** |
Note. Correlations between acculturation level and the percentage of time the child is exposed to their heritage language were the same magnitude and significance level as those reported above (because % exposure to heritage language = 100 – % exposure to English) but all were negative correlations.
p ≤ .05,
p ≤ .01,
p ≤ .001
Determinants of Input
Because the direction and magnitude of relations between acculturation, percentage of exposure to English, and vocabulary size in English were similar for all three bilingual groups, these samples were combined for path analyses. We attributed nonsignificance of correlations for some bilingual groups in Tables 3 and 4 to small sample sizes (because the magnitudes of the correlations were moderate, r = .30, to large, r = .50; Cohen 1988), and we performed a multiple group model to see whether the models fit equally well for each of the bilingual groups. The difference in model fit, as indexed by the change in chi-square between the pooled and multigroup models, was not significant for predicting English, Δχ2 (df = 6) = 2.41, ns, or heritage language, Δχ2 (df = 6) = 6.28, ns, suggesting that the model structure fit equally well for each of the bilingual groups therefore we were justified in grouping them (Jöreskog & Sörbom, 1993). The AMOS program was used for path analyses. Means and intercepts were estimated for missing data; less than 15% of data was missing at random. A model was considered to have good fit if the χ2 test was nonsignificant (p > .05), CFI was at or above .90 (Bentler, 1990; Marsh, Balla, & Hau, 1996), and RMSEA was .06 or smaller (Hu & Bentler, 1999). A path analysis was performed to investigate whether mothers’ acculturation level in infancy predicted children’s exposure to English which in turn predicted children’s vocabulary size in English; results showed that this model did not fit the data well, as indicated by χ2 (df = 1) = 9.72, p = .002, low CFI = .68, and high RMSEA = .31. Addition of a direct path from mothers’ acculturation level to child’s vocabulary size in English improved model fit, χ2 (df = 1) = 0.00, ns, high CFI = 1.00, and low RMSEA = .00. Figure 1 suggests that mothers who were more acculturated to American society exposed their children to more English which in turn resulted in larger English vocabulary sizes for their children, but that exposure to English only partially mediated the relation between mothers’ acculturation level and children’s English vocabulary size.
Figure 1. Model Predicting Immigrant Children’s English Vocabulary Size from Mothers’ Acculturation Level with Child’s Exposure to English as a Mediator.
Note. Data are standardized path coefficients.
*p < .05, ** p < .01, *** p ≤ .001
Additionally, we investigated whether mothers’ acculturation level in infancy predicted children’s heritage-language vocabulary size after children’s exposure to the heritage language was controlled. The path analysis results suggested that the model in Figure 2 fit the data well, as indicated by χ2 (df = 1) = 1.13, ns, high CFI = .99, and low RMSEA = .04. Thus, immigrant mothers who were less acculturated to U.S. American society (more acculturated to their heritage society) exposed their children to more of their heritage language, which in turn resulted in larger heritage-language vocabularies for their children. Exposure to the heritage language fully mediated the effect of acculturation on heritage-language vocabulary size.
Figure 2. Model Predicting Immigrant Children’s Heritage-Language Vocabulary Size from Mothers’ Acculturation Level with Child’s Exposure to Heritage Language as a Mediator.
Note. Data are standardized path coefficients.
** p < .01, *** p < .001
Discussion
This study extends previous research in three ways. First, the overwhelming majority of American studies examining bilingual children’s vocabulary development have focused on Spanish-English bilingual children; in addition to Spanish-English bilingual children we studied children acquiring English and either Japanese or Korean bilingually from birth. Second, in addition to comparing bilingual children to monolingual English speakers in the United States (immigrant mothers’ country of destination), in this study two groups of bilingual children (Spanish-English and Korean-English) were also compared to monolingual speakers of their heritage language in their mothers’ country-of-origin. Third, we assessed bilingual toddlers’ vocabulary development by comparing bilingual children to each other.
Input and Group Differences in Vocabulary Development
We took a snapshot of bilingual children’s vocabulary development in comparison to their monolingual peers at 20 months of age. Bilingual children’s vocabulary sizes were uniformly smaller than their monolingual peers in both the culture of destination and the culture of origin when each language was considered separately. These findings are consistent with previous research comparing Spanish-English bilinguals with English monolinguals (Hoff et al., 2012) and extend the findings to other English bilinguals (Japanese-English and Korean-English) and to vocabulary size in bilinguals’ heritage languages (Spanish and Korean). As Hoff et al. (2012) point out, although it may be the case that children can learn two languages at once without experiencing confusion or interference from one language to the other, it may simply take bilingual children longer to learn two languages than it takes monolingual children to learn one language because of lower relative exposure to each language (e.g., Bialystok, 2001; Paradis, Genessee, & Crago, 2011). The lack of differences among bilingual children’s vocabulary sizes in either English or their heritage language when there were no differences at the group level in terms of relative exposure to each language further suggests that input is a central feature of vocabulary learning. Future research should investigate whether and when the rate of language learning among bilingual children catches up to their monolingual peers, and whether their slower rate of vocabulary development has any short- or long-term consequences for children’s development. In essence, more research is needed to determine whether a slower rate of language development for bilingual children in toddlerhood is developmentally meaningful. Research reporting that bilingual children are at risk for poor academic performance (Federal Interagency Forum on Child and Family Statistics, 2013) suggests that it may be developmentally meaningful (although in the United States bilingualism and SES are often confounded).
Similar to previous studies (Hoff et al., 2012; Pearson et al., 1993), no significant differences in total vocabulary size between (socioeconomically-comparable) bilingual and monolingual children emerged in two of our samples (Japanese American, Korean American). An initial difference in total vocabulary size favoring monolingual English speakers over bilingual Japanese American children appears to be driven by gender, demonstrating the importance of including gender and other variables known or suspected of being associated with language development in statistical analyses because such variables may moderate effects of bilingualism (Bornstein, 2014).
Inconsistent with previous research (Pearson et al., 1993, Hoff et al., 2012), bilingual Latino children had smaller total vocabularies than their monolingual peers (both Argentine and European American children). This is particularly surprising because the socioeconomic status of the bilingual sample was comparable to the English monolinguals just as it was in the Hoff et al. (2012) study and the dependent variable was the same (total vocabulary size). Two possible sources of this difference come to mind, both implicating the importance of cultural context for multiple language acquisition (Bornstein, 2014). First, our Latino sample was more homogenous than Hoff et al.’s (2012) in terms of uniformity of generation level and country-of-origin, implicating potential cultural differences. Although the majority of parents in their bilingual sample (88.5%) were immigrants, all of ours were. Second, the immigrants in Hoff et al.’s (2012) study were from “Latin America or the Caribbean”, and previous research has shown differences in parenting and infant development between mothers from South America and those from the Caribbean (Puerto Rico and Cuba; Field & Widmayer, 1981). Third, the receiving communities in the current study and the Hoff et al. (2012) study were different: Hoff’s bilingual sample lived in South Florida where the status of Spanish speakers is relatively high, but in the community sampled in this study, the status of Spanish is comparatively low (Paradis et al., 2011), and bilingual language learning is less likely to be supported by the larger community.
Input and Individual Differences in Vocabulary Development
Consistent with previous research on Spanish-English bilingual children, in this study more relative exposure to a language was related to larger vocabulary size in that language for bilingual Latino children (Hoff et al., 2012; Marchman et al., 2004; Patterson, 2002; Pearson et al., 1997; Place & Hoff, 2011), and this was true for both English and the heritage language. Importantly, these results were also found for two sets of bilingual children rarely studied: Japanese-English and Korean-English bilinguals. Thus, this relation between input and vocabulary size may be a more universal feature of language acquisition regardless of which languages children are learning and whether they are acquiring one language or two from birth. More research with bilingual children learning a variety of languages is needed to substantiate this claim.
Determinants of Input
Consistent with theories of language socialization which posit that culture fundamentally shapes children’s linguistic input (Schieffelin & Ochs, 1986), immigrant mothers’ acculturation level predicted children’s exposure to English, which in turn predicted children’s total English vocabulary size, suggesting that mothers’ psychological acculturation influences the extent to which mothers choose to expose their children to English which in turn influences children’s English-language vocabulary development (productive vocabulary size in English), and the results were similar for heritage language. As many bilingual mothers intuitively know, if they want their children to retain their heritage language, they need to speak the heritage language to their children. Indeed, these results support the group-level finding that heritage language is typically lost after about three generations among immigrants (Bialystok, 2001; Paradis et al., 2011), but do so at the individual level. Mothers’ acculturation level also predicted children’s vocabulary size in English beyond what was mediated by input; future research ought to attempt to identify additional factors that mediate the relation between maternal acculturation and children’s vocabulary (see Bornstein, 2014).
Caveats and Future Directions
Methodology can influence results when children’s language development is the subject of study (Bornstein, Cote, et al., 2004). Ours is a maternal-report study, and different results may obtain when information about children’s productive vocabulary size is gathered during observation or standardized tests. It is important to note, however, that previous studies have shown positive relations between maternal report checklist measures of vocabulary development and observational measures of children’s vocabulary in bilingual children (e.g., Marchman & Martínez-Sussmann, 2002; Patterson, 2000). In research and in life, the term “bilingual children” encompasses heterogenous experiences and there is no prototypical bilingual child. We studied specific groups of bilingual children: those being reared in the United States bilingually simultaneously from birth with at least 10% exposure to the less frequently-heard language; thus, our results may only generalize to bilingual groups defined in a similar way. The Specificity Principle in multiple language learning (Bornstein, 2014) cites six factors that moderate dual language learning, including the specific languages being learned and setting conditions (e.g., parents’ desire to pass on the heritage language, whether children are linguistically isolated or live in heritage-language enclaves, whether bilingualism is the cultural norm, and whether the heritage language has a high or low status); when these conditions vary, differences in bilingual children’s language acquisition may result. Clearly, more international research in a variety of linguistic communities that differ with respect to the six factors that can moderate dual language learning is needed to uncover whether there are universals to bilingual language acquisition or whether specificity prevails. Finally, our sample size was not large relative to the number of analyses performed, so our results should be considered exploratory. However, to combat the number of analyses performed, in our analysis of group differences we controlled the Type I error rate using Bonferroni’s correction. Significance levels for correlation coefficients are highly influenced by sample size, so the magnitude of the correlation coefficient is always more revealing than the significance level. Using Cohen’s (1988) guidelines, many of our correlations were moderate to large.
These data are part of a longitudinal multimethod study of the development of children in immigrant families. Thus, our next steps are to investigate whether similar findings obtain when different methodologies are used (e.g., observation, standardized tests) and to ascertain the short- and long-term implications of differential rates of vocabulary development.
Acknowledgments
We thank M. Ogino, N. Okazaki, K. Painter, S. Park, L. Pascual, D. Putnick, M. Sandoval, K. Schulthess, W. Smith, and S. Toda. This research was supported by the Intramural Research Program of the NIH, NICHD, and by a Marymount University Faculty Development Grant to the first author.
Contributor Information
Linda R. Cote, Department of Psychology, Marymount University and Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health, Public Health Service
Marc H. Bornstein, Child and Family Research, Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health, Public Health Service
References
- Bates E, Beeghly M, Bretherton I, McNew S, O’Connell B, Reznick JS, et al. Unpublished measure. University of California; San Diego: 1984. Early Language Inventory. [Google Scholar]
- Bates E, Goodman JC. On the inseparability of grammar and the lexicon: Evidence from acquisition. In: Tomasello M, Bates E, editors. Language development: The essential readings. Malden, MA: Blackwell Publishing; 2001. pp. 134–162. [Google Scholar]
- Bates E, O’Connell B, Shore C. Language and communication in infancy. In: Osofsky JD, editor. Handbook of infant development. 2. New York: Wiley; 1987. pp. 149–203. [Google Scholar]
- Bentler PM. Comparative fit indexes in structural models. Psychological Bulletin. 1990;107:238–246. doi: 10.1037/0033-2909.107.2.238. [DOI] [PubMed] [Google Scholar]
- Bialystok E. Bilingualism in development: Language, literacy, and cognition. Cambridge, UK: Cambridge University Press; 2001. [Google Scholar]
- Bornstein MH. The Specificity Principle in multiple language learning. SRCD Social Policy Report. 2014;27(4):5. [Google Scholar]
- Bornstein MH, Cote LR, Maital S, Painter K, Park S, Pascual L, Vyt A. Cross-linguistic analysis of vocabulary in toddlers: Spanish, Dutch, French, Hebrew, Italian, Korean, and English. Child Development. 2004;75:1115–1139. doi: 10.1111/j.1467-8624.2004.00729.x. [DOI] [PubMed] [Google Scholar]
- Bornstein MH, Haynes OM. Vocabulary competence in early childhood: Measurement, latent construct, and predictive validity. Child Development. 1998;69:654–671. [PubMed] [Google Scholar]
- Bornstein MH, Hahn CS, Haynes OM. Specific and general language performance across early childhood: Stability and gender considerations. First Language. 2004;24:267–304. [Google Scholar]
- Bornstein MH, Putnick DL, Lansford JE, Pastorelli C, Skinner AT, Sorbring E, Oburu P. Mother and father socially desirable responding in nine countries: Two kinds of agreement and relations to parenting self-reports. International Journal of Psychology. doi: 10.1002/ijop.12084. (in press) [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bornstein MH, Tal J, Rahn C, Galperin CZ, Pecheux M, Lamour M, Tamis-LeMonda CS. Functional analysis of the contents of maternal speech to infants of 5 and 13 months in four cultures: Argentina, France, Japan, and the United States. Developmental Psychology. 1992;28:593–603. [Google Scholar]
- Boyce LK, Gillam SL, Innocenti MS, Cook GA, Ortiz E. An examination of language input and vocabulary development of young Latino dual language learners living in poverty. First Language. 2013;33:572–593. [Google Scholar]
- Brislin RW. The wording and translation of research instruments. In: Lonner WJ, Berry JW, editors. Field methods in cross-cultural research. Newbury Park, CA: Sage; 1986. pp. 137–164. [Google Scholar]
- Chomsky N. Rules and representations. New York: Columbia University Press; 1980. [Google Scholar]
- Cohen J. Statistical power analysis for the behavioral sciences. 2. Hillsdale, NJ: Erlbaum; 1988. [Google Scholar]
- Conboy BT, Thal DJ. Ties between the lexicon and grammar: Cross-sectional and longitudinal studies of bilingual toddlers. Child Development. 2006;77:712–735. doi: 10.1111/j.1467-8624.2006.00899.x. [DOI] [PubMed] [Google Scholar]
- Crowne DP, Marlowe D. A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology. 1960;24:349–354. doi: 10.1037/h0047358. [DOI] [PubMed] [Google Scholar]
- Dale P, Bates E, Reznick S, Morisset C. The validity of a parent report instrument of child language at 20 months. Journal of Child Language. 1989;16:239–249. doi: 10.1017/s0305000900010394. [DOI] [PubMed] [Google Scholar]
- Eilers RE, Pearson BZ, Cobo-Lewis AB. Social factors in bilingual development: The Miami experience. In: McCardle P, Hoff E, editors. Childhood bilingualism. Buffalo, NY: Multilingual Matters LTD; 2006. pp. 68–90. [Google Scholar]
- Fenson L, Dale PS, Reznick JS, Bates E, Thal DJ, Pethick SJ. Variability in early communicative development. Monographs of the Society for Research in Child Development. 1994;59 (5, Serial No. 242) [PubMed] [Google Scholar]
- Fenson L, Dale PS, Reznick JS, Thal DJ, Bates E, Hartung JP, et al. MacArthur Communicative Development Inventories: User’s guide and technical manual. San Diego, CA: Singular Publishing Group; 1993. [Google Scholar]
- Fenson L, Thal DJ, Bates E. Unpublished manuscript. San Diego State University; CA: 1990. Normed values for the ‘Early Language Inventory’ and three associated parent report forms for language assessment. [Google Scholar]
- Field TM, Widmayer SM. Mother-infant interactions among lower SES Black, Cuban, Puerto Rican and South American immigrants. In: Field TM, Sostek AM, Vietze P, Leiderman PH, editors. Child psychology: Cultural and early interactions. Hillsdale, NJ: Erlbaum; 1981. pp. 41–62. [Google Scholar]
- Gathercole VCM, Hoff E. Input and the acquisition of language: Three questions. In: Hoff E, Shatz M, editors. Blackwell handbook of language development. Malden, MA: Blackwell; 2007. pp. 107–127. [Google Scholar]
- Hammer CS, Miccio AW, Rodriguez BL. Bilingual language acquisition and the child socialization process. In: Goldstein BA, editor. Bilingual language development and disorders in Spanish-English speakers. Baltimore, MD: Brookes; 2004. pp. 21–50. [Google Scholar]
- Hoff E, Core C, Place S, Rumiche R, Señor M, Parra M. Dual language exposure and early bilingual development. Journal of Child Language. 2012;39:1–27. doi: 10.1017/S0305000910000759. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hollingshead AB. A four-factor index of social status. Yale University; New Haven, CT: 1975. Unpublished manuscript. Available: http://www.yale.edu/sociology/archives.html (retrieved July 10 2013) [Google Scholar]
- Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6:1–55. [Google Scholar]
- Huttenlocher J, Haight W, Bryk A, Seltzer M, Lyons T. Early vocabulary growth: Relation to language input and gender. Developmental Psychology. 1991;27:236–248. [Google Scholar]
- Jöreskog K, Sörbom D. Lisrel 8: Structural Equation Modeling with the SIMPLIS Command Language. Lincolnwood, IL: Scientific Software International; 1993. [Google Scholar]
- Kamada LD. Monographs on Bilingualism. 5. Vol. 2. Osaka, Japan: Bilingualism National Special Interest Group of the Japan Association for Language Teaching; 1997. Bilingual family case studies. [Google Scholar]
- Lenneberg E. Biological foundations of language. New York: Wiley; 1967. [Google Scholar]
- MacWhinney B. A multiple process solution to the logical problem of language acquisition. Journal of Child Language. 2004;31:883–914. doi: 10.1017/s0305000904006336. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Marchman VA, Martínez-Sussmann C. Concurrent validity of caregiver/parent report measures of language for children who are learning both English and Spanish. Journal of Speech, Language, and Hearing Research. 2002;45:983–997. doi: 10.1044/1092-4388(2002/080). [DOI] [PubMed] [Google Scholar]
- Marchman VA, Martínez-Sussmann C, Dale PS. The language-specific nature of grammatical development: Evidence from bilingual language learners. Developmental Science. 2004;7:212–224. doi: 10.1111/j.1467-7687.2004.00340.x. [DOI] [PubMed] [Google Scholar]
- Marsh HW, Balla JR, Hau K-T. An evaluation of incremental fit indices: A clarification of mathematical and empirical properties. In: Marcoulides GA, Schumacker RE, editors. Advanced structural equation modeling: Issues and techniques. Mahwah, NJ: Erlbaum; 1996. [Google Scholar]
- Nicoladis E. Bilingualism and language cognitive development. In: Altarriba J, Heredia RR, editors. An introduction to bilingualism: Principles and processes. New York: Erlbaum; 2008. pp. 167–181. [Google Scholar]
- Paradis J, Genesee F, Crago MB. Dual language development and disorders: A handbook of bilingualism and second language learning. 2. Baltimore: Paul H. Brookes; 2011. [Google Scholar]
- Parra M, Hoff E, Core C. Relations among language exposure, phonological memory, and language development in Spanish–English bilingually-developing two-year-olds. Journal of Experimental Child Psychology. 2011;108:113–25. doi: 10.1016/j.jecp.2010.07.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Patterson JL. Observed and reported expressive vocabulary and word combinations in bilingual toddlers. Journal of Speech, Language, and Hearing Research. 2000;43:121–128. doi: 10.1044/jslhr.4301.121. [DOI] [PubMed] [Google Scholar]
- Patterson JL. Relationships of expressive vocabulary to frequency of reading and television experience among bilingual toddlers. Applied Psycholinguistics. 2002;23:493–508. [Google Scholar]
- Pearson BZ. Assessing lexical development in bilingual babies and toddlers. The International Journal of Bilingualism. 1998;2:347–372. [Google Scholar]
- Pearson BZ, Fernández SC, Lewedeg V, Oller DK. The relation of input factors to lexical learning by bilingual infants. Applied Psycholinguistics. 1997;18:41–58. [Google Scholar]
- Pearson BZ, Fernández SC, Oller DK. Lexical development in bilingual infants and toddlers: Comparison to monolingual norms. Language Learning. 1993;43:93–120. [Google Scholar]
- Peña ED. Lost in translation: Methodological considerations in cross-cultural research. Child Development. 2007;78:1255–1264. doi: 10.1111/j.1467-8624.2007.01064.x. [DOI] [PubMed] [Google Scholar]
- Place S, Hoff E. Properties of dual language exposure that influence 2-year-olds’ bilingual proficiency. Child Development. 2011;82:1834–1849. doi: 10.1111/j.1467-8624.2011.01660.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rescorla L. Comparing expressive vocabulary size in children from bilingual and monolingual home environments: A reply to Patterson’s (2004) commentary. Journal of Speech, Language, and Hearing Research. 2004;47:1213–1217. doi: 10.1044/1092-4388(2004/089). [DOI] [PubMed] [Google Scholar]
- Schieffelin BB, Ochs E, editors. Language socialization across cultures. Cambridge: Cambridge University Press; 1986. [Google Scholar]
- Song J. Language ideology and identity in transnational space: Globalization, migration, and bilingualism among Korean families in the USA. International Journal of Bilingual Education and Bilingualism. 2010;13:23–42. [Google Scholar]
- Thal D, Jackson-Maldonado D, Acosta D. Validity of a parent-report measure of vocabulary and grammar for Spanish-speaking toddlers. Journal of Speech, Language, and Hearing Research. 2000;43:1087–1100. doi: 10.1044/jslhr.4305.1087. [DOI] [PubMed] [Google Scholar]
- Vagh SB, Pan BA, Mancilla-Martinez J. Measuring growth in bilingual and monolingual children’s English productive vocabulary development: The utility of combining parent and teacher report. Child Development. 2009;80:1545–1563. doi: 10.1111/j.1467-8624.2009.01350.x. [DOI] [PubMed] [Google Scholar]
- van de Vijver FJR, Leung K. Methods and data analysis for cross-cultural research. Thousand Oaks, CA: Sage; 1997. [Google Scholar]


