Measurement Invariance of Language Brokering Extent and Attitudes in Linguistic Minority Adolescents: Item Response Theory Analyses

Abstract

Objective:

Using item response theory, we sought to evaluate measurement invariance of language brokering extent and attitudes in three linguistic minority groups of adolescents.

Methods:

The sample included 765 (302 Chinese American, 327 Korean American, and 136 Mexican American; 57% females) ninth graders from immigrant families.

Results:

Differential item functioning was detected for several items, and we retained items with equivalent parameters across three groups. The final items showed adequate internal consistency reliability and criterion validity.

Conclusion:

The refined scale is cross-ethnically invariant and appropriate for use with Chinese American, Korean American, and Mexican American adolescents to compare their language brokering experiences.

Today’s U.S. demography is rapidly changing, and 41 percent of its population growth over the past five decades is attributable to an influx of Latino and Asian immigrants (Lopez, Passel, & Rohal, 2015). Due to the limited English proficiency of many Latino and Asian parents, children in these families often assist their parents by translating from English to the heritage language, an activity known as language brokering (McQuillan & Tse, 1995). There has been a growing body of literature investigating adolescents’ language brokering experiences, with the most represented ethnicities being Mexican Americans, Chinese Americans, followed by Korean Americans (Shen, Tilton, & Kim, 2017). However, it is not known whether language brokering measures are interpreted equivalently across different ethnic groups. Therefore, we sought to investigate measurement invariance of existing language brokering scales (Buriel, Perez, De Ment, Chavez, & Moran, 1998; Chao, 2006; Tse, 1996) using item response theory (IRT) analyses in Chinese American, Korean American, and Mexican American adolescents.

Language Brokering Dimensions and the Need to Examine Cross-ethnic Invariance

Since the development of the initial language brokering scale by Tse (1996), which assessed Latino youth’s language brokering prevalence and attitudes, researchers have used various modifications of this early measurement (e.g., Buriel, et al., 1998; Chao, 2006). Despite the different instruments, scholars have consistently measured two main aspects of language brokering: the extent (i.e., how often adolescents provide brokering) and attitudes (i.e., how adolescents feel about brokering). Recent investigations of language brokering have further identified sub-dimensions of the two main domains. For example, Roche and colleagues (2015) found that the extent of language brokering could be divided into the extent of school-related brokering (e.g., homework) and home management-related brokering (e.g., household bills), based on different contexts and contents. In addition, recent theorization (Kam & Lazarevic, 2014a) and empirical evidence (Kam & Lazarevic, 2014b) identified that language brokering attitudes included not only positive (e.g., feeling independent) and negative (e.g., annoyed) feelings, but also encompassed another dimension of viewing brokering as a burden (e.g., burden on one’s time).

Although there is growing research on the relations between different dimensions of language brokering and adolescents’ psychosocial well-being and family dynamics, evidence has been quite mixed regarding the role of language brokering extent (Kam & Lazarevic, 2014a). Among other factors, cross-ethnic differences seem to have contributed to the mixed findings. For example, Chao (2006) averaged the frequencies of translating nine different materials among Mexican American, Chinese American, and Korean American adolescents, and found that frequent language brokering for mothers was related to increased internalizing symptoms in the two Asian American groups, but not in Mexican American adolescents. Despite the high internal consistency of the measure (Chao, 2006), however, it is not known whether the items were understood equivalently across different ethnic and linguistic groups, and in turn, whether the ethnic differences were true differences or measurement artifacts.

In terms of brokering attitudes, the theory of language brokering (Kam & Lazarevic, 2014a) suggests that different language brokering attitudes are associated with adolescents’ psychosocial well-being and family dynamics in distinct ways. Supporting this theory, empirical research suggests that Latino adolescents’ positive attitudes toward brokering are related to their psychosocial well-being (Kam, 2011; Weisskirch, 2007) and parent-child bonding (Buriel, Love, & De Ment, 2006). In contrast, Mexican American adolescents’ negative feelings about brokering were associated with problematic family relationships (Weisskirch, 2007). Similarly, a sense of burden in brokering was predictive of Chinese American adolescents’ depressive symptoms (Kim et al., 2014). Although these studies have used adaptations of Tse’s (1996) language brokering scale that are similar in content, the measures were either not validated or only validated within a single ethnic or linguistic group (e.g., Kim, Hou, Shen, & Zhang, 2017; Kim et al., 2014). For any cross-group comparison to be meaningful, the items must function equivalently across groups (Meade & Lautenschlager, 2004), and as such, the cross-ethnic measurement invariance of language brokering attitudes (Buriel et al., 1998; Tse, 1996) needs to be examined.

Differences in social contexts and linguistic backgrounds may contribute to differential item functioning (DIF). For example, Latinos are less likely to read newspapers than other ethnic groups (Pew Research Center, 2012). Thus, for youth who provide home management language brokering to a similar extent, perhaps Mexican American youth provide less translation for media than do Asian American youth. Furthermore, compared to Asian languages, because of the linguistic similarity and cross-language transferability of vocabulary between Spanish and English (Chiswick & Miller, 2005; Pasquarella, Chen, Gottardo, & Geva, 2015), Mexican American youth’s negative attitudes toward brokering may be related to a lesser degree to their concerns about translating poorly.

Current Study

The goal of this study was to examine measurement invariance of language brokering extent (Chao, 2006) and attitudes (Buriel et al., 1998; Tse, 1996) across Mexican American, Chinese American, and Korean American adolescents using IRT DIF analyses. The psychometric information (i.e., item discrimination and location¹) provided by IRT is sample independent, and the scores describing participant latent characteristics (i.e., theta) are scale independent (Hambleton & Jones, 1993; Rouse, Finger & Butcher, 1999). Moreover, IRT models allow for visual inspections of categorical response curves and information function curves (Baker & Kim, 2004). Therefore, IRT models are particularly useful for assembling equivalent forms of measurements across different groups.

Upon retaining a list of cross-ethnically equivalent items, we aimed to further examine the reliability and criterion validity of the refined list of items. We selected two individual psychosocial outcomes (i.e., internalizing and externalizing symptoms) and three measures of family dynamics (i.e. parental psychological control, parent-child communication, and child respect for parents) as criterion variables. While we did not have a priori hypotheses about the effects of language brokering extent due to mixed evidence in the literature, we hypothesized that positive brokering attitudes (versus negative attitudes and burden) would generally relate to positive (versus negative) individual psychosocial outcomes and family dynamics.

Methods

Data Source

Participants were ninth graders (in 2003; Wave 1; n_Chinese = 302, 57% female; n_Korean = 327, 54% female; n_Mexican = 136, 66% female) residing in the Los Angeles area, who were followed up one year (Wave 2; retention rate: 68%) and two years later (retention rate from Wave 2: 77%). The Mexican American sample was excluded for Wave 3 due to the small sample size (n = 28). Research procedures were approved by the fifth author’s Institutional Review Board. Participants completed paper-and-pencil surveys in English within fifty minutes of a class period. Thirty-five percent of the participants reported that they were born outside of the United States. Fewer Mexican American parents had college degrees (mother: 19%, father: 18.3%) than their Chinese (mother: 74%, father: 83%) and Korean American counterparts (mother: 73%, father: 84%).

Measures

Language brokering.

Since the majority of adolescents (86%) reported their primary caregivers to be mothers, we used participants’ responses for language brokering for mothers. Language Brokering Extent was assessed at Wave 1, and Language Brokering Attitudes subscale was added to the project at Wave 2. To reexamine item properties at the same, later wave, Wave 3 data were also included (see top panel of Table 3 for waves of measures). Language Brokering Extent was measured with ten items (Chao, 2006) assessing how frequently adolescents translate from English to a non-English language on a scale ranging from 0 (not at all) to 4 (daily). Language Brokering Attitudes were measured with 22 items assessing adolescents’ subjective feelings toward language brokering on a scale ranging from 0 (strongly disagree) to 4 (strongly agree). These items were adapted from two previous language brokering scales (Buriel et al., 1998; Tse, 1996). Eight items were either directly adopted from the previous language brokering scales or slightly modified (e.g., “embarrassed” modified as “ashamed”), and the remaining items were adapted based on four original items on brokering-related efficacy, stress, and negative feelings. The items are presented in Table 1, and Cronbach’s αs for the final subscales are presented in Table 2.

Table 3.

Time points of variables for item property and criterion validity examinations

Item Property Examination
	Language Brokering Extent		Language Brokering Attitudes
Wave 1	Initial		------	------
Wave 2			Initial
Wave 3		Follow-up		Follow-up
	Criterion Validity Examination
	Language Brokering Extent		Language Brokering Attitudes
Wave 1	Concurrent		------	------
Wave 2		Predictive	Concurrent
Wave 3				Predictive

Note. Language brokering attitudes were not assessed at Wave 1.

Table 1.

Items and factor loadings of the Language Brokering Inventory

Language Brokering Extent
How often have you translated the following things from English to the other non-English language for mother? (0: not at all, 1: a few times a year, 2: a few times a month, 3: a few times a week, 4: daily)	SCH	HOM
1. Homework or assignments from school	.81	−.05
2. Materials from school (notices, newsletters, permission slips, progress reports)	.85	.03
3. Meetings or conversations between your parent and teacher or other school staff.	.49	.33
4. Household bills or financial materials	−.07	.88
5. Household matters not related to money (chores, siblings or relatives)	.10	.69
6. Medical or health issues (doctor visits, medication)	.07	.80
7. Immigration and naturalization papers	−.07	.86
8. Media (T.V. programs, news, newspapers, etc.)	.35	.38
9. Items or issues for your parent’s work / business.	−.04	.85
10. Other types of things not listed above.	.22	.50
Language Brokering Attitudes
Please think about your translating experiences with your parents, and indicate how much you agree with each statement below. (0: strongly disagree, 1: disagree, 2: neutral, 3: agree, 4: strongly agree)	BUR	POS	NEG
11. It is stressful to translate for my parent.	.82	.03	−.02
12. Translating for my parent is a burden.	.76	−.04	.17
13. Translating for my parent takes time away from other things I want to do.	.60	.05	.20
14. My parent thinks highly of me because I translate for him/her.	.13	.59	.13
15. Translating for my parent makes me feel independent and mature.	.14	.62	.11
16. I feel useful when I translate for my parent.	.15	.78	−.18
17. I have a deeper understanding of my parent because I translate for him/her.	−.07	.77	.13
18. My parent trusts me because I translate for him/her.	−.04	.72	.12
19. I am proud to translate for my parent.	.00	.84	−.11
20. I like the responsibility of translating for my parent.	.03	.79	−.08
21. I feel important when I translate for my parent.	.03	.82	.03
22. Translating for my parent is personally rewarding.	−.10	.86	−.01
23. I am closer to my parent because I translate for him/her.	−.16	.77	.12
24. I have disappointed my parent by translating poorly.	.19	.09	.52
25. I do not have respect for my parent because I translate for him/her.	−.06	−.04	.76
26. I feel more knowledgeable than my parent because I translate for him/her.	.04	.24	.53
27. I feel it is unfair for my parent to ask me to translate.	−.04	.06	.81
28. I feel annoyed when my parent asks me to translate.	.20	−.05	.68
29. When my parent asks me to translate. I feel ashamed that he/she cannot speak English very well.	−.03	−.01	.84
30. I feel angry when my parent asks me to translate.	.12	−.08	.78
31. Some of the things my parent asks me to translate make me feel uncomfortable.	.04	.03	.76
32. I feel a lot of pressure when I translate for my parent.	.24	.12	.50

Note. Bolded items are those included in the recommended final scale. SCH = School-related language brokering; HOM = Home management language brokering; BUR = Burden of language brokering; POS = Positive attitudes; NEG = Negative attitudes.

Table 2.

Reliability, means, standard deviations (SD), analysis of variance (ANOVA), and correlations of the language brokering subscales

	Cronbach alpha			Mean (SD)				Correlation
Subscale	C	K	M	C	K	M	F	1	2	3	4
Extent
1. School-related	.81	.78	79	1.55a (1.09)	1.72ab (1.04)	1.91b (1.13)	5.45**	-
2. Home management	.89	.89	.91	0.95a (1.03)	1.13a (1.06)	1.55b (1.22)	13.30***	.61***	-
Attitudes
3. Burden	.86	.77	.84	2.69a (0.89)	2.56a (0.96)	2.31b (0.99)	3.48*	−.001	.03	-
4. Positive	.91	.94	.89	2.91a (0.75)	2.91a (0.92)	3.09a (1.05)	1.04	.24**	.15+	.20***	-
5. Negative	.90	.79	.84	2.18a (0.83)	2.15a (0.84)	2.19a (1.00)	0.05	−.003	−.08	.61***	.29***

Note. C = Chinese. K = Korean. M = Mexican. Means within a row with non-common subscripts were statistically significantly different using Tukey post-hoc tests (p < .05). For example, a value with the subscript a is significantly different from one with the subscript b, but not from one with the subscript ab.

⁺p < .10;

^*p < .05;

^**p < .01;

^***p < .001.

Criterion measures.

We included individual psychosocial outcomes to examine both concurrent and predictive validity (see bottom panel of Table 3 for waves of criterion measures). All Cronbach’s αs are for the current sample. Adolescents rated items from the Youth Self Report (Achenbach, 1991), including internalizing (32 items, αs = .92~.96, e.g., “I feel worthless or inferior”) and externalizing symptoms (31 items, αs = 88~.91, e.g., “I physically attack people”) on a scale ranging from 0 (not true) to 2 (very true or often true). The measures have been shown to demonstrate adequate validity (Achenbach, 1991).

For family dynamics measures, we only included concurrent measures, as the directionality of the relation between adolescents’ language brokering and family dynamics has not been established in the literature. Parental psychological control was measured with eight items (Barber, 1996; αs = .74~.83, e.g., “My parent is always trying to change how I feel or think about things”). Communication with the mother was measured using eight items adapted from the Parent Adolescent Communication Scale (Barnes & Olson, 1982; αs = .82~.85, e.g., “I spend enough time communicating with my mother”). Finally, respect for the mother was measured with eight items adapted from the Parental Identification measure (Bowerman & Bahr, 1973; αs = .89~.91, e.g., “I respect my mother’s opinions about important things in my life”). All scales ranged from 1 (strongly disagree) to 5 (strongly agree). These measures have demonstrated adequate validity with ethnically diverse samples (e.g., Bean & Northrup, 2009; Chao & Aque, 2009; Chao & Kanatsu, 2008; Shen, Kim, Wang, & Chao, 2014).

Data Analyses

We conducted exploratory factor analysis using IBM SPSS 24 with direct oblimin rotation to identify subdimensions. Then, for each subdimension, we estimated graded response IRT models (Samejima, 1996) using IRTPRO (Cai, Thissen, & du Toit, 2011), which uses maximum likelihood estimation to assess item properties, and Wald tests to examine DIF in item discrimination and locations simultaneously. Good item properties were indicated by discrimination parameters greater than 1.35 and no DIF across groups (Baker, 2001). Two sets of comparisons were conducted simultaneously: Mexican Americans versus Chinese and Korean Americans together (reference group) and Korean Americans versus Chinese Americans (reference group). Additionally, DIF analyses were conducted iteratively: when a Wald test detected multiple items with DIF, we eliminated one item at a time that displayed the largest differences in expected scores across groups until no DIF was detected. We also conducted additional IRT analyses using the data collected at Wave 3 to ensure that each item displayed appropriate properties after one or two years. Finally, we conducted reliability, descriptive, and partial correlational analyses using the retained items.

Results

Dimension Identification

The results of exploratory factor analyses (Table 1) suggest that the language brokering extent scale was composed of two factors: school-related (items 1-3) and home management (items 4-9). Eigenvalue was 5.43 for the first factor (54.32% of variance explained), and 1.14 for the second factor (11.43% of variance explained). We eliminated item 10 due to its ambiguity (“other types of things”). The language brokering attitudes scale had three factors: burden (items 11-13), positive attitudes (items 14-23), and negative attitudes (items 24-32). Eigenvalue was 9.35 (42.5% variance explained) for the first factor, 3.98 for the second factor (18.09% variance explained), and 1.16 for the final factor (5.26% variance explained).

IRT Analyses

For School-related Language Brokering, all items displayed very high item discrimination parameters (> 1.70). No item displayed DIF for either set of comparisons. Therefore, we retained all three items. Six items from the Home Management Language Brokering component showed high (> 1.35) item discrimination parameters. Items 6 and 7 exhibited statistically significant DIF in item discrimination, χ²₆ (1) = 3.9, p₆ = .047, χ²₇(1) = 4, p₇ = .045, and item 8 displayed DIF in item location, χ²(4) = 10.3, p = .036. Item 8 showed the largest differences in expected scores across ethnic groups (Figures 1-2). Thus, a follow-up IRT analysis was conducted after eliminating item 8, and the remaining items did not show DIF.

Figure 1.

Expected score differences for excluded items. Chinese American = Orange solid line. Korean American = Blue dashed line. Mexican American = Black dotted line.

Figure 2.

Expected score differences across ethnicities for retained items. Chinese American = Orange solid line. Korean American = Blue dashed line. Mexican American = Black dotted line.

All three items from the Burden component had very high discrimination parameters, with no DIF. Thus, we retained all three items. Ten items from Positive Attitudes had very high discrimination parameters (> 1.75). Items 14 and 23 exhibited DIF in item discrimination, χ²₁₄ (1) = 7.3, p₁₄ = .007, χ²₂₃ (1) = 9.7, p₂₃ = .002, and item 19 showed DIF in item location, χ² (4) = 10.7, p = .03. Significant differences in expected scores were detected for item 14 and 23 but not for item 19 (Figures 1-2). After eliminating items 14 and 23, there was no additional item with DIF. Finally, for Negative Attitudes, no Chinese American adolescents endorsed the final response category on item 31. Because item location for the threshold between the last two response categories could not be calculated, we omitted item 31. All other items had high (> 1.35) discrimination parameters. Item 26 showed DIF in item discrimination and item location, χ²_total (5) = 12.1, p_total = .034. A follow-up analysis excluding item 26 suggested that item 25 also had DIF in item discrimination, χ²(1) = 3.9, p = .05. After dropping item 25, no DIF was found.

Additional IRT analyses using Wave 3 data showed high item discrimination (> 1.35) for all items except for item 24 (a = 1.06 for Korean Americans). Therefore, we eliminated item 24 from our final list of items. The remaining items all displayed similar item locations (rs = .65~.93) and similarly high item discrimination parameters (r = .84) across initial and followup item property examinations.

Reliability and Criterion Validity

Cronbach’s alpha coefficients, means, standard deviations, correlations, and ANOVA omnibus and post-hoc test results for each subscale are shown in Table 2. Reliabilities were within the range of .77 to 94. Mexican American adolescents reported higher home management brokering frequency than Chinese and Korean Americans and higher school-related brokering frequency than Chinese Americans but reported lower levels of burden than Chinese and Korean American adolescents. Partial correlations (Tables 4-5) suggested that burden and negative attitudes of brokering were generally related to greater psychological control, less communication, and less respect for parents.

Table 4.

Partial correlations among language brokering subscales and adolescents’ individual psychosocial outcomes.

LB Subscale	Internalizing symptoms						Externalizing symptoms
	Concurrent			Predictive			Concurrent			Predictive
	C	K	M	C	K	M	C	K	M	C	K	M
Extent
School-related	.08	.16**	−.02	.00	.15*	.01	−.09	.03	−.12	−.11	.01	−.19
Home management	.12	.11	.02	.05	.18**	.03	−.02	.03	.09	−.02	.06	−.17
Attitudes
Burden	.35**	.21	.18	.30**	−.04	.23	.12	.25	.15	.09	.00	.56*
Positive	.06	.18	−.18	−.05	−.20	.05	−.03	.05	−.36*	−.09	−.12	−.07
Negative	.27*	.14	.15	.26*	−.10	−.12	.08	.29	.14	.02	−.01	.44

Note. C = Chinese. K = Korean. M = Mexican. Partial correlations after controlling for maternal education are presented. Concurrent correlations were based on internalizing and externalizing symptoms measured during the same year that language brokering was measured (Wave 1 for language brokering extent subscales and Wave 2 language brokering attitudes subscales). Predictive correlations were based on internalizing and externalizing symptoms measured one year after language brokering was measured (Wave 2 for language brokering extent subscales and Wave 3 for language brokering attitudes subscales).

^*p < .05;

^**p < .01.

Table 5.

Partial correlations among language brokering subscales and concurrent family dynamics measures.

LB Subscale	Psychological control			Communication			Respect for parents
	C	K	M	C	K	M	C	K	M
Extent
School-related	.05	.03	−.11	.11	.00	.21*	.15*	.12*	.29**
Home management	.21***	.11	.04	.01	.02	−.11	.10	.05	.02
Attitudes
Burden	.19	.20	.10	−.31**	−.32**	−.34*	−.29*	−.23*	−.22
Positive	.00	−.04	.03	.20	.20	.42**	.10	.22	.32*
Negative	.20*	.25*	.44**	−.25*	−.21	−.46**	−.25*	.01	−.44**

Note. C = Chinese. K = Korean. M = Mexican. Partial correlations after controlling for maternal education are presented. Correlations were based on family dynamic measures assessed during the same year that language brokering was measured (Wave 1 for language brokering extent subscales and Wave 2 for language brokering attitudes subscales).

^*p < .05;

^**p < .01;

^***p < .001.

Discussion

Using IRT analyses, we investigated the measurement equivalence of a previously used language brokering extent measure (Chao, 2006) and a modified version of a language brokering attitudes measure (Buriel et al., 1998; Tse, 1996) across three ethnic and linguistic groups: Chinese American, Korean American, and Mexican American adolescents. We detected DIF for several items, suggesting cross-ethnic measurement nonequivalence, and recommend using the refined list of psychometrically invariant items for future cross-ethnic investigations of adolescent language brokering experiences.

In line with our expectation, the item assessing the extent of translation of media was dropped due to differential item location for Mexican American adolescents. Contrary to our hypothesis, however, the item measuring adolescents’ concerns about translating poorly was dropped because of low item discrimination for Korean Americans, rather than for Mexican Americans. Additionally, measurement noninvariance was observed multiple items for Korean American adolescents, as they tended to rate their brokering experiences—both positive and negative—more extremely. While more studies are needed to understand the causes of group differences in these items, we recommend that research on language brokering adopt the refined, invariant list of items to avoid making false conclusions about cross-group differences. Additionally, although we obtained invariant measures for three of the most represented ethnic and linguistic groups in the United States (Shen et al., 2017), measurement invariance testing with other immigrant groups (e.g., Arab Americans) is a necessary future direction for meaningful comparisons of brokering experiences across groups.

Turning to mean-level differences from post-hoc analyses, the finding that Mexican American adolescents generally provide more frequent language brokering than Asian Americans partly reflects the socioeconomic differences in the ethnic groups, as parents with higher education tend to require less brokering (Chao, 2006). However, Mexican American adolescents are not as burdened by this additional obligation as Asian Americans, suggesting that being a language broker may be a central part of Latino adolescents’ self-concepts and a normal part of their everyday lives (Corona et al., 2012; Kim et al., 2017).

In terms of the associations between language brokering and adolescents’ outcomes, differences from previous findings were found. For example, Mexican American adolescents’ healthy parent-child relationships were not associated with the frequency of home management brokering, but positively associated with that of school-related brokering. This differs from a previous finding of the pernicious effects of home management, but not school-related, language brokering (Roche et al., 2015).

Additionally, cross-ethnic differences were observed, even after using the psychometrically invariant measures and controlling for maternal education, suggesting true ethnic differences. For example, psychological burden and negative experiences were associated with concurrent and later internalizing symptoms for Chinese American adolescents only. Future research is needed to investigate how contextual and individual differences may contribute to these variations. Furthermore, as this study only investigated adolescents’ language brokering for mothers, the role of parent gender also needs to be investigated in the future.

Conclusion

To our knowledge, this is the first study to examine cross-ethnic invariance of language brokering measurement. The refined scale is cross-ethnically equivalent and appropriate for use in comparing Chinese, Korean, and Mexican American adolescents’ language brokering experiences.