Measurement Invariance and Convergent Validity of Anger and Sadness Self-Regulation Scales Among Youth from Six Cultural Groups

Abstract

The present study examined measurement invariance and convergent validity of a novel vignette-based measure of emotion specific self-regulation that simultaneously assesses attributional bias, emotion-regulation, and self-efficacy beliefs about emotion regulation. Participants included 541 youth-mother dyads from three countries (Italy, United States, and Colombia) and six ethnic/cultural groups. Participants were 12.62 years old (SD = 0.69). In response to vignettes involving ambiguous peer interactions, children reported their hostile/depressive attribution bias, self-efficacy beliefs about anger and sadness regulation, and anger/sadness regulation strategies (i.e., dysregulated expression and rumination). Across the six cultural groups, anger and sadness self-regulation subscales had full metric and partial scalar invariance for a one-factor model, with some exceptions. We found support for both a four- and three-factor oblique model (dysregulated expression and rumination loaded on a second-order factor) for both anger and sadness. Anger subscales were related to externalizing problems, while sadness subscales were related to internalizing symptoms.

Self-regulation in childhood and adolescence has figured prominently into scientific inquiries about factors that contribute to psychological adjustment and well-being likely because both positive and negative adjustment in adulthood are predicted by consequences of self-regulation in childhood (Ayduk, Mendoza-Denton, Mischel, Downey, Peake, & Rodriguez, 2000; Rutter, 2011). The ways children interpret others’ intentions in social exchanges, children’s beliefs about the capacity to deal with emotions, and the strategies children use to manage emotions all have been significantly correlated with psychological maladjustment (e.g., Orobio de Castro, Veerman, Koops, Bosch, & Monshouwer, 2002; Eisenberg, Spinrad, & Eggum, 2010), and are often the focus of mental health interventions (Caprara et al., 2014; Conduct Problems Prevention Research Group, 1992). To expand the scientific evidence on self-regulation, more cross-cultural research is needed, as are evidence-based assessments that tap novel components of self-regulation. This study evaluates a measure of self-regulation in pre-adolescence that assesses self-regulation across multiple domains (i.e., cognitive and emotional) in reaction to social exchanges. The primary goals of this study were to investigate the structural properties of this measure across culturally diverse pre-adolescent samples and to examine its construct validity in relation to empirically validated measures and mental health symptoms. We focus our investigation on pre-adolescents because of the importance of identifying precursors of risk and resiliency at key developmental “switchpoints” such as adolescence (Dodge & Pettit, 2003). Evidence is clear that adolescence is associated with increases in the prevalence of behavioral and mental health issues (e.g., Cicchetti & Toth, 1998; Silk, Steinberg, & Morris, 2003). Better understanding self-regulation in pre-adolescence could advance our knowledge of key targets for prevention and early intervention strategies seeking to thwart the adolescent onset of behavioral and mental health issues.

Self-regulation encompasses a broad array of internal and external processes that are recruited in the service of goal-directed pursuits (Karoly, 1993). These processes include the management of one’s cognitions, attentional allocations, emotions, and behaviors (Karoly, 1993). Importantly, many self-regulation processes are believed to be under the voluntary control of the individual though they are not always controlled consciously (Eisenberg & Spinrad, 2004). Self-regulation processes are also psycho-social in nature, being flexibly initiated moment-to-moment within ever changing social contexts (Morf & Mischel, 2002). As such, the interpersonal context is crucial for fully understanding the functions and outcomes of self-regulation. Culture plays a prominent role in defining guidelines and expectations for what is adaptive and maladaptive within daily interpersonal exchanges. Culture therefore likely provides the foundational framework for the development and implementation of self-regulation (Yasui & Dishion, 2007). It is thus crucial to understand whether measures of self-regulation are invariant across cultures. The current study assessed the cross-cultural invariance of three self-regulation processes: attribution biases, emotion regulation, and self-efficacy beliefs about emotion regulation.

Attribution Biases from Social Information Processing

Attributions of intent require the interpretation of social cues and identification of the reasons behind someone’s activities within a social exchange. In ambiguous social situations, attributional biases often arise. Some attributional biases may be self-blaming, depressive attribution biases, and others may be other-blaming, hostile attribution biases. Concurrent and longitudinal evidence suggests that negative self-blaming attribution biases are related to depressive symptoms (e.g., Luebbe, Bell, Allwood, Swenson, & Early, 2010; Prinstein, Cheah, & Guyer, 2005), whereas hostile other-blaming attribution biases are related to externalizing symptoms (e.g., Orobio de Castro et al., 2002).

Attribution biases are universally applicable characteristics (Dodge, 2006), although much of the evidence on attribution biases and their external correlates is based on samples from the United States. A few intriguing studies have been conducted in other countries. These studies suggest that hostile attribution biases are related to aggression among German adolescents (Möller & Krahé, 2009) and Dutch children (Reijntjes, Thomaes, Kamphuis, Bushman, Orobio de Castro, & Telch, 2011). Although these studies offer support for the universality of attribution biases, they do not make cross-cultural comparisons of the equivalence of attribution biases.

Few researchers have examined attribution biases using cross-cultural designs. Dodge et al. (in press) found that children’s mean levels of hostile attribution biases varied across 12 ethnic groups and that levels of hostile attribution biases mediated the relation between cultural group and aggressive behavior. This study highlights the importance of examining hostile attribution biases across cultures, yet to our knowledge, there is no existing cross-cultural study solely investigating the psychometric invariance of either hostile or depressive attribution biases. The present study fills this gap, providing crucial psychometric evidence necessary for future cross-cultural work on both hostile and depressive attribution biases.

Emotion Regulation

Emotion-related self-regulation is a burgeoning arena for understanding the multi-faceted nature of self-regulation. Eisenberg and colleagues (e.g., Eisenberg & Spinrad, 2004) define emotion-related self-regulation as voluntary modulation of the presence, nature, strength, and extent of emotion-related experiences via activation, avoidance, suppression, and/or maintenance. Emotion-related experiences encompass emotion-focused internal and external processes, including internal sensations; physical, cognitive, and motivational states; and/or behavioral corollaries. Modulation of these emotion-related experiences occurs to achieve bio-psycho-social adjustment and personal objectives.

Emotion regulation is a broad construct comprised of specific strategies, such as emotion awareness (Halberstadt, Denham, & Dunsmore, 2001), emotional clarity (Gratz & Roemer, 2004), controlling impulsive emotional responses (Cole, Michel, & O’Donnell, 1994), controlling rumination (Caprara, 1986; Nolen-Hoeksema, 1991), and many other strategies. The current paper focuses on two types of emotion regulation: controlling impulsive emotional responses and controlling rumination. Failures in these emotion regulation strategies lead to dysregulated expressions of emotions and rumination, respectively. Dysregulated expression includes the over- or under-control of one’s emotions (Eisenberg et al., 2010). Rumination involves passively dwelling on one’s feelings and thoughts associated with negative events (Nolen-Hoeksema, Wisco, & Lyubomirsky, 2008). As we review empirical evidence on emotion regulation, we report findings on the broad construct of emotion regulation, highlighting these strategies where relevant.

There is clear empirical support for relations between emotion regulation and psychological adjustment in childhood and adolescence (see Eisenberg et al., 2010 for a review). The successful regulation of emotions has been concurrently associated with lower levels of internalizing symptoms (e.g., Yap, Allen, & Sheeber, 2007; Silk, Shaw, Forbes, Lane, & Kovacs, 2006) and both concurrently and longitudinally associated with lower levels of externalizing symptoms (e.g., Chaplin, Cole, & Zahn-Waxler, 2005; Eisenberg et al., 2009). Relations between emotion regulation and internalizing symptoms are less robust and potentially less stable across age than relations between emotion regulation and externalizing symptoms (Eisenberg et al., 2010).

Recent research differentiates specific emotions that fall under the broader construct of negative emotionality (e.g., Caprara, Di Giunta, Pastorelli, & Eisenberg, 2012; Zimmerman & Iwanski, 2014). Anger and sadness have received the greater part of this empirical attention in child and adolescent samples. There is strong evidence that failure to regulate anger is related to externalizing symptoms (e.g., Morris et al., 2011; Zeman, Shipman, & Suveg, 2002) and possibly internalizing symptoms (Eisenberg et al., 2009; Zeman et al., 2002). Failure to regulate sadness has been prospectively linked to depression (Feng et al., 2009) as well as to externalizing and internalizing symptoms (e.g.,Eisenberg et al., 2009; Folk, Zeman, Poon, & Dallaire, 2014). Some research even suggests that specific self-regulation processes are more or less effective depending on the specific emotion being examined (Rivers, Brackett, Katulak, & Salovey, 2007; Waters & Thompson, 2014). For example, hostile rumination has been associated with feelings of anger and violent and/or aggressive behaviors (Bushman, 2002; Caprara, Paciello, Gerbino, & Cugini, 2007). Likewise, research has found that depressive rumination increases feelings of sadness and consequently depression (Nolen-Hoeksema et al., 2008). Overall, research evidence highlights the importance of examining specific emotions when investigating emotion regulation because each might have a unique amalgamation of psychological outcomes.

The vast majority of research on emotion regulation has been conducted within the United States, with few notable exceptions. Rydell, Berlin, and Bohlin (2003) found significant negative relations between emotion regulation and internalizing and externalizing symptoms measured approximately 1½ years later in European samples. Moreover, a construct involved in emotion-related regulation (i.e., effortful control—the effortful, self-regulatory aspect of temperament) has been associated with externalizing symptoms among European adolescents (e.g., Hofer, Eisenberg, & Reiser, 2010; Oldehinkel, Hartman, Ferdinand, Verhulst, & Ormel, 2007). There is also some support for measurement invariance of emotion-regulation-related measures (i.e., effortful control) across three ethnic groups of children in the United States (i.e., European Americans, African Americans, and Hispanics; Sulik et al., 2010). There is at least preliminary support for the notion that emotion regulation processes are applicable and have similar outcomes across cultures. However, further research is necessary to support the use of emotion regulation related measures in preadolescent samples across multiple cultural contexts.

Self-Efficacy Beliefs About Emotion Regulation

Much of the emotion regulation research focuses on what individuals actually do or report that they would do to modulate their affective experiences, yet it is equally important to understand what individuals believe themselves to be capable of doing. Self-efficacy beliefs tap this related yet distinct aspect of self-regulation (Bandura, 2006). Self-efficacy beliefs about emotion regulation measure how well people believe they can control all aspects of an emotional experience, including exerting control over the origins and intensity of, reactions to, and consequences of one’s own emotions (Bandura, Caprara, Barbaranelli, Gerbino, & Pastorelli, 2003).

Poor self-efficacy beliefs about the regulation of negative emotions have been related to depressive feelings and delinquency (Bandura et al., 2003). Evidence suggests that self-efficacy beliefs about emotion regulation might be emotion specific. Among Italian young adults, Caprara, Di Giunta, Eisenberg, Gerbino, Pastorelli, & Tramontano (2008) found that self-efficacy beliefs about regulating anger were better predictors of irritability and aggression, whereas self-efficacy beliefs about regulating sadness were better predictors of shyness and depression.

There is some empirical support for measurement invariance of self-efficacy beliefs in dealing with anger and sadness among young adults from three different countries (i.e., Italy, United States, and Bolivia; Caprara et al., 2008). Because much of the work on self-efficacy beliefs about emotion regulation has been conducted with young adults, more research with adolescent samples from multiple countries is needed to advance our cross-cultural understanding of self-efficacy beliefs about emotion regulation.

Self-Regulation as Comprised of Attributions, Emotion Regulation, and Self-Efficacy Beliefs

The current study tests a unique model of the association between three self-regulation related mechanisms: attributions, emotion regulation and self-efficacy beliefs. Our rationale for modeling self-regulation as comprised of these three processes is two-fold. First, each process maps onto the defining components of self-regulation. Karoly (1993) posited that self-regulation includes the management of one’s cognitions, attentional allocation, emotions, and behaviors. Attribution biases pertain to one’s cognitions; emotion regulation pertains to one’s emotions and their expression (behavior); and self-efficacy beliefs pertain to one’s cognitions and behaviors. Second, empirical evidence indicates that these processes are inter-related. Emotions are related to children’s attributions of intent (Nelson & Coyne, 2009) and individuals are more prone to attribution biases when emotionally aroused (Dodge & Somberg, 1987). There is also evidence that self-efficacy beliefs about emotion regulation are related to emotion regulation. Self-efficacy beliefs about dealing with anger and sadness are positively related to choosing adaptive emotion-regulation coping strategies (Gunzenhauser et al., 2013). These inter-relations suggest these three processes might be tapping a similar construct (i.e., self-regulation). We therefore believe there is theoretical and empirical support for investigating the notion that self-regulation includes, among other processes, attributions, emotion regulation, and self-efficacy beliefs.

To our knowledge, current measures pertaining to self-regulation-related processes are compartmentalized, measuring only one facet of relevant processes for one emotion and without a specific reference to the social context surrounding the self-regulation (e.g., peer interactions). There is a need for a valid measure of multiple aspects of self-regulation that pertains to the regulation of more than just one emotion and is assessed within the context of social situations. The need for standardizing the social context when assessing self-regulation-related processes is especially important. People vary in terms of the kinds of situations they think about when reporting on their self-regulation. The use of social vignettes standardizes the context of their reporting. The self-regulation assessment literature would be enhanced by data on the measurement invariance of a tool that simultaneously taps attribution biases, emotion regulation, and self-efficacy beliefs about emotion regulation in multiple ethnic and national groups.

Current Study

Using a sample of preadolescents, this study examined the validity of a newly developed vignette-based measure of self-regulation. This measure is comprised of subscales tapping four constructs—attribution bias, dysregulated expression of emotion, rumination, and self-efficacy beliefs about emotion regulation—measured across both anger and sadness, resulting in 8 subscales total. First, we examined measurement invariance of a mono-factorial model of each subscale separately for anger self-regulation (4 subscales) and sadness self-regulation (4 subscales) across three countries and six cultural groups: Italy (two sites: Rome and Naples), United States (three ethnic groups: African American, European American, and Hispanic American), and Colombia. Evidence reviewed above suggests that the constructs tapped by this measure are likely applicable across cultures. We therefore predicted similarities in the structure (if not the intercepts) across the six groups. We further examined the latent structure of this measure using an aggregated total sample (i.e., collapsing across the six cultural groups). We tested two competing models separately for anger and sadness (four models total). In Model 1, items designed to assess attribution bias, rumination, dysregulated expression of emotion, and self-efficacy beliefs in dealing with emotions loaded on separate and related first-order factors. In Model 2, the first-order factors for dysregulated expression and rumination loaded on a second-order factor for emotion dysregulation. Model 2 accounted for the fact that dysregulated expression of emotions and rumination are both forms of the broader construct of emotion dysregulation.

Finally, to assess convergent validity, we examined the extent to which the eight subscales were related to empirically established measures of similar constructs. We expected positive correlations between our subscales and existing measures of similar constructs. To further evaluate convergent validity, we examined correlations between our eight subscales and mental health symptoms. We expected hostile attribution bias, dysregulated expression of anger, and hostile rumination to be positively related, and self-efficacy beliefs about anger regulation to be negatively related, to externalizing problems. We also expected depressive attribution biases, dysregulated expression of sadness and depressive rumination to be positively related, and self-efficacy beliefs about sadness regulation to be negatively related, to internalizing problems.

Method

Participants

Participants were recruited from the longitudinal blinded for review Study (e.g., blinded for review). An overarching goal was to include cultural groups that were diverse on several socio-demographic dimensions, including predominant ethnicity, predominant religion, economic indicators, and indices of child well-being. For example, on the Human Development Index, a composite indicator of a country’s status with respect to health, education, and income, participating countries had a rank of 5 for United States, 28 for Italy, and 98 for Colombia, out of 187 countries with available data (UNDP, 2012). However, it is important to note that our samples were convenience samples and were therefore not nationally representative.

Data for the present study included measures administered four years after initial recruitment into the study, at which time 87% of the original sample provided data. Participants included 541 children (age range = 10 to 14 years, M = 12.57, SD = .69; 50% female) and their mothers (n = 541). Families were drawn from Medellín, Colombia (n = 88); Naples, Italy (n = 90); Rome, Italy (n = 100);¹ and Durham, North Carolina, United States (n = 97 European Americans, n = 90 African Americans, n = 76 Hispanic Americans). Participants were recruited through letters sent from schools. To ensure economic diversity, we included students from private and public schools and from high to low income families, sampled in proportions representative of each site. Nearly all adult respondents were biological mothers, with 3% being grandmothers, stepmothers or other female adults. Overall, maternal reporters had 12 years of education (top section of Table 1).

Table 1.

Sample Descriptives and Descriptive Statistics by Site for Anger and Sadness Self-Regulation Subscales

	Italy - Naples (n = 90)	Italy - Rome (n = 100)	U.S.- European- American (n = 98)	U.S. - African- American (n = 90)	U.S. – Hispanic- American (n = 76)	Colombia (n = 88)
Maternal Years of Education	10.60 (4.51)	13.50 (3.93)	16.71 (2.99)	13.68 (2.26)	10.38 (3.85)	10.29 (5.12)
Females	56%	47%	41%	53%	50%	53%
Youth Age	12.36 (.49)	12.34 (.77)	12.88 (.57)	12.82 (.63)	12.73 (.75)	12.30 (.63)
Hostile attribution bias	3.34 (.76) [.73]	3.17 (.57) [.48]	3.08 (.58) [.69]	3.36 (.69) [.60]	3.32 (.60) [.56]	3.49 (.66) [ .57]
Self-efficacy beliefs about anger	3.35 (.69) [.73]	3.14 (.67) [.74]	3.57 (.70) [.79]	3.44 (.75) [.69]	3.40 (.74) [.80]	3.55 (.66) [.75]
Dysregulated anger expression	1.65 (.83) [.89]	1.65 (.77) [.85]	1.85 (.77) [ .83]	2.36 (.96) [.81]	2.15 (.94) [ .88]	1.35 (.51) [.69]
Hostile rumination	3.07 (.88) [.76]	2.83 (.98) [.86]	2.46 (.88) [.86]	2.85 (.87) [.76]	2.82 (.90) [.85]	3.04 (1.03) [.86]
Depressive attribution bias	2.84 (.72) [.73]	2.81 (.65) [.69]	2.70 (.61) [.75]	2.94 (.72) [.69]	2.89 (.62) [.66]	3.01 (.76) [.75]
Self-efficacy beliefs about sadness	3.57 (.66) [.73]	3.57 (.68) [.77]	3.85 (.66) [.79]	3.79 (.75) [.71]	3.72 (.66) [ .75]	3.70 (.61) [.74]
Dysregulated sadness expression	1.72 (.61) [.70]	1.66 (.61) [.79]	1.68 (.58) [.77]	1.72 (.74) [.80]	1.76 (.65) [.79]	1.91 (.85) [.80]
Depressive rumination	2.95 (.93) [.81]	2.80 (.98) [.85]	2.35 (.88) [.86]	2.51 (.83) [.70]	2.61 (.87) [.81]	2.95 (1.04) [.84]

Note. Standard deviations are in parentheses and α’s are in brackets.

Measures

Primary measure being validated: Anger and sadness self-regulation scale

The current study sought to validate a measure we created to simultaneously assess attribution biases, emotion dysregulation, and self-efficacy beliefs about emotion regulation. We created stories that present youth with ambiguous social situations involving peer interactions because of the salience of peers at this age (Prinstein et al., 2005) and the importance of situational-specificity in assessing self-regulation-related processes (Crick & Dodge, 1994).

Measure development and pilot testing

We used a three-stage process (Gould, 1996) to create vignettes. First, we derived our scenarios from existing social-information processing vignettes that have extensive empirical support (Crick & Dodge, 1996). Second, a panel of six experts on attributions, emotion regulation, and self-efficacy beliefs reviewed the stories, offering feedback and revisions. Finally, stories were pilot tested with five children from the United States, seven children from Rome, and three children from Naples.² Pilot participants offered feedback on (1) feasibility and believability of scenarios, (2) clarity of questions asked, and (3) factors that may have affected their responses.

Final measure

The final measure included six vignettes. For each vignette, youths answered questions about (1) why the peers acted the way they did (i.e., attribution biases; “How likely is it that the kids acted that way because they wanted to be mean to you?”), (2) how likely it was that they would engage in certain behaviors pertaining to dysregulated emotion expression (e.g., “You would angrily yell, say mean things, or hit, kick or slam something.”) and rumination (e.g., “You would keep thinking and thinking what went wrong and how angry you feel.”), and (3) how well they could deal with their emotion (e.g., “If you were in this situation, how well could you deal with your anger?”). Youths answered the same set of questions for both sadness and anger across all vignettes (counterbalanced across stories). Response options ranged from 1 (not at all likely; not at all sad/angry; not at all well) to 5 (very likely; very sad/angry; very well). This measure has eight subscales: (1) hostile attribution bias, (2) dysregulated expression of anger, (3) hostile rumination, (4) self-efficacy beliefs about anger regulation, (5) depressive attribution bias, (6) dysregulated expression of sadness, (7) depressive rumination, and (8) self-efficacy beliefs about sadness regulation. Total scores for each subscale were created by averaging item responses across the six vignettes. With few exceptions, reliability was adequate to very good across all samples (see bottom section of Table 1).

Convergent constructs and measures

Hostile attribution bias

Youth were read ten vignette-based stories portraying hypothetical ambiguous social situations in which a peer provokes him/her (see blinded for review for more details). Youth indicated whether the peer’s behavior was an accident (coded as 0) or done on purpose (coded as 1). This measure was administered 1 year prior to the current study and includes several of the same stories as those in the measure we are validating (M α across the 6 sites =.63).

Dysregulated expression of anger and sadness

Youths rated (1 = hardly ever to 3 = often) six items from the Children’s Anger Management Scale (CAMS; Zeman et al., 2002; dysregulated expression of anger and anger coping subscales; e.g., “I attack whatever it is that makes me very angry” and “I stay calm and keep my cool when I am feeling mad”). Children also rated six items from the Children’s Sadness Management Scale (CSMS; Zeman et al., 2002; dysregulated expression of sadness and sadness coping subscales; e.g., “I whine/fuss about what is making me sad” and “I can stop myself from losing control of my sad feelings”). One CSMS item (i.e., “I do things like mope around when I am sad”) was dropped only in Colombia due to a very low item-scale correlation (i.e., r = −.04). Total scores comprise the mean of the dysregulated expression and coping (reverse scored) subscales separately for anger and sadness. Mean reliability across the six sites was α = .66 for anger and α =.55 for sadness. Given the low reliability for sadness, we conducted a factor analysis and found that each of the indicators loaded on the factor of dysregulated expression of sadness at greater than the recommended .30 cutoff (values ranged from .48 − .56; Tabachnick & Fidell, 2001), suggesting these items are interrelated. We suspect this low reliability is the result of limited sampling of the content domain which introduces random measurement error. Random measurement error does not lead to spuriously significant correlational results but rather attenuates them (Cohen, Cohen, West, & Aiken, 2003).

Youths also rated (1= almost always untrue of you to 5 = almost always true of you) subscales from the Early Adolescent Temperament Questionnaire-Revised (EATQ-R; Capaldi & Rothbart, 1992) about their irritability (11 items; e.g., “I get irritated when I have to stop doing something that I am enjoying”; mean α across the six sites was α = .79) and sadness (9 items; e.g., “I feel depressed when unable to accomplish some task”; mean α across the six sites was α = .78).

Depressive rumination

(Children’s Response Styles Questionnaire; Abela, Brozina, & Haigh, 2002). Youths rated (1= almost never to 4=almost always) 13 items assessing the extent to which they respond to sad feelings with rumination (M α across the six sites = .86).

Hostile rumination

(Caprara, 1986). Youths rated (1= completely true for me to 6= completely false for me) ten items tapping the extent to which they respond to angry feelings with rumination (M α across the six sites = .78).

Self-efficacy in dealing with anger and sadness

(Regulatory Emotional Self-Efficacy Scale; Caprara & Gerbino, 2001; Caprara et al., 2008). Youths rated their ability to deal with anger and sadness across four and five items, respectively. Response options ranged from 1= not well at all to 5=very well. Mean reliability across the six sites was α =.75 for anger and α =.78 for sadness.

Externalizing and internalizing symptoms

Mothers completed 33 items from the externalizing scale and 31 items from the internalizing scale of the Child Behavior Checklist (CBCL; Achenbach, 1991). Youths completed 30 items from the externalizing scale and 29 items from the internalizing scale of the Youth Self-Report (YSR; Achenbach, 1991). Items were rated on a three-point scale of 0 (not true) to 2 (very true or often true). Mean reliability across the six sites was α = .87 and .84 for youth-reported internalizing and externalizing symptoms, respectively, and α = .87 and .89 for mother-reported internalizing and externalizing symptoms, respectively.

Procedure

Measures were administered in the predominant language at each site, following forward- and back-translation and meetings to resolve any item-by-item ambiguities in linguistic or semantic content (Erkut, 2010). Forward- and back-translation was used to ensure the linguistic and conceptual equivalence of measures across languages (see Maxwell, 1996). Translators were fluent in English and the target language. Translators also noted items that did not translate well, were inappropriate for participants, were culturally insensitive, or elicited multiple meanings and suggested improvements. Site coordinators and translators reviewed discrepant items and made appropriate modifications. Measures were administered in Spanish (Colombia and United States Hispanics), Italian (Italy), and American English (United States).

Interviews (1.5 to 2 hours) were conducted in participants’ homes, schools, or other private locations. Procedures were approved by local Institutional Review Boards at universities in each country. Mothers and children provided consent and assent, respectively, and were interviewed separately to ensure privacy. Mothers were given the option to have questionnaires administered orally (with rating scales provided as visual aids) or complete them on their own. All children completed the questionnaires orally, with questions read and responses recorded by trained interviewers. Families were paid a modest amount for their participation.

Analytic Approach

Multi-group Confirmatory Factor Analyses (CFA) were run in Mplus 3.0 (Muthén & Muthén, 1998–2007) to test our first hypothesis of measurement invariance across cultures for the mono-factorial solution of each of the eight subscales in our measure. We adopted a model-fitting process based on Vandenberg and Lance (2000) as well as others (e.g., Chan, 2000). We tested three models: configural invariance (constraining the pattern of fixed and free factor loadings across all groups), metric invariance (constraining equality of all factor loadings across all groups), and scalar invariance (constraining equality of all intercepts of like items’ regressions on the latent variables across all groups). The most frequent additional tests were of partial invariance at each step, and modification indices (MI) from each step were used to refine the structural models (Steenkamp & Baumgartner, 1998). Non-invariant items were identified by inspecting MIs on a one-by-one basis (Muthén & Muthén, 2006). Each form of invariance was nested in the previous model and involved added constraints at each step that built upon previous constraints. The logic was that invariance restrictions may hold for some but not all manifest measures across populations, and relaxing invariance constraints where they do not hold controls for partial measurement inequivalence (Vandenberg & Lance, 2000). We performed Chi-square difference tests to compare nested models (Kline, 1998). We focused on model fit indices that are less sensitive to sample size given that obtaining a nonsignificant χ² becomes increasingly unlikely with large samples (Kline, 1998). We accepted CFI values >.90 (Kelloway, 1998; Kline, 1998), RMSEA values <.07 (Browne & Cudeck, 1993), and SRMR values <.08 (Kelloway, 1998). For the RMSEA, a nonsignificant p-value means the hypothesized model is a good approximation of the population.

Based on a common rule of thumb for determining the minimum sample size needed per group for CFA (i.e., ratio of N to the number of variables in a model (p), N/p ≥ 10; Mundfrom, Shaw, & Ke, 2005), we could not examine measurement invariance of multi-factorial models. Our smallest sample size was 76, which resulted in a value of 3.17 (i.e., 76/[6 vignettes*4 scales]) which was substantially less than the necessary value of ten or greater.

We used CFA to test our second hypothesis on the latent structure of multiple distinct self-regulation-related processes tapped by our measure (separately for anger and sadness) using the aggregated total sample. In the first model, attribution bias, rumination, dysregulated expression, and self-efficacy beliefs were considered separate and correlated constructs at the first level. For example regarding anger, six hostile attribution bias questions from the six stories loaded on the first order factor of hostile attribution bias; six hostile rumination questions from the six stories loaded on the first order factor of hostile rumination; and so on for all hypothesized first order anger-related factors. In this model, it was also hypothesized that these first order factors were correlated with each other. In the second model, rumination and dysregulated expression were modeled as expressions of a second-order latent factor reflecting emotion dysregulation, whereas attribution bias and self-efficacy beliefs were modeled as expressions of two first-order factors correlated with the second-order factor (i.e., emotion dysregulation). Self-efficacy belief scores were reverse-coded (i.e., the latent factor refers to lack of self-efficacy beliefs) so that all factors in both models were positively associated. To test our two competing models, we examined model fit indices reported above as well as Akaike Information Criterion (AIC) (a helpful index when comparing non-nested models; Tabachnick & Fidell, 2001) and Bayesian Information Criterion (BIC). Finally, we used correlations to test our third hypothesis regarding convergent validity.

Results

Except for Dysregulated Expression of Anger (DEA), all skewness and kurtosis values across the six stories for each subscale for each cultural group were within normal ranges (skewness < 2 and kurtosis < 7; Curran, West, & Finch, 1996). We used maximum likelihood (ML) as the estimator for all variables except DEA, for which we used Robust Maximum Likelihood (MLR).

Measurement Invariance of Mono-Factorial Models across Six Cultural Groups

Independent multi-group CFAs were run to test measurement invariance across cultures for each of the subscales (i.e., as mono-factorial models), separately for anger and sadness (8 models total). We discuss results separately by subscale (see Table 2 for complete details).

Table 2.

Fit Indices for Multi-Group CFAs Testing Measurement Invariance of the Mono-Factorial Structure of Anger and Sadness Self-Regulation Subscales

		χ2	df	p	CFI	RMSEA (CI)	Δχ2 g	Δdf	p
Hostile attribution bias	CIa	64.25	52	.12	.96	.05 (.00−.08)
	MIa	94.05	77	.09	.94	.04(.00−.08)	29.80	25	0.23
	SIa	176.57	102	.00	.75	.09(.06−.11)	82.52	25	0.00
	PSIa	120.18	98	.06	.93	.05 (.00−.07)	26.13	21	0.20
Self-efficacy about anger regulation	CIb	73.61	52	.03	.97	.06(.02−.10)
	MIb	94.53	77	.09	.97	.05 (.00−.08)	20.92	25	0.69
	SIb	153.02	102	.00	.92	.07(.04−.09)	58.49	25	0.00
	PSIb	125.77	99	.04	.96	.05 (.01−.08)	31.24	22	0.09
Dysregulated expression of anger	CIc	70.71h	53	.05	.97	.06 (.00−.09)
	MIc	98.49i	81	.09	.98	.04(.00−.08)	48.50	31	0.02
	PMIc	84.74j	79	.31	.99	.03(.00−.06)	36.39	29	0.16
	PSIc	111.01k	102	.25	.99	.03(.00−.06)	28.09	21	0.14
Hostile rumination	CI	87.06	54	.01	.97	.08(.05−.11)
	MI	115.61	79	.01	.97	.07 (.04−.10)	28.54	25	0.28
	SI	180.95	104	.00	.93	.09 (.07−.11)	65.34	25	0.00
	PSI	149.81	101	.00	.95	.07 (.05−.10)	34.21	22	0.06
Depressive attribution bias	CId	61.14	50	.13	.98	.05(.00−.09)
	MId	100.48	75	.03	.95	.06(.02−.09)	39.34	25	0.03
	PMId	89.30	74	.11	.97	.05(.00−.08)	28.16	24	0.25
	PSId	117.97	94	.05	.96	.05(.01−.08)	28.67	20	0.09
Self-efficacy about sadness regulation	CI	74.98	54	.03	.97	.07(.02−.10)
	MI	91.221	74	.09	.97	.05(.00−.08)	16.24	20	0.70
	SI	136.29	99	.01	.94	.07(.03−.09)	45.07	25	0.01
	PSI	125.06	97	.03	.95	.06(.02−.08)	33.84	23	0.07
Dysregulated expression of sadness	CIe	71.68	52	.04	.98	.07(.02−.10)
	MIe	137.27	77	.00	.93	.09(.07−.12)	65.59	25	0.00
	PMIe	102.45	74	.02	.97	.06(.02−.09)	30.77	22	0.10
	PSIe	121.67	96	.04	.97	.05(.01−.08)	19.22	22	0.63
Depressive rumination	CIf	75.63	53	.02	.98	.07(.03−.10)
	MIf	120.14	78	.00	.96	.08(.05−.10)	44.51	25	0.00
	PMIf	109.40	77	.01	.97	.07(.03−.09)	33.77	24	0.08
	PSIf	138.70	100	.011	.96	.06(.03−.09)	18.56	22	0.23

Note: CI=configural invariance; MI=metric invariance; PMI=partial metric invariance; SI=scalar invariance; PSI=partial scalar invariance. Superscripts ^a-f indicate which covariations between errors were estimated:

^astories 2 and 3 in Rome and stories 1 and 3 in US African Americans;

^bstories 5 and 6 in Rome and stories 2 and 3 in Colombia;

^cstories 1 and 4 in US European American and stories 2 and 5 in US African American ;

^dstories 1 and 3 in Rome and US Hispanic, and stories 5 and 6 in US European American and US African American;

^estories 2 and 5 in US European American and US African American;

^fstories 1 and 3 in US European American;

^gonly for dysregulated expression of anger, when conducting χ2 difference tests using MLR we adjusted the χ2 using the Satorra-Bentler scaling correction.

^hscaling correction factor for MLR = 1.43,

ⁱscaling correction factor for MLR =1.65,

^jscaling correction factor for MLR = 1.65,

^kscaling correction factor for MLR =1.52

Hostile attribution bias (HAB)

We found full metric invariance across the six groups. We also found partial scalar invariance: intercepts for story 2 in Rome and US African Americans and stories 3 and 5 in US European Americans were relaxed to differ from the other groups.

Self-efficacy beliefs about anger regulation (SEAR)

We found full metric invariance across the six groups. We also found partial scalar invariance: intercepts for stories 3 and 4 in US European Americans and story 6 in Colombia were relaxed to differ from the other groups.

Dysregulated expression of anger (DEA)

We found partial metric invariance across the six groups: factor loadings for story 4 in Rome and story 6 in US Hispanic Americans were relaxed to differ from the other groups. We also found partial scalar invariance: intercepts were relaxed for stories for which factor loadings were already relaxed.

Hostile rumination (HR)

We found full metric invariance across the six groups. We also found partial scalar invariance: intercepts for story 4 in Rome, story 1 in US European Americans, and story 6 in Colombia were relaxed to differ from the other groups.

Depressive attribution bias (DAB)

We found partial metric invariance across the six groups: the factor loading for story 5 in US European Americans was relaxed to differ from other groups. We also found partial scalar invariance: intercepts for story 5 in US European Americans; story 3 in Naples, US European Americans, and Colombia; and story 2 in US African Americans were relaxed to differ from other groups.

Self-efficacy beliefs about sadness regulation (SESR)

We found full metric invariance across the six groups. We also found partial scalar invariance: intercepts for story 2 in Rome and Naples were relaxed to differ from other groups.

Dysregulated expression of sadness (DES)

We found partial metric invariance across the six groups: factor loadings for story 4 in US European Americans, story 6 in US African Americans, and story 2 in Colombia were relaxed to differ from other groups. We also found partial scalar invariance: intercepts were relaxed for stories for which factor loadings were already relaxed.

Depressive rumination (DR)

We found partial metric invariance across the six groups: the factor loading for story 4 in Rome was relaxed to differ from other groups. We also found partial scalar invariance: intercepts for story 4 in Rome and story 6 in US European Americans were relaxed to differ from other groups.

Independent multi-group CFAs indicated that the mono-factorial structure of our anger and sadness self-regulation subscales generally replicated across the six cultural groups. It is important to note that for all examined models modification indices suggested estimation of covariations between story errors for some of the groups. We estimated five covariations between errors when we had substantive reasons in support of such covariation based on similarities in vignette content.³

Latent Structure of Multiple Distinct Self-Regulation-Related Processes (Total Sample)

We tested two alternative models using CFA to examine the latent structure of multiple distinct self-regulation related processes (i.e., subscales) separately for anger and sadness (4 models total). For these analyses, we used the aggregate sample (N = 541). In Model 1, the four subscales were considered as separate and correlated constructs at the first level. Results for anger self-regulation were χ² (226; N = 541) = 405.29; p < .001; CFI = .95; RMSEA = .04 (.03 − .04); AIC = 36329.94; BIC = 36751.05) and for sadness self-regulation: χ² (226; N = 541) = 406.68; p < .001; CFI = .95; RMSEA = .04 (.03 − .04); AIC = 35469.13; BIC = 35890.25). In Model 2, rumination and dysregulated expression were modeled as expressions of a second-order latent factor reflecting emotion dysregulation, whereas attribution bias and self-efficacy beliefs were modeled as expressions of two first-order factors correlated with the second order factor⁴. Results for anger self-regulation were χ² (228; N = 541) = 414.18; p < .001; CFI = .94; RMSEA = .04 (.03 − .04); AIC = 36336.77; BIC = 36749.30) and for sadness self-regulation: χ² (228; N = 541) = 415.82; p < .001; CFI = .95; RMSEA = .04 (.03 − .04); AIC = 35474.27; BIC =35886.79). For both anger and sadness, both models provided good fit to the data. Given these findings, rumination and dysregulated expression of both anger and sadness may be viewed as separate and correlated factors (i.e., Model 1) as well as expressions of a second-order factor reflecting emotion dysregulation (i.e., Model 2).

Given the small differences between model fit indices, we provide some guidance regarding when one model might be preferred over the other based on model selection goals associated with AIC and BIC. If our scale is used to predict mental and behavioral health outcomes, the model selection goals associated with AIC (minimizing Type II errors; Dziak et al., 2012) would support Model 1 as representative of the structure of our measure (Figures 1 and 2). If the focus of model selection is on choosing the most parsimonious model and both Type I and Type II errors are considered equally adverse, the model selection goals associated with BIC (Dziak et al., 2012) would support Model 2 as representative of the structure of our measure (Figures 3 and 4).

Figure 1.

Path Diagram of the Four Oblique Factor Model for the Anger Self-Regulation Scale in the Overall Sample. S= story; each square box refers to the raw score of a specific item within a specific story (e.g., 1/S1 refers to item 1 in story 1). Standardized factor loadings are shown on the arrows. Factor loading coefficients in relation to the story factors are not reported because of limited space. All story factors were related to each other.

Figure 2.

Path Diagram of the Four Oblique Factor Model for the Sadness Self-Regulation Scale in the Overall Sample. S= story; each square box refers to the raw score of a specific item within a specific story (e.g., 1/S1 refers to item 1 in story 1). Standardized factor loadings are shown on the arrows. Factor loading coefficients for story factors are not reported because of limited space. All story factors were related to each other.

Figure 3.

Path Diagram of the Three Oblique Factor Model for the Anger Self-Regulation Scale in the Overall Sample. S= story; each square box refers to the raw score of a specific item within a specific story (e.g., 1/S1 refers to item 1 in story 1). Standardized factor loadings are shown on the arrows. Factor loading coefficients in relation to the story factors are not reported because of limited space. All story factors were related to each other.

Figure 4.

Path Diagram of the Three Oblique Factor Model for the Sadness Self-Regulation Scale in the Overall Sample. S= story; each square box refers to the raw score of a specific item within a specific story (e.g., 1/S1 refers to item 1 in story 1). Standardized factor loadings are shown on the arrows. Factor loading coefficients in relation to the story factors are not reported because of limited space. All story factors were related to each other.

Convergent Validity of Anger and Sadness Self-Regulation Subscales

Using a Bonferonni adjustment to control for Type I errors, we consider only correlations significant at the .001 level (bolded values in Table 3). Regarding anger, significant correlations held across the six groups for self-efficacy beliefs about anger regulation (SEAR) and dysregulated expression of anger (DEA) and across four groups for hostile rumination (HR). Among the entire sample, hostile attribution bias (HAB) was significantly related to mother-reported externalizing symptoms; HR and SEAR were significantly related to child-reported externalizing symptoms; and DEA was significantly related to both. Regarding sadness, significant correlations held across five groups for depressive rumination (DR), across four groups for self-efficacy beliefs about sadness regulation (SESR), and across three groups for dysregulated expression of sadness (DES). Among the entire sample, DAB, DR, and SESR were significantly related to only child-reported internalizing symptoms, whereas DES was significantly related to both mother- and child-reported internalizing symptoms.

Table 3.

Convergent Validity: Correlations Between Anger and Sadness Self-Regulation Subscales and Related Constructs

Correlated Variables	Italy - Naples (n=90)	Italy - Rome (n=100)	U.S.- European- American (n=98)	U.S.- African- American (n=90)	U.S.- Hispanic American (n=76)	Colombia (n=88)	Externalizing problems (n=541) Child-report (mother report)
HAB with HAB 1 year prior	.12	.14	.31**	.20	.38**	−.04	.11** (.18**)
SEAR with anger on RESE	.58**	.69**	.66**	.37**	.52**	.54**	−.38**(−.12**)
DEA with CAMS	.54**	.44**	.52**	.51**	.62**	.47**	.60** (.27**)
DEA with irritability on EATQ-R	.38**	.34**	.44**	.40**	.47**	.21*	.48** (.23**)
HR with HRS	.41**	.19*	.51**	.37**	.38**	.08	.15**(.08)
							Internalizing problems (n=541) Child-report(mother report)
DAB with internalizing on YSR and CBCL	.30** (.05)	.26** (.17)	.19 (−.01)	.14 (−.03)	.10 (.01)	.09 (−.01)	.17** (.03)
SESR with sadness on RESE	.34**	.43**	.60**	.29**	.40**	.27**	−.21**(−.08)
DES with CSMS	.38**	.39**	.43**	.27**	.30**	.20	.39**(.22**)
DES with sadness on EATQ-R	.37**	.28**	.38**	.23*	.33**	.27**	.64**(.25**)
DR with DRS	.38**	.32**	.41**	.35**	.44**	.33**	.31** (.11*)

Note: The first variable in the correlated variable column is from the Anger and Sadness Self-Regulation measure being validated in the current study. The second variable in the correlated variable column is from convergent measures. HAB=Hostile Attribution Bias; DEA=Dysregulated Expression of Anger; SEAR=Self-Efficacy beliefs about Anger Regulation; RESE =Regulatory Emotional Self-Efficacy Scale; CAMS=Children’s Anger Management Scale; EATQ-R=Early Adolescent Temperament Questionnaire-Revised; HR=Hostile Rumination; HRS =Hostile Rumination Scale; DAB=Depressive Attribution Bias; YSR=Youth Self Report; CBCL=Child Behavior Checklist; SESR=Self-Efficacy beliefs about Sadness Regulation; DES=Dysregulated Expression of Sadness; CSMS=Children’s Sadness Management Scale; DR=Depressive Rumination; DRS=Depressive Rumination Scale;

^*p≤.05

^**p≤.01; bolded values indicate significant correlations after Bonferonni correction.

Discussion

The current study evaluated the internal and convergent validity of a vignette-based scale measuring four self-regulation related processes: attribution biases, dysregulated expression of emotions, rumination, and self-efficacy beliefs about emotion regulation. We examined these processes separately for anger and sadness among preadolescents from three countries and six cultural groups. Participants came from cultural groups and countries that varied on socio-demographic indicators, which provided a good initial test of the external validity of the scale’s psychometric properties.

Overall, results were generally consistent with our first hypothesis that the internal structure of our measure would be similar across cultures. Full metric invariance held for all but one of our anger self-regulation subscales—dysregulated expression. Dysregulated expression of anger demonstrated partial metric invariance with half of the factor loadings invariant across the six cultural groups. We found partial scalar invariance for all anger self-regulation scales; however, the majority of story intercepts were invariant across the six groups. Regarding our sadness self-regulation subscales, we found partial support for measurement invariance. Full metric invariance held for one of the four subscales (i.e., self-efficacy beliefs about sadness regulation). Partial metric invariance held for the remaining three subscales although factor loadings were invariant for the majority of the six stories. Partial scalar invariance was evident for all sadness self-regulation subscales; however, the majority of story intercepts were invariant across the six groups.

Overall, story factor loadings and intercepts that needed to be freely estimated across groups were not consistent. For both anger and sadness self-regulation subscales, factor mean comparisons across the six groups may still be regarded as meaningful given that at least one story intercept, in addition to the anchor one, was invariant (Steenkamp & Baumgartner, 1998). Nonetheless, two caveats are warranted. There is no consensus on a minimum level of partial invariance (i.e., the amount of freely estimated parameters) that is required for meaningful factor mean comparisons (Millsap & Kwok, 2004; Widaman & Reise, 1997). Furthermore, measurement invariance results may change depending on which item (or story in our case) is chosen as the anchor one (Vandenberg, 2002). The data on scalar invariance suggests that the constructs related to anger and sadness self-regulation do not have exactly the same meaning across our six cultural groups. This is not entirely surprising given that the cultures examined in this study likely differ on important characteristics, such as social norms, that influence the way one perceives, interprets, manages, and reacts to social exchanges. Caution must, therefore, be exercised when comparing mean values of the subscales (or stories) across groups.

Prior research on hostile attribution bias (Dodge et al., in press) indicates that mean levels vary across cultures. Our study adds to this growing literature base by providing some evidence of measurement invariance for hostile and depressive attribution biases across cultures, with stronger support for hostile attribution biases. To our knowledge, this is the first study to include evidence on the metric invariance of attribution biases. Moreover, our results offer preliminary support for making mean comparisons of both hostile and depressive attribution biases in preadolescence across multiple culturally different contexts. Similar to the attribution biases literature, evidence on the measurement invariance of emotion regulation is lacking. Prior work on effortful control (which is highly related to emotion regulation) indicates that its internal structure is consistent across three ethnic groups of low-income preschoolers in the United States (Sulik et al., 2010). Our findings, which spanned more diverse and older samples of youth, were generally consistent with this prior work. This study offers preliminary support for making mean comparisons of rumination and dysregulated expression of both anger and sadness in preadolescence across multiple culturally diverse contexts. Self-efficacy beliefs about emotion regulation have a much larger evidence base in support of measurement invariance (Caprara et al., 2008; 2012), and our results are consistent with this prior work. Our findings suggest that measurement invariance of self-efficacy beliefs about emotion regulation holds across diverse ages and cultures.

Using the total sample, we found support for the latent structure of multiple distinct self-regulation-related processes tapped by our self-regulation subscales. Both competing models fit the data well. Model choice will therefore depend on model selection goals and how the measure will be used. A four-factor oblique model, where subscales were represented on separate factors, appears appropriate (based on AIC) when the measure will be used to predict mental health and behavioral outcomes and goals are to minimize Type II errors (Dziak et al., 2012). A three-factor oblique model, where dysregulated expression and rumination loaded on a second-order factor (i.e., emotion dysregulation) and attribution bias and self-efficacy beliefs were modeled as expressions of two first-order factors, appears appropriate (based on BIC) when the goal is to select the most parsimonious model and minimize both Type I and Type II errors (Dziak et al., 2012).

Factors within both the four- and three-factor oblique models were positively and moderately associated with each other, except for attribution bias and lack of self-efficacy beliefs about emotion regulation. Correlational patterns between factors were generally similar for anger and sadness regulation-related constructs. One notable difference occurred in the four-factor model: whereas the correlation between latent factors for hostile rumination and dysregulated expression of anger was weak (r = .19), the correlation between latent factors for depressive rumination and dysregulated expression of sadness was strong (r = .52). This suggests that when youth are in a peer context that makes them feel angry, if they repetitively think about what went wrong and how angry they feel, they do not necessarily act upon those feelings (e.g., yell or slam something). However, when youth are in a peer context that makes them feel sad, if they repetitively think about what went wrong and how sad they feel, they are much more likely to act upon those feelings (e.g., make a sad face or cry). It is important to note that we cannot rule out the possibility that such correlations may exist in specific cultural groups but not in others. More studies are needed to determine whether these models replicate and to fully interpret correlations between these constructs. Studies using samples from more cultural groups, samples that are more demographically diverse (e.g., SES), and clinical samples would improve our understanding of the psychometric properties and appropriate applications of this measure.

Our vignette-based measure demonstrated good preliminary evidence of convergent validity, generally correlating with correspondent measures assessed via questionnaire. Dysregulated expression of anger and self-efficacy beliefs about anger regulation were strongly and positively associated with correspondent measures in all six cultural groups, whereas hostile rumination was moderately and positively associated with correspondent measures in four of the six cultural groups. We found strongest evidence of validity for dysregulated expression of anger, which was strongly related to both mother- and child-reported externalizing behaviors. Hostile attribution bias was strongly related only to mother-reported externalizing behaviors. Hostile rumination and self-efficacy beliefs about anger regulation were strongly related to only child-reported externalizing behaviors. This pattern of findings may have emerged because both children and parents are equally accurate reporters of overt failures in anger self-regulation (e.g., dysregulated expression), whereas children are more accurate reporters of internalized self-regulation processes (e.g., rumination) (e.g., De Los Reyes & Kazdin, 2003).

Depressive rumination was most consistently correlated with theoretically related questionnaires across groups. Dysregulated expression of sadness and self-efficacy beliefs about sadness regulation were significantly correlated with theoretically related questionnaires across at least half of the groups. Some of the findings for dysregulated expression of sadness may have been attenuated due to low reliability. We did not find evidence of construct validity for depressive attribution bias for any of the six groups. Construct validity for our sadness self-regulation subscales was strongest among participants of European descent (i.e., both Italian samples and U.S. European Americans). In the entire sample, all four sadness self-regulation subscales were strongly related to child- but not mother-reported internalizing symptoms. The lack of association with mother-reported internalizing problems was not surprising considering that previous studies suggest that youth are more accurate reporters of internalizing problems (De Los Reyes & Kazdin, 2003).

Hostile and depressive attribution biases were the only subscales with very few significant correlations with convergent measures across the six groups. The lack of significant correlations is likely due to limitations in our convergent measures; for hostile attribution bias the convergent measure was assessed one year prior and for depressive attribution bias the measure covered a broader range of characteristics beyond attribution biases. Nonetheless, among the entire sample, attribution biases were significantly related to concurrent levels of externalizing and internalizing symptoms, offering general evidence of their construct validity.

Limitations and Future Directions

The relatively small sample sizes within cultural groups are an important limitation to our findings. Moreover, our findings are based on youth from only three countries. Empirical inferences and conclusions about the cross-cultural validation of subscales within our sadness and anger self-regulation measures may vary when larger and more diverse cross-cultural samples are used. Further research is needed to collect data from larger and more culturally diverse samples to more completely establish the validity of our measure.

Latent structural models were tested within our total sample that aggregated across cultural groups because sample sizes within each group were too small relative to the number of parameters estimated. We found partial support for measurement invariance (i.e., equal structure, partially equal loadings, partially equal intercepts, but unequal error variances and covariances) of each factor related to each emotion across the six cultural groups. That may lead one to expect that the hypothesized latent structural models might be invariant within each cultural group. However, we cannot infer from measurement invariance tests whether correlations between multiple factors are invariant across groups. Future studies with larger sample sizes within cultural groups could examine the measurement invariance of our multi-factorial models separately by group status.

Conclusions

This study provides crucial psychometric evidence to support cross-cultural examinations of self-regulation mechanisms associated with discrete negative emotions. We examined multiple mechanisms reflecting independent and related aspects of anger and sadness regulation among participants from a variety of cultural contexts. Overall, our findings provide preliminary support for partial measurement invariance and construct validity of a novel tool measuring anger and sadness self-regulation mechanisms in preadolescence in six diverse cultural contexts. This assessment instrument is a tool that can be used in future studies examining the complex relations among emotion-specific self-regulation-related processes and psychological adjustment among youth in cross-cultural contexts.