Assessment of Anxiety Symptoms in School Children: A Cross-Sex and Ethnic Examination

Abstract

We evaluated the cross-sex and -ethnic (Hispanic/ Latino, non-Hispanic White) measurement invariance of anxiety symptoms based on the Spence Children’s Anxiety Scale (SCAS) as well as SCAS anxiety symptoms’ correspondence with scores on the 5-item Screen for Child Anxiety Related Emotional Disorders (SCARED) and teacher ratings of child anxiety. Based on data corresponding to 702 children (M age= 9.65, SD=0.70; 51.9 % girls; 55 % Hispanic/Latino), findings showed some sex and ethnic variations in SCAS measured anxiety at the item and scale levels. Moreover, SCAS correspondence to the 5-item SCARED was found across ethnicity and sex. SCAS correspondence to teacher ratings was found for non-Hispanic White boys and non-Hispanic White girls, marginally in Hispanic/Latino boys, and poorly in Hispanic/ Latino girls.

The past two decades have witnessed increasing efforts to develop, evaluate, and promote assessment methods and instruments to identify the mental health needs of children, especially in school settings with vested interests in preventive interventions (e.g., Institute of Medicine 2009; Pina, Gonzales, Holly, Zerr, and Wynne 2013; Silverman and Ollendick 2005; Trent et al. 2013). With these efforts, evidence-based assessments have gradually shifted from broad and extensive measures to brief, relatively inexpensive, and focused measures (Mash and Hunsley 2005; Mash and Terdal 1997; Muris, Merckelbach, Ollendick, King, and Bogie 2002), many of which show excellent specificity and sensitivity (e.g., Beidel, Turner, and Morris 1995; March, Parker, Sullivan, Stallings and Conners 1997).

In the assessment of child anxiety, comprehensive reviews and practice guidelines (e.g., Pina et al. 2013a, b; Silverman and Ollendick 2005) have identified the Spence Children’s Anxiety Scale (SCAS; Spence 1997, 1998) as a promising instrument for school-based universal screenings (Barrett, Farrell, Ollendick, and Dadds 2006; Barrett, Lock, and Farrell 2005; Barrett and Turner 2001; Spence 1997, 1998; Spence, Barrett, and Turner 2003). The SCAS is a child self-report measure developed to closely align with Diagnostic and Statistical Manual of Mental Disorder diagnoses (e.g., generalized anxiety disorder, separation anxiety disorder, and social phobia; APA 2013), offering clinical cutoff scores as well as translations into more than 20 languages. Scores on the SCAS differentiate children with anxiety from those with other disorders (Spence 1998; Spence, Barrett, and Turner 2003), a limitation well-documented in the literature about other measures (e.g., Revised Children’s Manifest Anxiety Scale: RCMAS; Reynolds and Richmond 1978, Fear Survey Schedule for Children-Revised: FSSC-R; Ollendick 1983; Perrin and Last 1992; Silverman and Ollendick 2005), and show sensitivity to change following intervention efforts (Barrett et al. 2005, 2006; Barrett and Turner 2001).

In schools, the SCAS could be especially useful because child anxiety is associated with poor attendance, poor testing performance, academic failure, and difficulties in relationships with student peers (e.g., Duchesne, Vitaro, Larose, and Tremblay 2008; Grover, Ginsburg, and Ialongo 2007; Kearney 2008). Moreover, the SCAS can help address school mandates for the identification and provision of services to students with social and emotional difficulties. In fact, in the United States, the Individuals with Disabilities Education Improvement Act (IDEA; 2004), the No Child Left Behind Act (NCLB; 2002), and the President’s Commission on Excellence in Special Education (2002) direct that the mental health needs of children (including anxiety) be identified via evidence-based approaches. Despite these advantages, there is no research examining the SCAS’ cross-ethnic (Latino/ Hispanic vs. non-Hispanic White) and cross-sex robustness. Examining how a measure functions across ethnic groups is important because there is evidence of extreme response bias among Hispanic/Latino adults (e.g., Hui and Triandis 1989; Marin, Gamba, and Marin 1992) and scales with limited response sets (e.g., RCMAS; FSSC-R) do not afford examining this possibility in children. The SCAS, however, uses a 4-point response scale (never to always) that lends itself to exploring response bias via strong invariance testing. The SCAS also contains items that might inform non-invariance in a way that reflects immigration hardships among the Hispanic/Latino child population in our sample (e.g., “something awful will happen to someone in my family (or me),” “feel scared to travel in the car, or on a bus”). This is possible because a significant proportion of our Hispanic/Latino sample resides in a geographical area in Arizona with police-driven immigration raids well documented in the national media and research reports (e.g., Hing 2012; Santos 2013). Moreover, unlike other measures of child anxiety and related problems with demonstrated measurement invariance (Children’s Depression Inventory: CDI; Kovacs 1992; FSSR-C; Multidimensional Anxiety Scale for Children: MASC; March et al. 1997; RCMAS), the SCAS has been earmarked by past research for screening child anxiety with the intention of identifying children who could benefit from preventive interventions (e.g., Barrett et al. 2005, 2006; Barrett and Turner 2001; Essau, Conradt, Sasagawa, and Ollendick 2012; Neil and Christensen 2009). And, as highlighted by the Institute of Medicine (2009), the SCAS meets the screening criteria for cost effectiveness, acceptability, and mandates (IDEA 2004; NCLB 2002; President’s Commission on Excellence in Special Education 2002).

Building on this knowledge, the present study relied on item-response theory and data analytic strategies used in similar published research (e.g., Millsap 2010, 2011; Pina, Little, Knight, and Silverman 2009; Pina, Little, Wynne, and Beidel 2013) to explore the cross-group measurement invariance of the SCAS (Spence 1997, 1998) using secondary data (K01MH086687-A1). More specifically, we used configural invariance tests to examine whether the anxiety construct measured by the SCAS and its assumed factors manifests itself similarly across groups (Ghorpade, Hattrup, and Lackritz 1999; Millsap and Tein 2004; Vandenberg and Lance 2000). Next, weak invariance tests were conducted to elucidate whether items on the SCAS have the same meaning across groups, thereby offering the opportunity to identify additional cross-group differences in the construct (Labouvie and Ruetsch 1995; Raykov 2004). Lastly, strong and strict tests were used to examine whether the same level of severity in anxiety is necessary across groups for respondents to endorse an item and whether there is cross group bias in the endorsement of an item, respectively (Byrne, Shavelson, and Muthén 1989; Widaman and Reise 1997). Thus, measurement invariance tests on the SCAS have the potential to shed light, for example, on the nature of sex differences in child anxiety. For instance, girls are generally found to report higher anxiety levels compared to boys (e.g., Lewinsohn, Gotlib, Lewinsohn, Seeley, and Allen 1998; Silverman and Carter 2006) and while variations are partially corrected by using sex- norms and cutoff scores (e.g., Albano and Krain 2005), the nature of sex differences in anxiety could be further revealed via strong invariance tests. That is, girls’ endorsement of SCAS items, such as “I am popular among other kids my age,” “I worry what other people think of me,” “I like myself,” in a manner different from boys might indicate specific gender role socialization effects emphasizing more desirable behaviors in social situations for girls and the possible role of these effects in anxiety development (e.g., Carter, Silverman, and Jaccard 2011; Cross and Madson 1997; Muris, Meesters, and Knoops 2005). As such, the present study offers an opportunity to identify in what ways, if any, SCAS measured anxiety manifests itself differently in girls versus boys.

This research also examined, in an exploratory way, SCAS measured anxiety’s correspondence to the 5-item Screen for Child Anxiety Related Emotional Disorders (SCARED; Birmaher et al. 1997; Birmaher et al. 1999) as well as teacher ratings of their students’ anxiety and its correspondence to SCAS measured anxiety. Inclusion of the 5-item SCARED and teacher ratings was pragmatic given their brevity and thereby viability for use in school settings (Campbell 2004; Dadds, Spence, Holland, Barrett, and Laurens 1997). The focus on 4th and 5th grade children also was pragmatic given that the primary study from which these data were drawn focused on these school grades (K01MH086687-A1). In addition, data on the SCAS show that younger children (8–11 years) report more SCAS anxiety symptoms compared to older children (12–15 years) (see Spence 2008) suggesting that respondents might not approach the SCAS in a similar fashion. For example, there might be differences in the ways younger children approach the item “I feel scared if I have to sleep on my own” compared to older children. This is consistent with theory and research suggesting that the amount and type of anxiety experienced by youth varies across developmental periods (e.g., Kendall and Ollendick 2004; Warren and Sroufe 2004; Weems and Stickle 2005; Westenberg, Siebelink, and Treffers 2001). For this reason, it was sensible for us to focus on one of these age groups, younger children (8–11 years) or 4th and 5th graders.

Method

Participants

Data corresponding to 702 Hispanic/Latino (mostly Mexican origin; n=385) and non-Hispanic White (NHW; n=317) children (M age=9.65, SD=0.70; 51.9 % girls) were examined in this study. Between group comparisons showed that children were more similar than different with the exception that Hispanic/Latinos were slightly older (M_age=9.70, SD=0.71) than NHWs (M_age=9.60, SD=0.69; t=2.02 (699), p=0.04; one Hispanic/Latino child did not report age). In addition, a total of 58 teachers (Mage=44.06, SD=10.17; 88.9 % female; 77.8 % NHW; 70.4 % with 10 or more years of experience teaching) provided data about children in their classrooms who were participating in the study.

Measures

Spence Children’s Anxiety Scale (SCAS; Spence 1997,1998) comprises 38 items designed to assess child anxiety symptoms across six domains: social phobia, separation anxiety, panic attack/agoraphobia, obsessive compulsive disorder, generalized anxiety, and physical injury fears. For each item, children select the response that best describes how often a given feeling, thought, or behavior is experienced. Items are scored as 0 (never), 1 (sometimes), 2 (often), or 3 (always). Spence, Barrett, and Turner (2003) reported internal consistencies for the total and subscale scores ranging from 0.60 to 0.92 and 12 week test-retest reliability estimates ranging between 0.51 and 0.75. The SCAS correlates significantly with the Revised Children’s Manifest Anxiety Scale (RCMAS; Reynolds and Richmond 1978) (r=0.40 to 0.75, p<0.001; Spence et al. 2003). The internal consistency (alpha) coefficient for the SCAS total scale is 0.93 in this sample.

Screen for Child Anxiety Related Emotional Disorders (SCARED; Birmaher et al. 1997; Birmaher et al. 1999) comprises 41 items designed to assess anxiety across five domains: generalized anxiety, separation anxiety, panic/somatic, social phobia, and school phobia. For each item, children report the extent to which the statement is true for them based on a three-point scale. Items are scored as 0 (not true or hardly ever true), 1 (sometimes true), or 2 (true or often true). The SCARED has been shown to be psychometrically sound with internal consistency (alpha) coefficients ranging from 0.74 to 0.93 across subscales and total score and test-retest reliability rates of 0.70 to 0.90 (Birmaher et al. 1997). In Birmaher et al. (1999), it was recommended that the item with the highest loading in each subscale could be used to assess anxiety on the basis of a 5-item SCARED (brief version). In the present study, we used the 5-items from the SCARED recommended by Birmaher et al. (1999). Our data showed a relatively low internal consistency (alpha) coefficient for the 5-item SCARED of 0.52. This low alpha likely occurred because the 5-items belong to different subscales as noted above.

Brief Teacher Rating Scale (Arizona Anxiety Resilience Building Project 2011) is a single item rating of the child’s anxiety adapted from the Fear or Feelings Thermometer (e.g., DiNardo and Barlow 1988; Israel, Becker, and Neilans 1977; Silverman and Albano 1996; Silverman, Saavedra, and Pina 2001). Teachers provide a rating of the child’s anxiety (defined as shy, nervous, afraid, and/or inhibited) on a scale from 0 to 10 with higher ratings equating to more severe estimations of child anxiety.

Procedures

All study procedures were approved by the university’s Institutional Review Board. Children were recruited from nine public elementary schools after the primary caregiver (or legal guardian) provided consent for an intervention study (K01MH086687-A1). Briefly, a total of 1,539 letters describing the study were sent to the parents (or legal guardians) of 4th and 5th grade children in regular classes. In response, 874 parents provided consent, 326 students did not return the signed consent form and 338 parents did not provide consent. With parent consent (assent from child), a battery of questionnaires was administered in the classroom to all participating children. All measures were administered in English. Non-participating children were provided with activity sheets (e.g., puzzles, mazes) during the administration of the questionnaire battery. During administration, a trained research assistant read aloud the questions and response choices while two other research assistants monitored administration and provided individual assistance to children, as necessary. While children responded to the questions, teachers were provided with a class roster including only those students in the classroom whose parents had provided consent. Teachers then rated the children in the classroom using the brief teacher rating scale.

Data Analytic Plan

Multi-group confirmatory factor analyses (CFAs) in MPlus software were used to assess measurement and factor invariance of the 6-factor SCAS across sex and ethnicity (Brown 2006; Muthén and Muthén 1998–2011). Measurement invariance (MI) analyses were then used to estimate the cross-ethnic and cross-sex equivalence of the SCAS via nested multi-group CFAs. More specifically, MI testing began with configural analyses, which examine the overall model fit and significance of factor loadings for a multi-group model with no constraints across the groups of interest (i.e., ethnicity, sex). In turn, stepwise cross-group invariance constraints were used to examine weak, strong, and strict invariance (Widaman and Reise 1997). That is, weak invariance was tested by constraining item factor loadings to be equal across groups and by comparing the fit of the constrained model to the original configural model. Strong invariance was tested by comparing a model with both factor loadings and item thresholds constrained to equality across groups to the model with only factor loadings constrained across groups. Next, strict invariance was tested by comparing a fully constrained model (i.e., constrained loadings, thresholds, and item residuals) to a model with constrained loadings and thresholds but free item residuals. Finally, factor structure differences across groups were compared using factor variance, factor covariance and latent mean equality constraints in separate sets of nested model tests. To ensure accurate nested tests of MI at each successive step, partial MI models, which allowed known variant parameters to vary across groups, were compared to more constrained models (Vandenberg and Lance 2000).

Given that SCAS items use ordinal categorical response sets, a weighted least squares mean variance (WLSMV) estimator was used, which is robust to violations of normality (Flora and Curran 2004; Muthén and Muthén 1998–2011). Additionally, we used model identification and MI analyses procedures recommended for ordinal categorical variables (Millsap and Tein 2004). We evaluated model fit of initial configural models using the chi-square measure of absolute fit and two practical fit indices: the comparative fit index (CFI), and root mean square error of approximation (RMSEA). Established cutoffs of CFI and RMSEA were used to evaluate fit; cutoffs of CFI ≥0.95 and RMSEA ≤0.05 suggest good fit and cutoffs of CFI ≥0.90 and RMSEA ≤0.08 suggest adequate fit (Cheung and Rensvold 2002; Hu and Bentler 1998). Next, MI was evaluated by determining whether the value of the chi-square measure of fit changed significantly in nested model tests.¹ Based on current recommendations of Sass, Schmitt and Marsh (2014), additional evaluation of change in alternative fit indices (AFIs) were not used as a primary threshold of measurement non-invariance.² However, change in AFIs (CFI and RMSEA) in nested model testing were calculated and presented to supplement evaluation of nested model comparisons. Current research suggests that a positive change in the RMSEA of 0.007 ormore and a negative change in the CFI of ≤-002 are indicative of notable decrement in AFI fit (Meade, Johnson and Braddy 2008; Sass et al. 2014). Subsequent analyses were devoted to evaluating rates of clinically significant SCAS scores across the sex by ethnicity groups. Further, raw score equivalents and rates of clinically significant SCAS T-scores and percentiles based on published norms were compared to corresponding raw score equivalents and clinical score rates based on this sample’s sub-group distributions.

Finally, concordance of the SCAS sum score with the 5-item SCARED sum score and teachers’ global rating of child anxiety was examined using multi-group regressions across the four sex by ethnicity groups. Given that 58 teachers provided reports on 702 students, potential interdependency of outcomes by teacher report was accounted for using the Mplus complex procedure in the initial set of regression models. Predictors were grand mean centered for regression analyses.

Results

Measurement Invariance Across Sex

We examined MI of the 6-factor Spence Child Anxiety Scale across sex and ethnicity in separate models. The length of the SCAS prohibited using four sex by ethnicity groups in MI testing (Sass et al. 2014). The established 6-factor configuration of the SCAS was used in CFA analyses (Spence 1997). As shown in Table 1, the initial configural model of the SCAS showed adequate fit across sex. Configural fit was also evinced by significant standard factor loadings ≥0.30 for all items on their corresponding factors (see Table 2). A subsequent weak invariance test constraining factor loadings across sex showed a significant change in chi-square fit, thus suggesting metric non-invariance, although AFI fit did not diminish. Follow-up Wald difference tests were used to simultaneously test factor loading differences on specific items. Wald tests showed significant non-invariance on items 15 and 30. Item 15 showed a higher factor loading for boys versus girls on separation anxiety disorder (SAD). Item 30 showed a higher factor loading for girls versus boys on panic disorder. With non-invariant items identified, we examined a partial weak invariance model allowing loadings of items 15 and 30 to vary across sex. As shown in Table 1, compared to the configural model, this partial weak invariant model did not show evidence of factor loading invariance.

Table 1.

Summary of model fit statistics for cross-sex and cross-ethnicity measurement and factor structure invariance tests of the spence children’s anxiety scale

	χ2(df)	CFI	RMSEA	Δχ2(df)	ΔCFI	ΔRMSEA
Cross-Sex
Configural Invariance	503.610 (271)	0.937	0.049
Weak Invariance				37.849* (24)	0.003	0
Partial Weak Invariance				22.739 (23)	0.005	−0.01
Strong Invariance				83.680*** (41)	−0.001	0
Partial Strong Invariance				26.361 (32)	0.004	−0.002
Strict Invariance				52.501** (29)	0.005	0.001
Partial Strict Invariance				22.729 (24)	0.009	−0.002
Factor Variance Invariance				3.034 (4)	0.004	−0.001
Factor Covariances Invariance				3.261 (4)	0.015	−0.002
Latent Means Invariance				30.226*** (2)	−0.014	0.01
Cross-Ethnicity
Configural Invariance	460.490*** (273)	0.944	0.044
Weak Invariance				25.650 (24)	0.005	−0.001
Strong Invariance				66.270*** (41)	−0.001	0
Partial Strong Invariance				42.540 (40)	0.002	−0.001
Strict Invariance				49.930** (29)	0.003	0
Partial Strict Invariance				35.140 (28)	0.007	−0.002
Factor Variances Invariance				3.919 (3)	0.003	0.000
Factor Covariances Invariance				2.685 (3)	0.021	−0.002
Latent Means Invariance				7.028* (2)	−0.002	0.002

χ²chi-square measure of absolute fit, CFI comparative fit index, RMSEA root mean square error of approximation

^*p<0.05,

^**p<0.01,

^***p<0.001

Table 2.

Cross-sex standard factor loadings of the 6-factor configural model of the spence children’s anxiety scale

	Injury fear		SAD		Soc. anx.		GAD		OCD		Panic
# Item	Boy	Girl	Boy	Girl	Boy	Girl	Boy	Girl	Boy	Girl	Boy	Girl
2 scared of the dark	0.54	0.51
16 scared of dogs	0.48	0.40
21 scared of doctors	0.69	0.66
23 scared high places	0.64	0.56
29 scared of insects	0.66	0.50
5 afraid of on my own			0.42	0.35
8 worry away from parents			0.54	0.57
11 worry awful happen family			0.61	0.66
14 scared sleep on own			0.56	0.57
15 trouble going to school			0.42	0.68
38 scared away overnight			0.50	0.45
6 scared take a test					0.55	0.59
7 afraid use public toilets					0.46	0.54
10 afraid make fool of myself					0.59	0.62
9 worry do badly at school					0.74	0.70
26 worry people think of me					0.75	0.70
31 afraid talk in front of class					0.56	0.53
1 worry about things							0.65	0.56
3 funny feeling stomach							0.57	0.53
4 feel afraid							0.64	0.50
18 (if) problem heart beats fast							0.55	0.62
20 worry bad happen to me							0.70	0.72
22 when problem feel shaky							0.61	0.68
13 keep checking things									0.52	0.58
17 can’t get (rid of) bad thoughts									0.59	0.66
24 think thoughts to stop bad									0.71	0.69
35 do things over and over									0.39	0.47
36 bothered by bad thoughts									0.76	0.75
37 do right to stop bad thoughts									0.61	0.75
12 can’t breathe for no reason											0.65	0.75
19 tremble when no reason											0.71	0.77
25 scared if travel car, bus, train											0.64	0.60
27 afraid of crowded places											0.60	0.57
28 scared for no reason											0.86	0.78
30 dizzy or faint when no reason											0.50	0.75
32 heart beats quick no reason											0.71	0.71
33 worry scared of nothing											0.76	0.75
34 afraid in small closed places											0.62	0.53

SAD separation anxiety disorder, Soc Anx. social anxiety disorder, GAD generalized anxiety disorder, OCD obsessive compulsive disorder, Injury fear physical injury fears, Panic panic disorder. All represented loadings are significant at p<0.05

Non-invariance of item thresholds and residuals was also identified. A strong invariance by sex test showed a significant chi-square difference and slight decrement in CFI fit compared to the partial weak invariant model. Follow-up Wald difference tests identified non-invariant thresholds for seven items: item 4 on generalized anxiety disorder (GAD), items 6, 26 and 31 on social anxiety disorder, item 15 on SAD, item 24 on obsessive compulsive disorder (OCD), and item 29 on physical injury fears. Therefore, non-invariant thresholds were allowed to vary across sex in a subsequent strong invariance test. Fit of the partially invariant test of strong invariance did not differ from the partially invariant weak invariance model (see Table 1). Cross-sex strict invariance was next examined by comparing the strong partial invariance model to a model that added equality constraints on items residuals across sex. Results revealed a significant chi-square difference and decrement in RMSEA fit, thus suggesting non-invariance of item residuals. Follow-up chi-square difference testing suggested that residuals of six items were non-invariant across sex: items 4, 12, 29, 19, 37, 22.³ As shown in Table 1, allowing these items residuals to vary across sex resulted in a non-significant chi-square difference when compared to the partial strong invariant model.

Factorial invariance tests affirmed the invariance of variances and covariances of the 6-factor SCAS. However, nested comparisons suggested non-invariance of SCAS latent means as evinced by a significant decrement in chi-square and AFIs. Subsequent chi-square difference tests revealed that girls reported higher levels of SAD, social anxiety disorder, GAD, physical injury fears, and panic disorder compared to boys.

Measurement Invariance Across Ethnicity

MI and factor in-variance tests of the SCAS across ethnic groups suggested non-invariance of item thresholds, item residuals and latent factor means. The initial configural model showed adequate fit across ethnicity. Additionally, cross-ethnicity configural invariance was supported by significant standard factor loadings ≥0.30 for all items on their corresponding factors (see Table 3). As shown in Table 1, a subsequent weak invariance test also affirmed invariance of factor loadings across ethnic groups. In contrast, however, a strong invariant model with thresholds constrained to equality across ethnicity showed a significant decrement in chi-square fit, thus suggesting threshold non-invariance. Wald difference tests revealed that thresholds of item 25 were non-invariant across ethnic groups. As shown in Table 1, fit of a follow-up partial strong invariance model with item 25 allowed to vary across ethnicity did not differ significantly from the weak invariance model.

Table 3.

Cross-ethnicity standard factor loadings of the 6-factor configural model of the spence children’s anxiety scale

	Injury fear	SAD	Soc. anx.	GAD	OCD	Panic
# Item	W	H	W	H	W	H	W	H	W	H	W	H
2 scared of the dark	0.61	0.49
16 scared of dogs	0.43	0.42
21 scared of doctors	0.67	0.68
23 scared high places	0.58	0.60
29 scared of insects	0.63	0.58
5 afraid of on my own			0.42	0.35
8 worry away from parents			0.63	0.44
11 worry awful happen family			0.68	0.55
14 scared sleep on own			0.59	0.51
15 trouble going to school			0.76	0.67
38 scared away overnight			0.43	0.46
6 scared take a test					0.59	0.60
7 afraid use public toilets					0.59	0.39
10 afraid make fool of myself					0.78	0.59
9 worry do badly at school					0.65	0.69
26 worry people think of me					0.74	0.76
31 afraid talk in front of class					0.52	0.58
1 worry about things							0.64	0.59
3 funny feeling stomach							0.54	0.58
4 feel afraid							0.60	0.56
18 (if) problem heart beats fast							0.61	0.59
20 worry bad happen to me							0.73	0.69
22 when problem feel shaky							0.68	0.63
13 keep checking things									0.58	0.52
17 can’t get (rid of) bad thoughts									0.67	0.57
24 think thoughts to stop bad									0.74	0.65
35 do things over and over									0.49	0.36
36 bothered by bad thoughts									0.77	0.75
37 do right to stop bad thoughts									0.65	0.67
12 can’t breathe for no reason											0.70	0.68
19 tremble when no reason											0.72	0.76
25 scared if travel car, bus, train											0.49	0.69
27 afraid of crowded places											0.61	0.58
28 scared for no reason											0.81	0.82
30 dizzy or faint when no reason											0.65	0.82
32 heart beats quick no reason											0.76	0.61
33 worry scared of nothing											0.79	0.72
34 afraid in small closed places											0.56	0.60

W NHW, H Hispanic/Latino, SAD separation anxiety disorder, Soc Anx. social anxiety disorder, GAD generalized anxiety disorder, OCD obsessive compulsive disorder, Injury fear physical injury fears, Panic panic disorder. All represented loadings are significant at p<0.05

Furthermore, a test of strict invariance suggested non-invariance of items residuals across ethnicity, as evinced by a significant change in chi-square fit compared to the partial strong invariance model. Follow-up chi-square difference tests suggested that residuals of items 25 and 7 were non-invariant across ethnicity. Allowing those item residuals to vary resulted in a non-significant change in model fit compared to the partial strong invariance model (see Table 1). Further, factor invariance tests affirmed invariance of SCAS latent variances and latent covariances across ethnicity. However, tests of the equality of latent means revealed a significant change in chi-square fit and notable decrement in CFI fit. Specifically, Hispanic/Latino children showed greater panic disorder, SAD and OCD levels than their NHW counterparts.

Evaluation of T-scores Across Sex by Ethnicity Groups

As shown in Table 4, we next compared rates of clinically significant SCAS total scores across sex by ethnicity groups using both published norms and this sample’s sub-group distributions. Based on published data, a T-score of 65 corresponding to the 93.32rd percentile of the distribution and +1.5 SD was used as a clinically significant threshold. Using suggested raw score equivalents of the published T-scores for each sex, we found that a higher percentage of Hispanic/ Latino boys, 7.1 to 8.8 %, and Hispanic/Latino girls, 8.4 to 9.9 %, would be identified as experiencing clinically significant anxiety than their non-minority counterparts, NHW boys = 4.5-5.8 %; NHW girls = 3.1–3.7 %. However, by computing percentiles and T-scores based on this sample, we found that lower raw scores than those published were equivalent to T-scores of 65 or more for NHW boys (this sample’s raw cutoff = 49) and NHW girls (this sample’s raw cutoff = 55). In addition, higher raw scores than those published were equivalent to T-scores of 65 or more for Hispanic/Latino boys (this sample’s raw cutoff = 58) and Hispanic/Latino girls (this sample’s raw cutoff = 65). Thus, when adjusting via this sample’s raw score equivalents, the distributions yielded anxiety rates ranging from 6 to 7 % for all groups but NHW girls’ rate was 9.3 %.

Table 4.

Comparison of clinically significant t -score rates on the spence children’s anxiety scale across sex by ethnicity groups: published norms versus sample distribution

	NHW males	NHW females	Hispanic/Latino males	Hispanic/Latino females
Clinical Scores Based on Published Norms
Raw Scores	51–54	59–61	51–54	59–61
Percentile	93	93	93	93
T-score	65	65	65	65
Sample Rate(=/>cutoff)	5.8–4.5 %	3.7–3.1 %	8.8–7.1 %	9.9–8.4 %
Clinical Scores Based on Sample Distribution ≥
Raw Score	49.16	55.00	58.10	65.37
Percentile	93	93	93	93
T-score	65	65	65	65
Sample Rate	6.5 %	9.3 %	6%	6.4 %

NHW non-Hispanic White. Multiple raw scores are suggested as clinical cut-offs in published norms. Sample rates based on the sample distribution are based on available whole SCAS scores

Measuring Child Anxiety via the SCAS, the SCARED, and Teacher Ratings

We examined concurrence of the SCAS sum score with the 5-item SCARED and teachers’ anxiety ratings across sex by ethnicity groups. Regression results revealed that the 5-item SCARED was significantly associated with the SCAS total score in NHW boys, B=0.67, p<0.001, NHW girls, B=0.66, p<0.001, Hispanic/Latino boys, B=0.69, p<0.001, and Hispanic/Latino girls, B=0.65, p<0.001. A second set of regression analyses showed that teachers’ global anxiety ratings of the child were significantly associated with the SCAS total score in NHW boys, B=0.26, p<0.01, and NHW girls, B=0.28, p<0.01. A trend toward significance was found for the prediction of SCAS total score from teachers’ global anxiety ratings of the child in Hispanic/Latino boys, B=0.13, p<0.10. No statistically significant relation was found for Hispanic/Latino girls. Given that the association between teachers’ global rating of child anxiety and SCAS total score was not significant in Hispanic sub-groups, we examined the potential that this association varied by teacher using by-group multilevel analyses. Multi-level models did not suggest that prediction of SCAS sum score from teachers’ global rating of child anxiety varied significantly by teacher in any of the sex by ethnicity groups, all ps>0.05.

Discussion

In the clinical child and adolescent area, significant progress has been made to establish measures for the assessment of social, emotional, and behavior problems, including those related to anxiety and its disorders (Mash and Hunsley 2005; McLeod, Jensen-Doss, and Ollendick 2013). However, little is known about the cross-ethnic (Hispanic/Latino vs. NHW) and cross-sex robustness of most measures earmarked for the assessment of child and adolescent anxiety. In fact, a recent review of this literature revealed that only four anxiety rating scales have been identified as invariant across ethnicity (Pina et al. 2013a, b). Varela, Sanchez-Sosa, Biggs, and Luis (2008), for example, reported on the cross cultural invariance of three child anxiety measures using a sample of Mexican youth living in Mexico and Hispanic and NHW youth living in Louisiana. Findings suggested that the RCMAS, FSSC-R, and the MASC are invariant across Hispanic/Latino and NHW children. In another study, Pina et al. (2009) reported on a sample of Latino and NHW youth living in South Florida and data showed measurement invariance of the RCMAS across these two groups. And, more recently, Trent et al. 2013 examined the Revised Child Anxiety and Depression Scale (RCADS; Chorpita, Yim, Moffitt, Umemoto, and Francis 2000) in a sample of youth in Mississippi and found support for the measurement invariance of this measure across African American and NHW youth. Building on this body of work, the present study examined the utility of selective anxiety screening armamentaria, with high feasibility for use in school settings, based on a sample of 702 children. More specifically, the focus of this investigation was on the Spence Children’s Anxiety Scale (SCAS) since this measure has been used extensively in recent research and has been identified as promising in real world settings (e.g., Barrett et al. 2005; Silverman and Ollendick 2005; Spence et al. 2003). We also report correspondence from SCAS anxiety to the 5-item SCARED and teacher ratings about the child’s anxiety.

Our findings indicated that the SCAS is a fairly robust measure across ethnicity (i.e., Hispanic/Latino, NHW) and sex, with more variations for the latter. Non-invariance for a few SCAS symptoms does not necessarily refute the potential clinical utility of the measure (Widaman and Reise 1997); rather, it could indicate the possibility that more clinically useful or meaningful information could be gained (e.g., Knight, Roosa, and Umaña-Taylor 2009; Pina et al. 2013a, b). For example, we found that several symptoms relevant to anxiety in the school’s context, such as “I feel scared when I have to take a test,” “I feel afraid if I have to talk in front of my class,” required a higher level or severity for endorsement on the part of girls. Sex differences might reflect greater social evaluative concerns among girls compared to boys (e.g., Cross and Madson 1997; Rudolph and Conley 2005) in which case following-up with additional probing relevant to those situations might help refine knowledge about anxiety in girls and thereby lead to more focused anxiety prevention or reduction intervention efforts. If these findings replicate, positive endorsement of anxiety provoking school situations on the part of girls could offer an opportunity for school counselors, social workers, and psychologists to learn about the meanings girls versus boys might be assigning to non-invariant items. And, the additional probing might help guide more appropriate referrals or education placements in school contexts.

In the present study, we also had the opportunity to explore cross-ethnic differences in anxiety as measured by the SCAS. Findings were generally consistent across Hispanic/Latinos and NHW children. There was, however, some cross-ethnic non-invariance. First, two items related to public spaces showed non-invariance: “I feel afraid if I have to use public toilets or bathrooms” and “I feel scared if I have to travel in the car or on a bus or on a train” with Hispanic/Latino children showing greater endorsement. Consistent with our earlier recommendation, it is important to probe about the meaning children are assigning to these items. For example, it might be the case that endorsement of anxiety-related to traveling in a car, bus, or train is related to the immigration experiences of the Hispanic/Latino children in our sample. More specifically, some immigration policies and practices have been associated with poorer child well-being (Santos and Menjívar 2013; Santos, Menjivar, and Godfrey 2013) as well as reports that immigrant youth and families (particularly those with an undocumented parent) minimize time away from home and are reluctant to utilize public services (e.g., Capps, Fix, Ost, Reardon-Anderson, and Passel 2005; see Yoshikawa and Kalil 2011). Thus, it is possible that children who endorse anxiety related to some shared public spaces may also be showing fears related potential deportation (whether true or not). Second, Hispanic/Latino children had significantly higher levels of separation anxiety compared to NHWs. In a clinic-referred sample of children (mostly highly acculturated Cuban Americans), Ginsburg and Silverman (1996) also found greater separation anxiety disorder diagnostic rates for Hispanics compared to NHWs. In general, it might be the case that our separation anxiety scale variations are in keeping with cultural values that emphasize family closeness and reliance on family members as the primary source of support rather than a disorder. To be assured, it would be important to follow-up positive endorsement of separation anxiety disorder symptoms by Hispanic/Latino children with administration of both a familismo measure such as the Mexican American Cultural Values Scale (MACVS; Knight et al. 2010) and the Separation Anxiety Module of the Anxiety Disorders Interview Schedule for Children (Silverman and Albano 1996). Third, results suggested that standard SCAS cutoff scores could over-identify Hispanic/Latino children as clinically anxious. It appears that cutoff scores of 58 and 65 for boys and girls, respectively, may be more adequate. We thus suggest application of these revised cutoff as way to refine identification of anxious children (for prevention, early intervention, or referrals), especially in the context of supplemental school-based testing services mandated by the No Child Left Behind Act (2002) while avoiding the over-pathologizing of this ethnic group.

When it comes to anxiety symptom correspondence between the SCAS and the 5-item SCARED, we found robust agreement in all sex by ethnicity groups; thus supporting the use of the SCAS as a possible screen for child anxiety. Additionally, this finding suggests that the brief 5-item SCARED may be a useful supplemental school-based screen for the assessment of child anxiety problems for Hispanic/ Latino and NHW boys and girls. This finding is interesting in light of the low alpha reliability coefficient found for the 5-item SCARED in this study; thus, highlighting the robustness of measurement invariance tests over Cronbach’s formula (e.g., Vandenberg and Lance 2000). Teacher anxiety ratings about the child, on the other hand, showed less consistency across sex and ethnicity. That is, teachers’ global ratings of anxiety had significant correspondence with children’s own ratings of anxiety for NHW boys and girls, marginal correspondence for Hispanic/Latino boys, and no correspondence for Hispanic/Latino girls. This level of convergence for some children is consistent with published research showing that adults and child informants can offer unique (and often divergent) information about child anxiety and related problems (De los Reyes and Kazdin 2005; De los Reyes, Thomas, Goodman, and Kundey 2013). As such, it might the case that convergence results between the SCAS and the SCARED reflect a true difference in teacher identification of anxiety in Hispanic/Latino children. Another possibility, however, is that similar to past research, Hispanic/Latino child anxiety is manifesting itself in our sample in terms of physical symptoms or concerns (e.g., Pina and Silverman 2004; Varela et al. 2004, 2008; Varela, Weems, Berman, Hensley, and de Bernal 2007), which may not be easily recognized by teachers.

Several limitations and directions for future research are noteworthy. First, the current study included both self-report and teacher ratings of child anxiety but parent or clinician data were not available in the dataset for these secondary analyses. Given our focus on child anxiety and school settings, the emphasis on children’s self-ratings and teacher ratings is sensible. Nonetheless, studying cross-group predictive validity of the SCAS against a clinical criterion, like DSM-V clinician interviews can help determine cross-cultural validity of the SCAS (Millsap 2007; Vandenberg and Lance 2000), especially as a functional equivalence framework. As current research suggests, even partial measurement non-invariance may exacerbate cross-group predictive biases when using an external criterion (Millsap 2007). Thus, optimizing diagnostic sensitivity by gender and ethnic groups against a clinical criterion using receiver operator characteristic (ROC) analyses is recommended for future research (e.g., Pina et al. 2013a, b). Second, the current study focused exclusively on children in 4th and 5th grade, resulting in a relatively narrow age range. Given prior research showing that anxiety problems stabilize during this developmental period (e.g., Weems and Stickle 2005), accurate assessment of child anxiety is particularly important for children at this age. In addition, theory and research suggest that normal development influences the type of anxiety symptoms a child may experience (Kendall and Ollendick 2004; Warren and Sroufe 2004; Westenberg, Siebelink, and Treffers 2001). As such, focusing on 4th and 5th graders specifically allows us to be confident that our findings for this age group are robust given our sample size. Third, whereas our focus was on Hispanic/Latino children who were mostly Mexican-origin, it is important to note that within ethnic group variations exist (Knight et al. 2009). As such, examining invariance as a function of more ethnically and culturally meaningful focal variables, such as cultural orientation, would offer a more stringent test of a measure’s ability to capture the construct of interest in the context of culture. Fourth, despite our large sample size, we could not examine measurement invariance of the SCAS across the four sex by ethnicity groups, or predictive associations between the SCAS latent trait and the other anxiety measures (Wolf, Harrington, Clark, and Miller 2013). Future research using a larger sample may elucidate whether further measurement and structural invariance differences across specific sub-groups of youth exist (e.g., using exploratory bifactor analysis; Ebesutani et al. 2012).

Despite these limitations, we suggest the SCAS be used in school-based assessments of student anxiety, a problem that interferes with school success in some children as well as teachers’ ability to effectively deliver academic content. The recommendation that the SCAS could be used regularly by schools comes with the added advantage that this scale is available online at no charge; an issue that can help alleviate the economic hardships some schools face. The SCAS also could aid schools in gathering outcome data from their program implementation efforts to demonstrate accountability for services offered by school staff members dedicated to working with children with academic, social, and emotional difficulties related to student anxiety. To supplement the SCAS, the 5-item Screen for Child Anxiety Related Emotional Disorders (5-item SCARED) also could be used along with teacher ratings of primary NHW children. In fact, together, these measures fit well with mandates from President’s Commission on Excellence in Special Education (2002) and the No Child Left Behind Act (2002) to identify and address the mental health needs of children via evidence-based approaches.