Anxiety in youth with and without autism spectrum disorder: Examination of factorial equivalence

Abstract

Although anxiety is frequently reported among children and adolescents with autism spectrum disorder (ASD), it has not been established that the manifest symptoms of anxiety in the context of ASD are the same as those seen in youth without ASD. This study sought to examine the metric and latent factor equivalence of anxiety as measured by the Multidimensional Anxiety Scale for Children, parent-report (MASC-P) and child-report (MASC-C), in youth with anxiety disorders and ASD with intact verbal ability (n = 109, M_age = 11.67 yrs, 99 male) and a gender-matched comparison group of typically developing (TD) children and adolescents with anxiety disorders but without ASD (n = 342, M_age = 11.25 yrs, 246 male). Multigroup factorial invariance (MFI) using structural equation modeling indicated equivalent latent factors in youth with and without ASD on the MASC-C (metric invariance). However, the item means and covariances along with the relations among the factor scores were different for the youth with ASD (i.e., lack of evidence for scalar or structural invariance). The MASC-P data did not fit the measure’s established structure for either the ASD or TD group, and post-hoc exploratory factor analysis revealed a different factor structure in the ASD group. Findings suggest that the MASC-C may not measure identical constructs in anxious youth with and without ASD. Further research on the structure of the MASC in clinical samples is warranted.

Anxiety symptoms are among the most common, and often impairing, symptoms affecting children and adolescents (herein called youth unless distinctions need to be made) with Autism Spectrum Disorder (ASD) (van Steensel, Bogels, & Perrin, 2010; White, Oswald, Ollendick, & Scahill, 2009). Approximately 40% of youth with ASD have a comorbid anxiety disorder (van Steensel et al., 2010). The presence of comorbid anxiety compounds socialization difficulties and contributes to greater functional impairment (Chang, Quan, & Wood, 2012; Myles, Barnhill, Hagiwara, Griswold, & Simpson, 2001). However, it is unclear whether the manifestation of anxiety in youth with ASD is the same as anxiety seen in typically developing (TD) youth (Ollendick & White, 2012; Wood & Gadow, 2010). In other words, it is unknown if the underlying construct of anxiety in youth with ASD is equivalent to what is seen in TD youth. The purpose of the present study was to assess the structure of anxiety and its measurement in youth with ASD, compared to TD youth, using a commonly administered child- and parent-report measure of anxiety.

In their review of the presentation and classification of anxiety in ASD, Kerns and Kendall (2012) hypothesized that anxiety disorders can co-occur in people with ASD and concluded they are separable from ASD rather than being a phenomenon of ASD. They speculated that symptoms of anxiety in ASD may be atypical, or distinct, from anxiety as it presents in people without ASD. This presents a challenge for how to best conceptualize anxiety in people with ASD since, to be truly ‘comorbid’ some argue that the anxiety disorder should not only be separable from the ASD itself, but also resemble the disorder as it presents in people without ASD (Wood & Gadow, 2010). The requirement that phenotypic presentation of a disorder in comorbid form be identical to its presentation in isolation is valid, although historically it has been suggested that a pattern of comorbidity may in fact constitute a unique and meaningful syndrome, such that symptoms might manifest and respond (to treatment) differently in the comorbid syndrome than would be expected from either disorder in isolation (Caron & Rutter, 1991). Kerns and Kendall emphasized the need for the validation of measures with which to assess anxiety in people with ASD. In commentary, Ollendick and White (2012) remind us that there exists no such thing as a ‘valid measure;’ rather, the validity of any psychological measure is based on its intended use. Just because a measure has been established to be reliable and valid for measurement of anxiety in TD children, for instance, we cannot assume its reliability or validity for youth with ASD.

Considerable research has informed best practices for the assessment and diagnosis of anxiety disorders in children and adolescents (AACAP, 2007; Silverman & Ollendick, 2005), but there is little research on the degree to which commonly used measures of childhood anxiety are reliable and valid for use with youth who have ASD without comorbid intellectual disability. This situation is concerning given increasing recognition that anxiety is a common co-occurring problem in this population. Sensitive and valid measures of anxiety are imperative to the science of treatment development and outcome research (Scahill, 2012). Recently, efforts have been made to develop such measures. For example, Hallett and colleagues (2013) examined a 20-item anxiety scale (Sukhodolsky et al., 2008) that was adapted from the Child and Adolescent Symptom Inventories (CASI; Gadow & Sprafkin, 1998; 2002). Using data from a large sample (n = 445) of youth with ASD, Hallett et al. (2013) concluded that the CASI-Anxiety Scale measures a unique construct separate from ASD severity. In a recent structural equation modeling study, parent, child, and diagnostician reports of anxiety severity in children with ASD were found to correlate with each other at a level similar to that seen in the typical population. Moreover, youth who reportedly were more anxious were not consistently more severely affected by ASD (per clinician or parent reports), which is supportive of construct discrimination between anxiety and ASD (Renno & Wood, 2013).

To ascertain the nature of anxiety within ASD it is important to determine the degree to which the construct of anxiety and the psychometric properties of measures used to assess it show equivalence across youth with ASD and those without. Measurement invariance, or establishing that a given measure operates similarly in different clinical populations, is a prerequisite to establishing factorial invariance (i.e., that it is in fact the same latent variable across populations) (Ollendick & White, 2012). The latter cannot be presumed without the former. Measurement and factorial invariance are critical to evidence-based, valid assessment, yet they are often assumed rather than empirically established. If variance across clinical groups is present, the construct we seek to assess (and treat) in the population of interest cannot be equated with the construct it assesses in the originating (e.g., typically developing) population, which clearly has implications for treatment development and evaluation. Moreover, levels of invariance can be assessed using hierarchically organized factorial invariance analysis with progressively imposed observed and latent constraints (e.g., Byrne, 2004; Byrne, Shavelson, & Muthén, 1989; Gomez, Vance, & Gomez, 2012). No study to date has compared youth with ASD to TD youth with diagnosed anxiety disorders to explore the degree to which the underlying construct of anxiety and how it is assessed are psychometrically equivalent across these clinical populations.

The present study examined measurement and factorial invariance of anxiety across anxious youth with ASD and TD youth, using the Multidimensional Anxiety Scale for Children, parent report (MASC-P) and child self-report (MASC-C; Baldwin & Dadds, 2007; March et al., 1997; March, 1997). The MASC-C and MASC-P are among the most commonly used measures of anxiety in treatment trials of anxious children and adolescents, evincing strong psychometric properties and factorial structure as well as clinical efficiency (Villabø, Gere, Torgersen, March, & Kendall, 2012). Although parent-child concordance on the MASC is weak to moderate (Baldwin & Dadds, 2007), prior studies have found structural consistency between the MASC-C and MASC-P in TD children, with four correlated factors (Baldwin & Dadds, 2007; Grills-Taquechel, Ollendick, & Fisak, 2008). They are also among the most commonly used measures of anxiety in youth with ASD (Kreiser & White, 2014) and were used in the study of convergent and divergent validity of the anxiety construct within ASD mentioned above (Renno & Wood, 2013). We hypothesized that the MASC-P would show metric invariance (i.e. same latent factors, but different levels, and relations both to items and between the factors) across groups whereas there would be less evidence for invariance in the MASC-C data. Expectation of variance across groups in the youth self-reports stems from longstanding concerns about the ability of young people with ASD to introspect and report on their internal cognitive and emotional experiences leading to symptom underreporting, considerable discrepancies between self- and parent-report measures, a hallmark lack of insight into one’s own emotional experiences and thoughts, and alexithymia - or the inability to access and report on a range of emotions (e.g., Lainhart & Folstein, 1994; Lerner, Calhoun, Mikami, & De Los Reyes, 2012; Lopata et al., 2010; Mazefsky, Kao, & Oswald, 2011). In addition to these method-related factors, another potential source of variance is qualitative differences in the expression of anxiety in ASD (Kerns & Kendall, 2012).

Methods

Participants

A total of 465 children and adolescents (345 male; M_age = 11.25 yrs, SD_age = 2.75, range = 7–17.83) were included. Participants with ASD were aggregated across multiple sites from previous studies (see White, Ollendick, Scahill, Oswald, & Albano, 2009; White et al., 2013; Wood et al., 2009) to form two overlapping samples with data on the MASC-C (n = 82) and MASC-P (n = 109), respectively. The ASD samples (see Tables 1 and 2 for demographic information) were drawn from six previous intervention studies for anxiety in youth with ASD across two US sites (one rural, one urban). These participants all met criteria for ASD based on clinical judgment and confirmed by the Autism Diagnostic Observation Schedule (ADOS; Lord, Rutter, DiLavore, & Risi, 1999). As measures from these samples were recorded at baseline, intervention details are not relevant to the present study. The TD sample (see Tables 1 and 2) was drawn from a large, multisite intervention trial for TD youth with anxiety disorders (see Kendall et al., 2010). This previous trial included a nationally representative sample of 488 youth with anxiety disorders, which, while similar in age to the present aggregated ASD sample, diverged considerably from the 4:1 male to female gender ratio typically seen in people with ASD (Fombonne, 2005) and found in our sample. Thus, to mitigate the likely influence of gender differences on any obtained results, 146 female participants (representing 19.67% of the original TD sample) were randomly removed from the TD sample; this approach was used to ensure statistical gender correspondence to the ASD group, while maximizing the use of available participants in the TD sample. This yielded a subsample of TD participants with anxiety comprised of 342 youth (see Tables 1; 2), who were used as the comparison group for all subsequent analyses (henceforth referred to as the “TD sample”). All participants (both ASD and TD) met criteria for an anxiety disorder on the Anxiety Disorders Interview Schedule for Children/Parents, Fourth Edition (ADIS-IV-C/P; Silverman & Albano, 1996). Information on the types of anxiety disorders diagnosed in the ASD and TD samples are provided in prior reports (see White et al., 2009; White et al., 2013; Wood et al., 2009; and Kendall et al., 2010, respectively). Verbal ability was assessed using standardized measures such as the Wechsler Abbreviated Scales of Intelligence (WASI; Wechsler, 1999) VIQ, to ensure that all participants in both groups had intact verbal ability (i.e., Verbal IQ >70).

Table 1.

Demographic Characteristics of Gender-matched MASC-P ASD and TD Samples (N = 451)

Variable	ASD Sample Mean (SD)	TD Sample Mean (SD)	Between-group difference p-value1
Child Age	138.63 (29.91)	134.79 (34.39)	.296
VIQ2	97.77 (15.54)	101.30 (17.24)	.490
	ASD Sample (n = 109)	TD Sample (n = 342)	Between-group difference3
Male Gender	87 (79.82%)	246 (71.93%)	.103

ASD = Autism Spectrum Disorder. TD = Typically-Developing. VIQ = Intelligence Quotient.

¹p-values for continuous variables calculated via univariate ANOVA.

²Measured VIQ available for 22 ASD participants and 20 TD participants.

³p-values for categorical variables calculated via χ² test.

Table 2.

Demographic Characteristics of Gender-matched MASC-C ASD and TD Samples (N = 424)

Variable	ASD Sample Mean (SD)	TD Sample Mean (SD)	Between-group difference p-value1
Child Age2	142.66 (29.82)	134.79 (34.39)	.057
VIQ3	97.77 (15.54)	101.30 (17.24)	.490
	ASD Sample (n = 82)	TD Sample (n = 342)	Between-group difference3
Child Gender	64 (78.04%) male	246 (71.93%) male	.262

ASD = Autism Spectrum Disorder. TD = Typically-Developing. VIQ = Intelligence Quotient.

¹p-values for continuous variables calculated via univariate ANOVA.

²Child Age is given in months.

³Measured VIQ available for 22 ASD participants and 20 TD participants.

³p-values for categorical variables calculated via χ² test

Measures

Multidimensional Anxiety Scale for Children (MASC; March, 1997)

Originally developed as a child self-report measure, the MASC-C and MASC-P are each comprised of 39 items that yield subscale scores for harm avoidance, physical symptoms, social anxiety, and separation anxiety/panic. Items are answered on a 4-point Likert Scale ranging from never true about me to often true about me and total scores are treated as continuous datapoints. Both scales are widely used in research on childhood anxiety and possess strong internal consistency, retest stability, a reliable four-factor structure (i.e., physical symptoms, harm avoidance, social anxiety, separation/panic), evidence of convergent validity, and clinical sensitivity, making them psychometrically sound (e.g., Grills-Taquechel, Ollendick, & Fisak, 2007; March et al., 1997; March, Conners, et al., 1999; March, Sullivan, et al., 1999; Wood et al., 2002). There is support for the psychometric properties (e.g., adequate internal consistency, discriminant validity in that it assesses something other than ASD severity) of the MASC in child ASD samples (Renno & Wood, 2013; White, Schry, & Maddox, 2012), but there has been no published research on its equivalence compared to a non-ASD clinical group.

Data Analyses

We compared the ASD sample to the TD sample to assess differences on basic demographic factors using univariate ANOVAs for continuous variables and χ² test for categorical variables. These analyses were conducted in SPSS Version 21.0 (“IBM SPSS Statistics for Windows,” Released 2012). We examined missing values to determine patterns of missingness and evaluated the necessity of consequent analytic adjustments. Confirmatory factor analysis (CFA), using structural equation modeling (SEM) in AMOS Version 21.0.0 (Arbuckle, 2012; Byrne, 2004), employing the standard MASC factor structure was then conducted on the MASC-C and MASC-P in the gender-matched TD sample to ensure an adequate reference group to which to compare the ASD sample, and in the combined sample to ensure sufficient fit to assess invariance. The same CFA was then run on the ASD group as an initial examination of overall fit.

Model fit was estimated using absolute (χ²; Root-Mean-Square Error of Approximation [RMSEA]), and incremental (Comparative Fit Index [CFI]) and relative (Tucker-Lewis Index [TLI]) fit indices (standardized root mean square residual [SRMR] could not be calculated because of missing data). However, we note that thresholds for CFI and TLI are incremental fit indices useful only if the absolute fit of the null model to which the estimated model is being compared is sufficiently poor (e.g., RMSEA > .158; Kenny, 2014). Thus, prior to examination of estimated model fit, the corresponding null models were examined. For all of the estimated models, the corresponding null models exhibited RMSEA < .13; thus, suggesting that only RMSEA be used for interpretation (though CFI and TLI are reported for interested readers). Consistent with standard practice (Hu & Bentler, 1999), .05 was considered the threshold for “good fit” and .08 was considered for “poor fit.” These thresholds were evaluated via the RMSEA 90% confidence interval [CI], whereby the null hypothesis of RMSEA < .05 was tested using the lower bound, and failure to reject was taken as support for a good fit; the null hypothesis of RMSEA > .08 was tested via the upper bound, and failure to reject is taken as support for a poor fit. However, to ensure sufficient evidence that subsequent models could be tested for invariance, base models were required to meet “good fit” criteria in both the TD and combined models to proceed with invariance testing. If fit of the CFA met these criteria, multigroup factorial invariance (MFI) analyses were conducted. If not then, in consideration of the well-established MASC factor structure and the goal of generalizability to other samples, fit was deemed inadequate for subsequent testing, and no invariance analyses were conducted. Analyses were conducted using Full Information Maximum Likelihood estimation with estimated means, which is considered robust to Missing Completely At Random (MCAR) and Missing at Random (MAR) data under conditions of limited missingness.

The path diagram used to estimate the MFI models is shown in Figure 1. This diagram includes four latent factors (social anxiety; physical symptoms; harm avoidance; separation/panic; March et al., 1997), which were allowed to covary (March, Conners, et al., 1999). Specification of covariances across item errors was determined empirically by examining univariate correlations and specifying covariances only among those items with very high correlations and considerable overlap in item content (Byrne, 2004). For standardization, item error means were set to zero and their loadings were set to one; latent factor means were also set to zero. To facilitate estimation, the loading of one observed variable for each latent factor was set to one. All other parameters were estimated or constrained hierarchically, as described below. Degradation or improvement across nested models was compared by examining change in absolute (Δχ²) and relative (Incremental Fit Index [IFI]) fit.

Figure 1.

Baseline path diagram of the four-factor MASC model used in the MFI CFA. Error (co)variances not shown.

Between-group model comparison proceeded first with the cross-group baseline model constraining the latent factor means to 0 and allowing all other unspecified parameters to be freely estimated, which was estimated to consider overall fit of the factor structure across groups. Then, eight hierarchically nested MFI models were to be tested (see Figure 2) for the MASC data. First, a baseline model testing configural invariance was specified. This model employed the cross-group baseline model constraints, examining equivalence between ASD and TD samples; this tested the hypothesis that the same pattern of covariances, loadings, means, variances, and covariances exists in both groups (though these values may not necessarily be equal). Support for this model would address the primary question of construct equivalence at the most basic level, indicating the presence of similar, but not perfectly equivalent, latent factors on the MASC across ASD and TD youth with anxiety. If there was support for configural invariance, metric invariance (also called weak factorial invariance) was then tested; this model imposed the constraint that the loadings of observed items onto the latent factors were equal across groups. If metric invariance was supported, it would suggest that the same latent constructs were being measured across groups, and that differences in covariances of observed items could be attributed to latent factors (i.e., relationships between latent variables may differ across groups). This would permit additional pathways of constraint across groups at either the item or the structural level, addressing the more nuanced questions of how, in what ways, to what extent, and under what methodological conditions the construct of anxiety as measured on the MASC may be said to be equivalent across ASD and TD youth with anxiety.

Figure 2.

Hierarchy of MFI CFA models. + = parameter constrained equal across groups.

The additional models tested with constraints at the item level in the event that metric invariance was supported include scalar invariance (also called strong factorial invariance; differences in means and covariances of items were the result of differences in the corresponding latent parameters) and strict factorial invariance (identical item structure). Subsequent models with constraints at the factor level include structural invariance (latent factors relate to each other similarly across populations, indicating measurement of a stable overall anxiety construct), invariant latent variances (identical item structure and identical distribution of latent factors), invariant latent variances and covariances (equivalently distributed, and related, latent constructs are measured in ASD and TD groups), and invariant latent means models (in addition to identical item structure, the ASD and TD groups also obtain equivalent mean levels of the latent constructs). We note that the nested analysis hierarchy is nonstandard; we applied a hybrid approach in which both item- and factor-level constraint pathways were effectively conducted in parallel. This approach includes both minimally and maximally restrictive constraints (Byrne, 2004), offering a richer and potentially more precise approach to assessing factorial invariance, while maintaining benefits of the essential analytic structure of multigroup factorial invariance analysis.

Results

Preliminary and Descriptive Statistics

Examining the MASC-P samples, youth participants did not differ on age, gender, or Verbal IQ (see Table 1). The overall MASC-P total score demonstrated small to medium inter-item correlations between the ASD (M_{inter-item correlation (IIC)} = .19) and TD (M_{inter-item correlation (IIC)} = .15) samples. For the subscale scores, inter-item correlations were also small to medium, and were similar, in both ASD (Physical Symptoms M_IIC = .33; Harm Avoidance M_IIC = .18; Social Anxiety M_IIC = .33; Separation/Panic M_IIC = .34) and TD (Physical Symptoms M_IIC = .26; Harm Avoidance M_IIC = .22; Social Anxiety M_IIC = .43; Separation/Panic M_IIC = .31) samples. Missing data analyses revealed a maximum level of missingness at 2.9% (1 item), with all other items at < 2% missingness (M_missing = 1.2%, SD_missing = .45%) and a minimum level of missingness of .7% (7 items). Little’s test indicated data were not MCAR (χ²(1369) = 1522.53, p < .01). Investigation of the missingness pattern suggested that, for items with greatest percent missing data (i.e., ≥1.9%), the pattern could be attributable to site or sample differences; thus, the data could be said to be MAR. Since the overall level of missingness was limited, and since site and sample were fully nested within each diagnostic group, MAR effects were considered modeled and accounted for, and no model adjustments were made.

Examining the MASC-C data, the samples also did not differ on age, gender, or Verbal IQ (see Table 2). The overall MASC-C also demonstrated similarly small to medium inter-item correlations between the ASD (M_IIC = .22) and TD (M_IIC = .15) samples. For the subscale scores, inter-item correlations were also small to medium, and were similar, in both ASD (Physical Symptoms M_IIC = .27; Harm Avoidance M_IIC = .30; Social Anxiety M_IIC = .39; Separation/Panic M_IIC = .26) and TD (Physical Symptoms M_IIC = .32; Harm Avoidance M_IIC = .17; Social Anxiety M_IIC = .41; Separation/Panic M_IIC = .27) samples. Missing data analyses revealed a similar quantity of missing data on the MASC-C, with a maximum level of missingness at 2.1% (5 items), with all other items at <1.93% missinginess (M_missing = 1.7%, SD_missing = .24%), and a minimum level of missingness of 1.4% (14 items). Little’s test indicated data were MCAR (χ²(977) = 1032.50, p > .10), so no model adjustments were made. Correlations between raw total MASC-P and MASC-C scores were moderate for the overall sample (r = .32, p < .001), the anxious TD subsample (r = .29, p < .001), and the ASD subsample (r = .38, p = .08).

MASC-P Baseline Models

Upon examination of the univariate correlation structure, no MASC-P items were observed to be so highly correlated (i.e., > 0.65) that overspecification was of concern. Thus, no errors were permitted to covary in the baseline or any subsequent MASC-P models. The baseline model in the combined sample (χ²(696) = 2330.28, p < .001; RMSEA = .072 [CI: .069 – .075]; CFI = .707; TLI = .672), TD sample (χ²(696) = 1980.05, p < .001; RMSEA = .074 [CI: .070 – .077]; CFI = .704; TLI = .668), and ASD sample (χ²(696) = 1228.00, p < .001; RMSEA = .084 [CI: .076 – .092]; CFI = .642; TLI = .598) did not meet criteria for a good fit. Thus, subsequent invariance testing was not conducted.

MASC-C Baseline Models

Examining the univariate correlation structure, MASC-C items 10 and 22 were found to be highly correlated (r = .68, p < .001); additionally their content (worry about what peers will think) was found to evince substantial overlap. As such, covariation of error terms between this pair of items was considered justified, and was specified in the baseline and all subsequent MASC-C models (note that Figure 1 does not visualize this covariance term). The baseline model demonstrated good fit in the combined (χ²(695) = 1359.70, p < .001; RMSEA = .048 [CI: .044 – .051]; CFI = .838; TLI = .819) and TD (χ²(695) = 1260.07, p < .001; RMSEA = .049 [CI: .045 – .054]; CFI = .829; TLI = .808) groups, thus providing sufficient evidence to conduct MFI. In the ASD sample, fit was poor (χ²(695) = 1080.49, p < .001; RMSEA = .083 [CI: .073 – .092]; CFI = .653; TLI = .610).

Hierarchy of MASC-C CFA Models for MFI across Samples

Following the indicated hierarchy (see Figure 2), the additional CFA MFI models were tested, examining both model fit and difference in fit compared to the appropriate reference model (see Table 3). First, the configural invariance model evinced a good absolute fit. Next, the metric invariance model also evinced a good absolute fit; additionally, model comparison indicated no degradation (Δχ²) and a small improvement (IFI) in fit. Thus, metric invariance (i.e., that the same latent factors were measured across groups, with differences in observed means, variances, and covariances attributable to differing relations between the factors) was considered supported, and subsequent model fitting was conducted. The factor correlations within each group are reported in Table 4.

Table 3.

Results of Tests for Invariance in MASC-C across Participants with ASD or TD

	Model Fit					Model Difference
Models (M)	χ2	df	RMSEA [CI]	CFI	TLI	ΔM	Δdf	Δχ2	IFI
M1: Configural Invariance (Unconstrained)	2346.47***	1390	.040 [.038 – .043]	.784	.757
M2: Weak Factorial/Metric Invariance1	2391.78***	1425	.040 [.037 – .043]	.781	.761	M2 – M1	35	45.31	.010
M3: Strong Factorial/Scalar Invariance	2505.20***	1464	.041 [.038 – .044]	.765	.749	M3 – M2	39	113.42***	.025
M4: Structural Invariance	2417.63***	1431	.040 [.038 – .043]	.777	.757	M4 – M2	6	25.82***	.006

^***p < .001.

^**p < .01.

^*p < .05.

RMSEA = Root-Mean-Square Error of Approximation. CI = confidence interval. CFI = Comparative Fit Index. TLI = Tucker-Lewis Index. PCFI = Parsimony-Adjusted CFI. IFI = Incremental Fit Index.

¹Significant degradation in fit is seen after this model.

Table 4.

MASC-C Factor Correlations in Metric Invariance Model ASD and TD Samples

	1. Social Anxiety	2. Separation/Panic	3. Harm Avoidance	4. Physical Symptoms
1. Social Anxiety	1	.20**	.29***	.56***
2. Separation/Panic	.73***	1	.55***	.31***
3. Harm Avoidance	.37*	.73***	1	.27**
4. Physical Symptoms	.61***	.77***	.59***	1

ASD = Autism Spectrum Disorder. TD = Typically-Developing. Values above diagonal are in the TD sample, values below the diagonal for in the ASD sample.

^*p < .05,

^**p < .01,

^***p < .001.

Scalar (constraint at the item level) and structural (constraint at the factor level) invariance were tested next, with both evincing good absolute fit. However, they both demonstrated significant degradation in fit relative to metric invariance, so they were not considered supported and no further models are reported. Overall, the MFI analysis indicated that the same self-report latent anxiety factors were measured (metric invariance supported) on the MASC-C for both ASD and TD groups, though the relations between these factors likely differed. The factor loadings of the metric invariance model (which were equivalent across groups) are shown in Table 5.

Table 5.

Standardized Factor Loadings on the MASC-C in Metric Invariance Model (Equivalent across ASD & TD Groups)

Factor and item number	Standardized factor loading	Factor and item number	Standardized factor loading	Factor and item number	Standardized factor loading	Factor and item number	Standardized factor loading
Physical symptoms		Harm avoidance		Separation anxiety/ panic		Social anxiety
1	.50	2	.33	4	.63	3	.73
6	.50	5	.41	6	.49	10	.82
8	.63	11	.40	9	.68	14	.54
12	.58	13	.45	17	.45	16	.72
15	.45	21	.37	19	.58	22	.75
18	.44	25	.42	23	.42	29	.65
20	.64	28	.47	26	.53	33	.42
24	.53	32	.39	30	.39	37	.56
27	.61	36	.55	34	.53	39	.54
31	.53
35	.64
38	.55

Post-hoc analysis

EFA of MASC-P in ASD Sample

Due to the finding of lack of fit using the established MASC structure, we explored the whether there may be a plausible alternative structure for the MASC-P in the ASD sample. An EFA was conducted on the ASD sample, using Maximum Likelihood extraction, with oblique rotation pairwise deletion for missing data. Using Cattell’s (1966) scree plot criterion, a four-factor solution was found to best describe the data. These four factors explained a total of 42.86% of the variance in MASC-P scores (see Table 6), with comparable distribution of eigenvalues to that seen in previous MASC validation studies (see March et al., 1997). Three out of the four factors yielded similar item groupings to the original MASC structure, but with no harm avoidance scale and with the social anxiety-related items factoring onto two separate factors. The first factor was comprised largely of separation anxiety/panic-related items (e.g., scared when my parents are away; avoid going out without the family), as well as one harm avoidance item (letting someone know when upset). On the second factor, many social anxiety-related items related to humiliation/rejection loaded, and with uniformly negative loadings (i.e., low parent-observed social anxiety). The third factor represented physical symptoms, with all original subscale items except difficulty breathing. The fourth factor was comprised of the four social anxiety-related items that characterize performance anxiety.

Table 6.

Rotated Factor Loadings in Exploratory Factor Analysis of MASC-P in ASD Sample

Item #	Content	Communalities	Factor 1	Factor 2	Factor 3	Factor 4
			R2 = 20.29 E = 7.91	R2 = 9.42 E = 3.67	R2 = 7.81 E = 3.05	R2 = 5.34 E = 2.08
1	Tense	.31	0.17	−0.04	0.42	−0.01
2	Ask permission	.30	−0.03	0.12	−0.09	0.00
3	Worry laughed at	.77	0.07	−0.88	−0.01	0.02
4	Scared parent	.67	0.72	−0.04	0.08	0.17
5	Danger	.14	−0.16	0.04	0.34	0.16
6	Trouble breathing	.31	0.21	−0.05	0.11	0.06
7	Camp scares me	.41	0.56	−0.04	0.04	0.26
8	Shaky, jittery	.61	0.05	0.07	0.74	0.17
9	Near mom, dad	.46	0.56	0.04	−0.05	−0.11
10	Kids make fun	.74	0.07	−0.83	−0.07	0.12
11	Try to obey	.20	0.35	0.01	0.03	0.14
12	Dizzy, faint	.42	−0.13	0.00	0.66	0.06
13	Check things out	.46	0.17	0.01	0.12	0.19
14	Worry called on	.38	−0.07	−0.21	0.12	0.48
15	jumpy	.37	0.17	0.07	0.52	−0.09
16	afraid stupid	.55	−0.09	−0.70	0.08	0.12
17	Keep light on	.26	0.51	−0.06	0.01	−0.07
18	Pains in chest	.44	−0.04	−0.10	0.65	−0.14
19	Avoid w/o family	.58	0.74	−0.14	−0.03	0.08
20	Feel strange	.27	−0.11	−0.22	0.43	−0.04
21	Do things-like	.19	0.12	−0.21	−0.09	−0.15
22	Think of me	.63	0.12	−0.76	−0.02	−0.07
23	Avoid scary TV	.33	0.50	−0.06	0.01	0.04
24	Heart races	.53	0.26	−0.02	0.63	−0.17
25	Avoid upset	.14	0.25	−0.02	0.05	0.05
26	Sleep near	.26	0.37	−0.01	0.20	−0.10
27	On edge	.38	0.24	0.01	0.50	−0.22
28	Do things right	.29	0.21	−0.13	0.09	−0.10
29	Embarrassing	.59	−0.17	−0.57	0.17	0.11
30	Scared in car	.18	0.13	−0.11	0.29	0.11
31	Sick to stomach	.41	0.13	0.00	0.58	−0.15
32	Let others know	.38	0.45	0.20	0.17	−0.12
33	Public perform	.54	0.02	−0.14	−0.01	0.68
34	Weather/dark/bugs	.38	0.50	0.07	0.14	0.07
35	Hands shake	.49	0.06	−0.01	0.66	0.06
36	Safety	.55	0.13	0.00	0.06	0.21
37	Ask others	.29	0.21	0.03	−0.02	0.41
38	Hands sweat	.29	0.02	0.02	0.44	0.26
39	Feel shy	.52	0.14	−0.15	−0.07	0.66

Loadings derived from Maximum Likelihood estimation, with Oblimin rotation. Bold loadings > |.40|. Content = question content summarized (not verbatim). E = Eigenvalue. Communalities reported are post-extraction. Reported R² and eigenvalues derived from unrotated factor solution.

Discussion

We evaluated whether the MASC operated as expected within a sample of youth with co-occurring ASD and anxiety disorders relative to the established factor structure and compared to a gender-matched sample of TD youth with anxiety disorders. In the primary analyses, we found evidence for metric invariance but not structural or scalar invariance across the two groups. Although identical latent factors were observed, there were different factor relationships, latent means, and observed error variances across the two groups. Thus, in the ASD sample, the MASC-C data appeared to represent the established latent factors, but the levels and relations among those factors differed. Between-group model testing of the MASC-P could not be pursued because the TD and combined sample’s data did not conform to the established MASC structure.

The nature of the ASD sample (i.e., intact verbal ability, no cognitive impairment) may help explain the evidence in favor of metric invariance between the two groups for the MASC-C. The general factor structure originally established, with clusters of interrelated items representing separation anxiety and panic symptoms, physical anxiety symptoms, social anxiety symptoms, and harm avoidance symptoms evidenced reasonable correspondence between the two groups. This suggests that rather than a completely different, or random, pattern of anxiety clusters and subtypes in youth with ASD, a similar structure of subscales was replicable in both groups at a broad level. Moreover, the same latent factors emerge in the ASD group. Speculating as to why we do not find evidence for structural invariance, it is possible that these youth sometimes experience anxiety in different ways from their TD peers, consistent with emerging evidence for an atypical anxiety in ASD (e.g., Kerns & Kendall, 2012). Thus, although the same factors emerge, they do not relate to each other syndromally the same as seen in typically developing youth without ASD. Research is needed to understand what mechanisms contribute to the observed factor equivalence in the context of less evidence for ‘deeper’ structural invariance.

Prior psychometric research on assessment of anxiety in youth with ASD has generally found support for convergent validity of child ratings of anxiety based on level of agreement with parent-reports being comparable to that found in typical samples (e.g., Renno & Wood, 2013; White et al., 2012). Internal consistency on the MASC-C in this sample and others has also been acceptable. All the same, the limited evidence for higher-level factorial invariance, in concert with established concerns about the ability of youth with ASD to report accurately on symptoms because of issues such as poor insight, alexithymia, and language difficulties (e.g., Cook, Brewer, Shah, & Bird, 2013; Didehabni et al., 2012; Mazefsky et al., 2011; but also see Renno & Wood, 2013), suggest that the MASC-C may not be an ideal tool for assessment of anxiety – as this construct is typically understood in – young people with ASD when used in isolation.

Between-group model testing of the MASC-P was not pursued because no adequate baseline model could be found in either group. The finding that the latent structure of the MASC-P in both our ASD and TD samples did not fit the established structure of the measure was unexpected. When interpreting these findings, it is important to consider the nature of the samples (i.e., clinically referred, TD sample gender-matched to the ASD sample) used in this study. Although the parent-report items mirror those on the child self-report, they require subjective inference about the child’s inner experiences (e.g., child feeling uptight or tense). Unless the child has verbalized the specific experience to the parent, parents are left to infer internally experienced anxiety symptoms, leading to inherent imprecision of measurement. The cause of our inability to fit the baseline model for the MASC-P in the TD group is unknown, and may perhaps be due to sub-sample selection. However, full theoretical or empirical consideration of the issue of poor fit of the MASC-P to the established structure of the measure is beyond the scope of the current paper.

Using EFA, a 4-factor model was established for the ASD sample, which resembled item groupings of the original MASC-P structure for separation/panic and physical symptoms subscales, but with the social anxiety scale separated into two factors (one representing concerns about humiliation and rejection, and the other related to performance anxiety) and with no harm avoidance scale. Of note, compared to the other MASC-P subscales, the concurrent validity of the harm avoidance scale has been less clear in other research on the MASC-P in other studies of children with anxiety disorders (e.g., Wood, Piacentini, Bergman, McCracken, & Barrios, 2002). It is also possible that some manifestations of anxiety, such as anxious coping and the need to do things correctly (assessed by the harm avoidance scale), are rarely observed by parents of youth with ASD. If this is the case, we must consider the possibility that some youth with ASD might express anxiety in ways that are different from typically developing youth with anxiety (e.g., Kerns et al, in press). If anxiety manifests differently in ASD, there could be implications for assessment and intervention approaches in this population. The EFA conducted herein was with a fairly small sample of youth with ASD and, as such, caution in interpretation must be taken. Future studies should examine patterns of parent reporting of symptoms across youth with and without ASD in the context of Item Response Theory (IRT), Differential Item Functioning (DIF), and, perhaps, using EFA and CFA to consider further the possibility (and structure) of different latent constructs being observed in youth with ASD.

A limitation of the present study is the inability to sequentially examine if individual items of the MASC-C or MASC-P functioned differentially between the groups (i.e., DIF). Likewise, complete information on verbal ability was not available for all participants (though previous studies using these samples indicate intact verbal ability; Kendall et al., 2010; White et al., 2009; White et al., 2013; Wood et al., 2009). It is possible that some participants in both groups presented with subtle cognitive or verbal limitations that could influence results. Additionally, our findings might be strengthened by inclusion of a secondary measure of anxiety or a construct similar to anxiety (e.g., negative affect), so that we could evaluate the extent to which severity of pathology in this domain could affect structure and model fit. Nevertheless, all participants in both samples met diagnostic criteria for at least one anxiety disorder. Finally, following Byrne (2004), we specified item-level covariances among highly-correlated items on the MASC-C; while this was theoretically and empirically supported, it may also limit the replicability of the measurement model. Nevertheless, this study used a large, multi-site, well-characterized sample of TD youth with clinical anxiety problems and demographically comparable sample of youth with ASD. Future studies would do well to extend such sampling to other measures (and informants) of anxiety while examining differences across populations at the item level and potential moderators of effects, to determine potential sources of increased invariances across these populations (i.e., subgroups or items for which the latent constructs may evince increased equivalence). Replication of these findings in other large samples of youth with anxiety both with and without ASD is surely warranted.

These limitations notwithstanding, this analytic approach can aid in understanding how, in what ways (e.g., at the measurement level vs. the latent factor level), to what extent (i.e., how rigidly), and under what conditions (i.e., constraints of equivalence) such constructs can be said to be the same across groups. To this end, this study’s findings complement a growing body of research on the nature of anxiety in ASD, some of which suggests that anxiety is separable from ASD and not simply emblematic of ASD itself (e.g., Gadow, DeVincent, & Schneider, 2008; Gadow, Roohi, DeVincent, Kirsch, & Hatchwell, 2009; Renno & Wood, 2013).

The mixed results with respect to both child self-reports and parent-reports do not allow us to definitively ascertain the degree to which comorbid anxiety in ASD is a true comorbidity as assessed by the MASC, if one requires that a secondary or comorbid condition should tightly resemble the condition (e.g., the anxiety disorder) as it presents in people without ASD (e.g., Wood & Gadow, 2010). In this study anxiety was conceptualized syndromally and measured continuously (with the MASC), not diagnostically. Nevertheless, the lack of higher-order invariance, despite surface-level similarities (metric invariance), seen in these results supplements recent research indicating that anxiety disorders in youth with ASD may sometimes manifest in an atypical way compared to anxiety in TD youth (e.g., Kerns et al., in press).

Further study is warranted to specify the nature of any actual phenotypic differences in anxiety manifesting in youth with ASD, as well as possible differences in determinants and mechanisms of anxiety in this population, all of which might inform treatment. Indeed, there is a growing body of research showing considerable promise for cognitive-behavioral treatments for anxiety specifically developed for children and adolescents with ASD (e.g., Reaven, Blakeley-Smith, Culhane-Shelburne, & Hepburn, 2012; White et al., 2013; Wood et al., 2009). Such individualization of treatment for anxiety in ASD, in light of the possibility that anxiety may present differently in youth with ASD, may be necessary for optimal outcomes.

This study is the first to examine construct equivalence of anxiety as measured by a well-established parent- and child-report measure across TD and ASD samples. Given our findings of metric but not scalar or structural invariance across samples, it cannot be assumed that the MASC-C measures the construct of anxiety in youth with ASD exactly as it does in youth without ASD. Because these results indicate that the MASC is used somewhat differently by youth with ASD and those without ASD, we encourage caution in its use with ASD samples. That is, the lack of complete psychometric equivalence suggests that anxiety in youth with ASD may differ in certain ways from anxiety in youth without ASD. Until further research on the expression and valid measurement of anxiety in youth with ASD is available, clinicians should exercise caution in their adoption and interpretation of the MASC, and other such anxiety measures developed for typically developing youth, with clients who have ASD.

Highlights.

• Equivalent latent factors are apparent across youth with and without autism spectrum disorders who also have clinical anxiety disorders, based on youth self-report MASC data.

• Parent-reported MASC data, from our gender-matched typically developing sample, did not conform to the established structure of the instrument, prohibiting between-group fit comparison.

• Anxiety may manifest differently in youth with ASD. As such, caution should be taken in interpreting MASC scores, from child- or parent-report, in youth who have ASD.