Abstract
Background:
Anxiety is highly prevalent in autistic adults and can cause a significant impact on functioning and quality of life. There are no existing measures of anxiety designed and validated specifically for autistic adults. In this study, we aimed to adapt an existing anxiety measure designed for autistic children, in collaboration with autistic adults and other professionals, to make it suitable for use for autistic adults and to examine the measurement properties of the newly designed measure.
Methods:
Through consultation with autistic adults and professionals working with autistic people, we developed a preliminary self-report anxiety measure. Five hundred fifty-one autistic adults completed the measure and another measure of anxiety and depression (the Hospital Anxiety and Depression Scale). Participants completed measures again after 1 month to determine test/retest reliability. We split the sample to enable exploratory and confirmatory factor analyses to be undertaken on different samples. We conducted internal consistency and convergent and divergent validity analyses. We completed receiver operator curve (ROC) analyses to investigate sensitivity and specificity and identify an indicative clinical cutoff.
Results:
Our findings indicate that the Anxiety Scale for Autism-Adults (ASA-A) has promising psychometric properties. Factor analysis indicated that a bifactor solution with orthogonal general and specific factors was an adequate fit and that minimal measurement bias would occur if the scale were treated as unidimensional, so the total score could be used as a valid measure of anxiety. We identified a General Anxiety factor and three group factors (Social Anxiety, Uncertainty, and Anxious Arousal). ROC analysis indicated a score of 28 could be considered an indicative clinical cutoff.
Conclusion:
The ASA-A is the first self-report anxiety questionnaire specifically developed and validated for autistic adults. Preliminary evaluation of the measurement properties indicates that the scale will be a useful tool in research and clinical contexts.
Lay summary
Why was this study done?
Autistic adults commonly experience anxiety, which can have a huge impact on their quality of life. One way of identifying anxiety is by using questionnaires. Research shows that anxiety may be experienced differently by autistic people, and so, questionnaires that have been created for neurotypical adults may not be suitable for autistic adults. At present, there are no anxiety measures that have been adapted specifically for autistic adults.
What was the purpose of this study?
Our team previously created an anxiety questionnaire for autistic children—the Anxiety Scale for Children-Autism Spectrum Disorder (ASC-ASD). We aimed to adapt this questionnaire to make it suitable for autistic adults.
What did the researchers do?
We met with autistic people and professionals, and asked them how to change the ASC-ASD to make it suitable for measuring anxiety in autistic adults. We discussed whether the questions were appropriate for adults, if the questions were worded clearly, and if anything was missing. We then created the Anxiety Scale for Autism-Adults (ASA-A).
What did we find?
Once the questionnaire was created, 551 autistic adults with anxiety completed it. We used this information to test whether the questionnaire was a good way of measuring anxiety. We also tested the questionnaire subscales, which were as follows: Anxious Arousal, Social Anxiety, and Uncertainty. We found that the questionnaire is useful in giving a total score for anxiety, and we found that a score of 28 indicated anxiety that was likely to have an impact on someone's daily life. Each subscale was also found to be useful for providing a “profile” of anxiety.
What do these findings add to what was already known?
These findings suggest that the ASA-A is a good way of measuring anxiety in autistic adults. This is the first anxiety measure to be developed for autistic adults. This study is the first step toward testing out the usefulness of the measure. We hope that our questionnaire can be further tested and that in the future it will help to identify anxiety experienced by autistic adults more accurately. This would mean that the results of future studies are more likely to be valid, which should help improve what we know about the anxiety autistic people experience.
What are potential weaknesses in the study?
We used another anxiety measure to measure anxiety levels when testing what the cutoff score should be on our measure. A clinical interview may have been a more accurate way, and should be done in the future to check that the cutoff of 28 on the ASA-A is still appropriate for identifying anxiety in autistic adults.
We recruited autistic adults from a database of people who are interested in taking part in research. This helped us to get enough people to test the questionnaire, but we cannot tell how well these results apply to all autistic people. Testing the questionnaire with other autistic people, including those who may need some assistance with reading or who may need someone else to complete it on their behalf will let us know.
How will these findings help autistic adults now or in the future?
The final questionnaire, the ASA-A, has been shown to accurately measure anxiety in autistic adults who do not have an intellectual disability. The questionnaire will be helpful in both clinical and research services.
Keywords: adults, ASD, autism, anxiety, scale, self-report
Introduction
Autistic adults experience anxiety at significantly higher rates than the general population,1 with the pooled estimate of current anxiety in autism spectrum condition (ASC) reported to be 27% (lifetime prevalence 42%)2 compared with a general population pooled prevalence of 10.6% (lifetime prevalence 16.6%).3 The reported prevalence of anxiety in autistic adults varies between studies (5%–77%).4,5 The wide variation in rates of anxiety in autism may be due to an autism-related presentation of anxiety not captured by tools validated with neurotypical people.6–10
A recognition of the need for autism-specific measures of anxiety has recently led to the development of two pediatric tools, the ASC-ASD and the Parent Rated Anxiety Scale for Youth with Autism (PRAS-ASD).11 The ASC-ASD includes both parent and child self-report version, whereas the PRAS-ASD is parent report only. The ASC-ASD is an anxiety measure validated specifically for autistic children.12 Studies report autistic children with a diagnosis of anxiety scored significantly higher on the scale than those with no diagnosis, indicating a good level of sensitivity.13,14 The ASC-ASD is now used widely in research and clinical services and has been translated into a number of languages. Currently there are no autism-specific anxiety measures for autistic adults.
The measurement properties of only one anxiety measure, the Hospital Anxiety and Depression Scale (HADS),15 have been investigated with autistic adults.16 The HADS is a general population measure that does not include autism-specific items, and so may miss some important aspects of anxiety experienced by autistic people. Based on the evidence that there may be a differential presentation of anxiety in autistic adults,6,10 there is a need to create an anxiety measure specifically for autistic adults, including items underpinned by existing knowledge of how anxiety presents in ASC, to ensure that anxiety is accurately identified. The purpose of the current study was therefore to develop and provide preliminary validation of the measurement properties of a self-report anxiety measure specifically developed for and validated with autistic adults. The content of the tool has been derived from previous work on the ASC-ASD and includes autism-related anxiety items, which are not included in general population measures, such as the HADS.
The aims of this study were to:
Modify the ASC-ASD through consultation with autistic adults and professionals working with autistic adults.
Pilot the new measure with autistic adults to investigate its measurement properties.
Methods
Stage 1
We used a consultation design to adapt the ASC-ASD12 to create a preliminary measure. Before the consultation, we altered the wording of items to make them more appropriate for adults.
Consultation
We recruited six professionals with academic or clinical experience working with autistic adults (two males). We recruited six autistic adults (two females) through autism services, social media, and by contacting individuals previously involved in research. Eligibility criteria included age 18 years or older, a diagnosis of ASC, and no intellectual disability.
We provided participants with the 24-item original measure and asked them to complete three tasks; were the items recognizable as a feature of anxiety; was the wording clear; and was anything relevant not included? Next we summarized the data and refined the measure, including adding 6 additional items. We removed one original item, as it was not felt to be appropriate for adults. This resulted in a 29-item preliminary measure for Stage 2. Items are rated on a 4-point Likert scale, from 0 (“never”) to 3 (“always”).
Stage 2
Participants
We collected data for this study as part of a wider project regarding anxiety in autistic adults. Participants were members of the Autism Spectrum Cohort-UK (ASC-UK), a cohort of autistic adults recruited to a longitudinal research study (https://research.ncl.ac.uk/adultautismspectrum). We invited members of the cohort to participate if they had experienced anxiety and had the capacity to provide informed consent. Full demographic characteristics of the overall sample and subsamples can be found in Table 1.
Table 1.
Demographic Information for the Overall Sample and Each Subsample
| Demographic | Overall sample (N = 505) | EFA sample (n = 193) | CFA sample (n = 312) | Test/retest sample (n = 426) |
|---|---|---|---|---|
| Age | ||||
| Range | 17–77 | 17–77 | 18–76 | 16–77 |
| Mean (SD) | 41.4 (13.7) | 41.4 (13.6) | 41.5 (13.9) | 42.63 (13.9) |
| Gender, n (%) | ||||
| Female | 278 (55.0) | 101 (52.3) | 177 (56.7) | 222 (52.1) |
| Male | 210 (46.6) | 83 (43.0) | 127 (40.7) | 189 (44.4) |
| Other | 15 (3.0) | 7 (3.6) | 8 (2.6) | 11 (2.6) |
| Rather not say | 2 (0.4) | 2 (1.0) | 0 (0.0) | 4 (0.9) |
| Ethnicity, n (%) | ||||
| White | 468 (92.6) | 180 (93.2) | 288 (92.3) | 387 (91.1) |
| Black | 1 (0.2) | 0 (0.0) | 1 (0.3) | 1 (0.2) |
| Asian | 1 (0.2) | 1 (0.5) | 0 (0.0) | 2 (0.4) |
| Mixed race | 9 (1.8) | 3 (1.6) | 6 (1.9) | 9 (2.1) |
| Other/rather not say | 26 (5.2) | 9 (4.7) | 17 (5.5) | 27 (6.2) |
| ASD diagnosis, n (%) | ||||
| Formal diagnosis | 457 (90.5) | 175 (90.7) | 362 (93.4) | 382 (89.7) |
| Self-diagnosed/awaiting diagnosis/other | 48 (9.5) | 18 (9.3) | 30 (7.6) | 44 (10.3) |
| Mean SRS-2 total score (SD)a | 106.5 (24.0) | 105.6 (24.7) | 106.5 (23.7) | 106.2 (23.9) |
| Support completing questionnaires when registered with ASC-UK? n (%) | ||||
| Without support | 432 (85.5) | 162 (83.9) | 270 (86.5) | 370 (86.9) |
| With support | 68 (13.5) | 31 (16.1) | 37 (11.9) | 51 (12.0) |
| Missing response | 5 (1.0) | 0 (0.0) | 5 (1.6) | 5 (1.1) |
| Employment, n (%) | ||||
| Paid employment or retired | 297 (58.8) | 115 (59.6) | 197 (63.1) | 225 (52.8) |
| Not in paid employment | 208 (41.2) | 78 (40.4) | 115 (36.9) | 201 (47.2) |
| Highest qualification, n (%) | ||||
| GCSE or below | 153 (30.3) | 59 (30.6) | 93 (29.8) | 128 (30.0) |
| Postschool qualification (e.g., A-level) but not degree | 135 (26.7) | 54 (28.0) | 82 (26.3) | 116 (27.2) |
| University education (e.g., bachelors or above) | 217 (43.0) | 80 (41.4) | 137 (43.9) | 182 (42.8) |
| Anxiety diagnoses (self-reported), n (%) | ||||
| Formally diagnosed | 355 (71.3) | 128 (66.3) | 227 (72.8) | 230 (54.0) |
| Suspected | 150 (29.7) | 65 (33.7) | 85 (27.2) | 196 (46.0) |
Raw scores ≥68 on the SRS-2 are indicative of clinically significant autism characteristics associated with functional impairment. The mean SRS-2 scores in the overall sample and each subset are in the borderline range between “moderate” and “severe” autism characteristics and functional impairment.
ASC-UK, Autism Spectrum Cohort-UK; CFA, confirmatory factor analysis; EFA, exploratory factor analysis; GCSE, General Certificate of Secondary Education: SRS-2, Social Responsiveness Scale-Version2.
Procedure
We provided information sheets, measures, and consent forms to participants either by post or online, depending on preference. We contacted those who consented 1 month later and asked them to repeat some of the measures, to allow evaluation of test/retest reliability.
Measures
In addition to the Anxiety Scale for Autism-Adults (ASA-A), participants completed the HADS.15 The HADS is a self-report measure consisting of 14 items; 7 related to anxiety and 7 to depression. Respondents indicated how often they have felt a certain way during the past week, with responses ranging from “never” to “all of the time.” We omitted participants from analyses if over 10% of the items were missing. If 10% or less, we replaced the missing values with the mean of that participant's subscale score.
Analyses
Factor analysis (FA) and item response theory (IRT) are two types of models used for scale analysis. In the current study, we opted to use the more frequently used method of FA. Certain variants of FA and IRT are equivalent. We estimated our model using polychoric correlations, which is reported to be equivalent to using a two-parameter normal-ogive IRT model. We deleted participants with missing data listwise before FA (n = 46), as prorating the missing items at an early stage of measure development may bias results. We retained four outliers on anxiety scores to maximize sample size.
There is debate about the most appropriate way of determining a suitable sample size for exploratory factor analysis (EFA); however, we considered the 6.5:1 participant-to-item ratio adequate.17 A commonly used guideline for confirmatory factor analysis (CFA) sample size is 5–10 participants per item.18
Using FACTOR software,19 we conducted EFA on the ASA-A data to determine the factor structure. We used polychoric correlations due to the use of ordinal data.20 We used unweighted least squares to extract the factors as kurtosis was present at item level21 and the rotation method was Promin. Using an oblique method of rotation allows for correlation among factors, which would generally be expected in scientific research.22 Promin is a superior method in the case of complexity.23
We calculated the internal consistency of the ASA-A total and subscale scores using Cronbach's alpha. We assessed test/retest reliability by repeating the ASA-A and HADS-Anxiety factor (HADS-A) after 1 month and reporting Spearman's rank correlations. We used nonparametric tests due to ordinal scores. We assessed convergent and divergent validity by examining the correlation between the ASA-A and HADS-A and HADS-Depression factor (HADS-D), respectively. Although depression and anxiety are highly correlated in autism,24 we anticipated that ASA-A scores would have significantly higher correlations with HADS-A than HADS-D.
We conducted CFA using AMOS software25 to cross-validate the fit of the factor-structure against the EFA group scores. We input data using a polychoric matrix26 and correlated factors to reflect the Promin rotation method. We examined a number of model-fit indices to ensure a comprehensive evaluation of the model,27 including the chi-square/degrees-of-freedom ratio (χ2/df), root mean square error of approximation (RMSEA), Comparative Fit Index (CFI), and Tucker/Lewis Index (TLI). We covaried modification index values higher than twenty from the factor28 to explore if this would improve fit.
Finally, we undertook a receiver operator curve (ROC) analysis to determine whether the total ASA-A score demonstrated sensitivity and specificity against total HADS-A scores (cutoff score of eight to identify caseness).15 We also used the ROC analysis to determine an indicative anxiety cutoff score for the ASA-A.
Ethics
We received a favorable ethical approval from Wales REC 5 (Ref. 18/WA/0014).
Results
We contacted 1127 potential participants and invited them to participate. Five hundred fifty-one participants consented, of whom 505 participants provided full data on the ASA-A. We divided these individuals into two subsamples (at a ratio of 40:60), stratified by age and gender to control for any demographic differences and then allocated into either the EFA group (n = 193) or CFA group (n = 312). This enabled us to undertake the two factor analyses on different samples, a more robust method of analysis. Four hundred twenty-six of these participants consented to be recontacted after 1 month (the retest sample).
Factor analysis
Step 1: Before FA, we removed one question from the data set—“I would feel anxious if I had to stay away from home overnight because I like to be close to people I am familiar with.” This item had the highest number of missing responses. The provisional measure, the ASA-A, utilized in the analyses consisted of 28 items.
Dunn's pairwise tests indicated differences between the scores of males and females (p < 0.001, adjusted with Bonferroni correction). The median score for females = 56, males = 48. We found a weak negative correlation with age (r's = −0.131, p = 0.003). Therefore, for allocation to the EFA and CFA subgroups, we stratified participants by age and gender at a split of 40:60 to provide sufficient numbers for EFA and CFA.
A chi-square goodness-of-fit indicated no significant difference in gender, χ2 (1, N = 493) = 0.568, p = 0.457, or age quartile groups, χ2 (3, N = 493) = 0.163, p = 0.984, between the EFA and CFA groups. An independent samples t-test displayed no significant difference in age between groups, t (491) = 0.091, p = 0.492. A Mann–Whitney U-test demonstrated no significant difference in total anxiety scores between groups (U = 27198.500, p = 0.320). We found a very good sampling adequacy for both EFA and CFA groups (Kaiser-Meyer-Olkin [KMO] = 0.90 and KMO = 0.91, respectively).
Step 2: We used a number of techniques to determine the number of factors to be retained.21 Parallel analysis indicated that two or three factors should be retained, and an inspection of the scree plot supported three factors. The ASC-ASD consisted of four factors, and therefore, it was appropriate for us to also test the four-, five-, and six-factor solutions.
We applied a loading criterion of ≥0.450 to the rotated loading matrices. If this is too high, it reduces reproducibility of the results, and too low may result in nonmeaningful “rubble” factors.29 We found no items loaded onto the sixth factor of the six-factor solution, and this was discarded. The five-factor structure resulted in two cross-loading items, and was not theoretically sound. Only two items loaded onto the fourth factor in the four-factor solution. The three-factor solution was the strongest solution, taking into account parallel analysis and the scree plot, and made theoretical sense.
We then repeated the EFA, omitting six questions that did not load onto any factor on the three-item solution. As well as a three-factor solution, we tested a four-factor solution. When these questions were deleted, 12 participants no longer had missing data, and we reallocated them to the EFA and CFA groups using the SPSS random allocation function. EFA indicated that no items loaded onto the fourth factor, and that the three-factor solution provided the best fit. Inspecting the three-factor solution rotated loading matrix indicated two questions now cross loaded onto two factors, and did not meet the loading criterion threshold. We omitted these items, and the final EFA using a three-factor solution resulted in all items loading ≥0.450 (Table 2). The new scale consisted of 20 items; 17 of which were from the ASC-ASD and three new items. We used the content of each factor to define the factors as follows: Uncertainty, Anxious Arousal, and Social Anxiety.
Table 2.
Final Three-Factor Solution Rotated Loading Matrix for Anxiety Scale for Autism-Adults
| Item | Factor 1 U | Factor 2 A | Factor 3 SA |
|---|---|---|---|
| 7. I am anxious about unfamiliar things, people, or places (U) | 0.691 | −0.011 | 0.195 |
| 9. I worry about changes to my surroundings or my routine (U, N) | 0.734 | 0.015 | 0.000 |
| 11. When I don't know what will happen, I can't do things (U) | 0.586 | 0.170 | −0.033 |
| 13. I need to be prepared before things happen (U) | 0.865 | −0.076 | −0.098 |
| 18. I worry if I don't know what will happen, e.g., if plans change (U) | 0.877 | −0.036 | −0.046 |
| 1. All of a sudden I feel really scared (A) | −0.034 | 0.685 | −0.002 |
| 3. My heart suddenly starts to beat too quickly (A) | −0.163 | 0.770 | 0.019 |
| 6. I suddenly feel as if I can't breathe (A) | −0.107 | 0.884 | −0.042 |
| 8. When I have a problem, I feel physically shaky (A) | 0.182 | 0.536 | 0.070 |
| 15. I worry that something awful will happen to someone I am familiar with (A) | 0.042 | 0.528 | 0.061 |
| 16. I suddenly start to physically shake or tremble (A) | −0.071 | 0.795 | 0.003 |
| 17. I suddenly become dizzy or faint (A) | 0.082 | 0.691 | −0.134 |
| 19. I worry that something bad will happen to me (U) | 0.064 | 0.574 | 0.075 |
| 20. I suddenly shut down and become unable to think, speak, or do things (A, N) | 0.146 | 0.547 | −0.002 |
| 2. I worry what other people think of me (SA) | −0.374 | 0.030 | 0.890 |
| 4. I feel anxious in situations where I could make a mistake (SA) | 0.259 | 0.041 | 0.573 |
| 5. I worry that I will do badly when I am being assessed (like work, college, interviews) (SA) | 0.085 | −0.045 | 0.749 |
| 10. I worry about making mistakes during social interactions (SA, N) | 0.159 | −0.015 | 0.689 |
| 12. I worry when I think I have not done as well as I hoped (SA) | 0.195 | −0.023 | 0.581 |
| 14. I feel anxious that I will make a fool of myself in front of people (SA) | −0.131 | 0.038 | 0.857 |
Bold values indicate allocated Factor.
A, Anxious Arousal; ASA-A, Anxiety Scale for Autism-Adults; N, new item; SA, Social Anxiety; U, Uncertainty.
Step 3: We then tested the three-factor solution using CFA.27 The χ2/df was 4.27, whereas scores close to 0 demonstrate a strong model fit. RMSEA was over the maximum threshold of 0.8, and the CFI and TLI were both under the minimum threshold of 0.9.27 After five large error terms were covaried, the indices still demonstrated a poor fit. There is debate surrounding whether it is appropriate to covary within-item errors, but we felt it was appropriate for the present study due to similarities in the item content.30
We tested further models (Table 3), including a unifactor solution and a bifactor solution. When three large modification indices were constrained, the bifactor model demonstrated an adequate fit (χ2/df = 2.74, RMSEA = 0.075, CFI = 0.931). Remaining modification indices did not show any high values, and only two standardized residual covariances were high. We chose the bifactor solution as the most suitable model (Fig. 1). It has been argued that psychological assessments should be modeled using a bifactor framework, as measures of psychological phenomenon typically include a number of items measuring how a specific trait manifests in a number of different domains. This is coherent with a bifactor model, which is when the item variance can be sectioned into one general factor, that is, anxiety, and multiple group factors, that is, Anxious Arousal.31
Table 3.
Fit Indices for Anxiety Scale for Autism-Adult Factor Models
| χ2 | df | χ2/df | RMSEA | CFI | TLI | |
|---|---|---|---|---|---|---|
| 3 factor correlated model | 713.743 | 167 | 4.274 | 0.103 | 0.852 | 0.831 |
| 3 factor correlated model (5 covaried error terms) | 514.626 | 162 | 3.177 | 0.084 | 0.904 | 0.888 |
| Unifactor | 1507.673 | 170 | 8.868 | 0.159 | 0.637 | 0.594 |
| Bifactor | 553.853 | 150 | 3.692 | 0.093 | 0.890 | 0.861 |
| Bifactor (3 covaried error terms) | 402.212 | 147 | 2.736 | 0.075 | 0.931 | 0.910 |
CFI, Comparative Fit Index; RMSEA, root mean square error of approximation; TLI, Tucker/Lewis Index.
FIG. 1.
Bifactor model of the ASA-A. Ellipses represent latent variables, and rectangles represent measured variables; errors are not shown in the diagram but were specified for each variable. Error covariances between Q5–12 ( = 0.487) and Q15–19 ( = 0.312) on the Anxious Arousal group factor and Q8–16 ( = 0.452) on the Social Anxiety group factor are not shown, but were included in the analysis. ASA-A, Anxiety Scale for Autism-Adults.
The majority of items on the ASA-A had factor scores above the threshold of 0.450 on either the General Anxiety factor or one of the group factors. The factor loadings of item six, “I suddenly feel as if I can't breathe,” were 0.443 on the General Anxiety factor, and 0.628 on the Anxious Arousal group factor. For item two, “I worry what other people think of me,” the factor loading was 0.366 for the General Anxiety factor and 0.644 for the Social Anxiety group factor. These two items may therefore be better predictors of the group factors rather than the General Anxiety factor. Finally, the factor scores for item 15, “I worry that something awful will happen to someone I'm familiar with,” were 0.392 for the General Anxiety factor, and 0.235 for Anxious Arousal group factor. This suggests that it is a weaker predictor of both factors, compared with the other items.
Reliability and validity
Reliability and validity results are shown in Table 4. We undertook further analyses relevant to a bifactor model using the Bifactor Indices Calculator (Table 5).32
Table 4.
Reliability and Validity Indices of the Anxiety Scale for Autism-Adults and Hospital Anxiety and Depression Scale
| Scale | Cronbach's alpha | ||
|---|---|---|---|
| Internal consistency | ASA-A | General Anxiety factor (20 items) | 0.899 |
| Uncertainty group factor (5 items) | 0.834 | ||
| Anxious Arousal group factor (9 items) | 0.845 | ||
| Social Anxiety group factor (6 items) | 0.847 | ||
| HADS | Full scale | 0.867 | |
| Anxiety subscale | 0.819 | ||
| Depression subscale | 0.819 |
| Scales | Spearman's rank correlation | ||
|---|---|---|---|
| Test/retest reliability (1 month) |
ASA-A (20 items) |
0.823 (n = 407)a |
|
| HADS-A |
0.739 (n = 402)a |
||
| Convergent validity |
ASA-A vs. HADS-A |
0.696 (n = 415)a |
|
| Divergent validity |
ASA-A vs. HADS-D |
0.474 (n = 414)a |
|
| HADS-A vs. HADS-D | 0.515 (n = 420)a |
Correlation is significant at 0.01 level (one-tailed).
HADS, Hospital Anxiety and Depression Scale; HADS-A, Hospital Anxiety and Depression Scale-Anxiety factor; HADS-D, Hospital Anxiety and Depression Scale-Depression factor.
Table 5.
Coefficient Omega and Omega Hierarchical of the Anxiety Scale for Autism-Adults
| Coefficient omega | |
| General Anxiety factor (20 items) | 0.949 |
| Uncertainty group factor (5 items) | 0.906 |
| Anxious Arousal group factor (9 items) | 0.845 |
| Social Anxiety group factor (6 items) | 0.907 |
| Omega hierarchical | |
| General Anxiety factor (20 items) | 0.733 |
| Uncertainty group factor (5 items) | 0.234 |
| Anxious Arousal group factor (9 items) | 0.884 |
| Social Anxiety group factor (6 items) | 0.395 |
The General Anxiety factor accounted for 59.5% of common variance, indicating a strong general anxiety factor. The Anxious Arousal group factor accounted for 17.2%, the Social Anxiety group factor accounted for 15.0%, and the Uncertainty group factor accounted for 8.5% of common variance. The percentage of uncontaminated correlations (PUC) indicates the percentage of covariance terms that reflect only variance of the General Anxiety factor. This was 67.9%, which is below the 80% cutoff that would indicate that relative bias was less than 5%.33 However, when PUC is lower than 80%, the general explained common variance value of 0.60 and Omega hierarchical of 0.773 for the General Anxiety factor would indicate that the multidimensionality is not extreme enough to prevent the scale being used as a unidimensional instrument.34 The absolute relative parameter bias (ARPB), comparing the multidimensional bifactor model against the unidimensional unifactor model, was 13.5%. ARPB <10%–15% is acceptable35 supporting that the scale can be used as a unidimensional tool to measure total levels of anxiety.
The total score correlated highly with HADS-anxiety and significantly higher than with HADS depression, (Z = 6.190, p = 0.000), indicating convergent and divergent validity, respectively. ROC analyses using the anxiety cutoff on the HADS were undertaken. The area under the ROC was 0.900 (confidence interval [95% CI] 0.852–0.948) indicating excellent discrimination.36 Sensitivity was 0.851 (95% CI 0.811–0.883) and specificity was 0.795 (95% CI 0.645–0.892) with agreement of 77.10% between participants who scored above the clinical threshold on HADS-A and ASA-A total scores. Based on these analyses, the ASA-A cutoff that maximizes both sensitivity and specificity in detecting anxiety is a score of 28.
Discussion
Our study examined the measurement properties of the first self-report anxiety questionnaire specifically designed for autistic adults. We adapted the ASC-ASD in collaboration with professionals and autistic adults to create the ASA-A, which we found to have promising psychometric properties. Our preliminary evaluation indicates that the scale will be a useful tool in research for autistic adults who can complete a self-report questionnaire. The ASA-A is based on an upward extension of the ASC-ASD, which was derived from extensive research on the presentation of anxiety in autism, largely with children and their parents. At present, there is a paucity of research on the presentation of anxiety in autistic adults where research is at a very much earlier stage. The small amount of work that has been undertaken with autistic adults exploring the potential mechanisms and experiences of anxiety in autistic adults indicates that models developed with child samples may have utility when considering anxiety in autistic adulthood.2,4,9,16 As such, a tool such as the ASC-ASD provides a good starting point from which to begin the process of developing a measure for use in adulthood, and it is for this reason that we used the ASC-ASD as a framework. We consulted with six autistic adults with experience of anxiety during the current study to ascertain the face validity, clarity, and comprehensiveness of the item set. Important next steps for this work will include additional exploration of the measurement properties of the tool using techniques such as cognitive interviewing, measurement invariance analysis, and IRT.
The purpose of the current study was to develop and provide preliminary validation of the measurement properties of the ASA-A. This adds to and therefore complements the work recently undertaken on the use of the HADS with autistic people.16 Having access to a range of validated measures, which may serve specific and different research and clinical purposes, is of benefit. The HADS is a brief well-known tool that has good psychometric properties in the general population, and has been found to have acceptable measurement properties in ASC.15,16 As such, the HADS has potential as a helpful tool for brief screening and for use in research comparing autistic and nonautistic populations, subsequent to measurement invariance analysis being undertaken. The content of ASA-A, however, has been derived from previous work with the autism community and the items have been generated in collaboration with autistic people. This has resulted in the inclusion of autism-related anxiety items, which, of course, are not included in general population measures, such as the HADS. As such, the ASA-A may capture the phenomenology of anxiety for autistic people more accurately. Thus, the ASA-A is an important addition to the researcher and clinician toolbox to enable accurate assessment of anxiety in autism and in the future subsequent to further evaluation of any interventions.
EFA identified three group factors of the ASA-A: Anxious Arousal, Uncertainty, and Social Anxiety. CFA indicated that a bifactor solution with orthogonal general and specific factors was an adequate fit. Bifactor analyses suggested that minimal measurement bias would occur if the scale was treated as unidimensional so the total score could be used as a valid measure of anxiety. There is evidence of convergent and divergent validity for the total score against the HADS, an established measure of anxiety and depression. An ROC analysis indicates that a score of 28 on the ASA-A could be considered an indicative clinical cutoff of anxiety.
Internal consistency and test/retest reliability were both high for both total factor scores and group factors of the ASA-A. However, under the bifactor model, Omega hierarchical was good for both the General Anxiety factor and Anxious Arousal group factors, but lower for the Uncertainty and Social Anxiety group factors, so caution should be used in interpretation. Like other psychological constructs, despite scales being designed to measure a single construct,35 it is common to find evidence of both unidimensionality and multidimensionality.37 In this case, results indicate that the total score is an adequate representation of the constructs measured and further studies will determine the reproducibility of these across samples.
Limitations of our study include using the HADS-A to determine validity rather than a clinical interview. This was done because currently there is no gold standard anxiety assessment for autism. Although the HADS has been evaluated among autistic adults and has acceptable properties, using the HADS-A clinical cutoff as a standard for the ROC analysis is less than ideal. There are currently no questionnaire measures or diagnostic clinical interviews for anxiety disorders validated in autism against which we could benchmark the ASA-A. Therefore, until such a tool is developed, we have adopted this compromise position using a well-established measure of anxiety that has undergone some evaluation among autistic adults. Future research could triangulate caseness of anxiety using a clinical interview with ASA-A scores, and confirm if the current cutoff of 28 is appropriate. To reduce participant burden, we used only one measure of divergent validity, the HADS depression subscale. Including more than one measure of divergent validity would strengthen the findings. We recruited people with anxiety from a longitudinal cohort study, including autistic adults, resulting in a large sample; however, how well the results can be generalized to the general autistic population is unknown. Our sample, of which 54.1% were female and 42.5% male, does not reflect the reported gender ratio in autism. At present, we are not able to ascertain how the ASA-A performs with autistic adults with differing levels of verbal or reading ability, or who experience difficulties identifying and labeling their emotional experiences. This important limitation could be addressed in a future study.
Strengths include the inclusion of autistic adults in the development of the measure. Autistic adults have reported that their voices are not heard when it comes to meaningful involvement in research.38,39 The consultations contributed to how the measure should be adapted. Our sample size enabled us to undertake both EFA and CFA. EFA retains factors that account for variance in the data, whereas CFA seeks to assess the goodness of fit based on the remaining variance once the factors have been taken into account.18 Although the initial three correlated factor solution was supported by parallel analysis and a scree plot, and made theoretical sense, CFA indicated that a significant amount of variance was left unaccounted for and that a bifactor model proved to be a better fit.
The ASA-A has been designed and validated specifically for autistic adults without intellectual disability. It has promising psychometric properties, suggesting that it will be a useful tool and so would contribute to the measurement and understanding of anxiety in autistic adults.
Acknowledgments
We are grateful to all the participants who gave their time to complete the measure; the ASC-UK team, including Dr. Alex Petrou (who supported data collection), and the ASC-UK investigators, including Professor Ann Le Couteur and Professor Helen McConachie; Mr. Colin Wilson for his role as part of the wider PATA team, including data entry. The sponsors for the study were Cumbria, Northumberland, Tyne, and Wear NHS Foundation Trust.
Authorship Confirmation Statement
J.R., M.F., J.P., and B.I. conceived the research study and were awarded funding. S.B. collected data from autistic adults, supervised by J.P. and J.R. K.F., S.W., D.M., and S.B. undertook data analysis with guidance from M.F. and J.R. K.F. and J.R. wrote the first draft of the article; all authors read, edited, and commented on subsequent drafts and revisions. The article is submitted solely to Autism in Adulthood and is not published, in press, or submitted elsewhere.
Author Disclosure Statement
No competing financial interests exist.
Funding Information
Data from autistic adults were collected as part of the Personalized Anxiety Treatment for Autistic Adults study funded by the U.K. autism research charity Autistica. Funding was awarded to J.P. and J.R. as part of the Autism Spectrum Adulthood and Aging Research Program at Newcastle University.
References
- 1. Croen LA, Zerbo O, Qian Y, et al. The health status of adults on the autism spectrum. Autism. 2015;19(7):814–823. [DOI] [PubMed] [Google Scholar]
- 2. Hollocks MJ, Lerh JW, Magiati I, Meiser-Stedman R, Brugha TS. Anxiety and depression in adults with autism spectrum disorder: A systematic review and meta-analysis. Psychol Med. 2019;49(4):559–572. [DOI] [PubMed] [Google Scholar]
- 3. Somers JM, Goldner EM, Waraich P, Hsu L. Prevalence and incidence studies of anxiety disorders: A systematic review of the literature. Can J Psychiatry. 2006;51(2):100–113. [DOI] [PubMed] [Google Scholar]
- 4. Mazefsky CA, Folstein SE, Lainhart JE. Overrepresentation of mood and anxiety disorders in adults with autism and their first-degree relatives: What does it mean? Autism Res. 2008;1(3):193–197. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Tsakanikos E, Underwood L, Kravariti E, Bouras N, McCarthy J. Gender differences in co-morbid psychopathology and clinical management in adults with autism spectrum disorders. Res Autism Spectr Disord. 2011;5(2):803–808. [Google Scholar]
- 6. Brugha TS, Doos L, Tempier A, Einfeld S, Howlin P. Outcome measures in intervention trials for adults with autism spectrum disorders; a systematic review of assessments of core autism features and associated emotional and behavioural problems. Int J Methods Psychiatr Res. 2015;24(2):99–115. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Gotham K, Brunwasser SM, Lord C. Depressive and anxiety symptom trajectories from school age through young adulthood in samples with autism spectrum disorder and developmental delay. J Am Acad Child Adolesc Psychiatry. 2015;54(5):369–376.e363. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. South M, Rodgers J. Sensory, emotional and cognitive contributions to anxiety in autism spectrum disorders. Front Hum Neurosci. 2017;11:20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Trembath D, Germano C, Johanson G, Dissanayake C. The experience of anxiety in young adults with autism spectrum disorders. Focus Autism Other Dev Disabil. 2012;27(4):213–224. [Google Scholar]
- 10. Wigham S, McConachie H. Systematic review of the properties of tools used to measure outcomes in anxiety intervention studies for children with autism spectrum disorders. PLoS One. 2014;9(1):e85268. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Scahill L, Lecavalier L, Schultz RT, et al. Development of the parent-rated anxiety scale for youth with autism spectrum disorder. J Am Acad Child Adolesc Psychiatry. 2019;58:887–896. [DOI] [PubMed] [Google Scholar]
- 12. Rodgers J, Wigham S, McConachie H, Freeston M, Honey E, Parr JR. Development of the anxiety scale for children with autism spectrum disorder (ASC-ASD). Autism Res. 2016;9(11):1205–1215. [DOI] [PubMed] [Google Scholar]
- 13. Den Houting J, Adams D, Roberts J, Keen D. Exploring anxiety symptomatology in school-aged autistic children using an autism-specific assessment. Res Autism Spectr Disord. 2018;50:73–82. [Google Scholar]
- 14. Keen D, Adams D, Simpson K, Den Houting J, Roberts J. Anxiety-related symptomatology in young children on the autism spectrum. Autism. 2019;23(2):350–358. [DOI] [PubMed] [Google Scholar]
- 15. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67(6):361–370. [DOI] [PubMed] [Google Scholar]
- 16. Uljarević M, Richdale AL, McConachie H, et al. The hospital anxiety and depression scale: Factor structure and psychometric properties in older adolescents and young adults with autism spectrum disorder. Autism Res. 2018;11(2):258–269. [DOI] [PubMed] [Google Scholar]
- 17. Osborne JW, Costello AB, Kellow JT. Best practices in exploratory factor analysis. Best Pract Quant Methods. 2008:86–99. [Google Scholar]
- 18. Floyd FJ, Widaman KF. Factor analysis in the development and refinement of clinical assessment instruments. Psychol Assess. 1995;7(3):286. [Google Scholar]
- 19. Lorenzo-Seva U, Ferrando PJ. Factor 9.2: A comprehensive program for fitting exploratory and semiconfirmatory factor analysis and IRT models. Appl Psychol Meas. 2013. [Google Scholar]
- 20. Baglin J. Improving your exploratory factor analysis for ordinal data: A demonstration using FACTOR. Pract Assess Res Eval. 2014;19(5):2. [Google Scholar]
- 21. Osborne JW. Best Practices in Exploratory Factor Analysis. CreateSpace Independent Publishing Platform; 2014.
- 22. Osborne JW. What is rotating in exploratory factor analysis. Pract Assess Res Eval. 2015;20(2):1–7. [Google Scholar]
- 23. Lorenzo-Seva U. Promin: A method for oblique factor rotation. Multivariate Behav Res. 1999;34(3):347–365. [Google Scholar]
- 24. Mayes SD, Calhoun SL, Murray MJ, Zahid J. Variables associated with anxiety and depression in children with autism. J Dev Phys Disabil. 2011;23(4):325–337. [Google Scholar]
- 25. Arbuckle JL. Amos (Version 23.0)[Computer Program]. Chicago: IBM SpSS; 2014. [Google Scholar]
- 26. Holgado-Tello FP, Chacón-Moscoso S, Barbero-García I, Vila-Abad E. Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Qual Quant. 2010;44(1):153. [Google Scholar]
- 27. Brown TA. Confirmatory Factor Analysis for Applied Research. New York: Guilford Publications; 2014. [Google Scholar]
- 28. Rossier J, Zecca G, Stauffer SD, Maggiori C, Dauwalder J-P. Career Adapt-Abilities Scale in a French-speaking Swiss sample: Psychometric properties and relationships to personality and work engagement. J Vocat Behav. 2012;80(3):734–743. [Google Scholar]
- 29. Hoyle RH, Duvall JL. Determining the number of factors in exploratory and confirmatory factor analysis. In: Kaplan, D., ed. Handbook of Quantitative Methodology for the Social Sciences. New York: Sage Publishing; 2004;301–315. [Google Scholar]
- 30. Byrne BM. Structural Equation Modelling with AMOS: Basic Concepts, Application and Programming. Byrne BM, ed. New York: Taylor & Francis Group; 2010. [Google Scholar]
- 31. Reise SP, Bonifay W, Haviland MG. Bifactor modelling and the evaluation of scale scores. In: Irwing P. ed., The Wiley Handbook of Psychometric Testing: A Multidisciplinary Reference on Survey, Scale and Test Development. Hoboken, NJ: John Wiley & Sons Ltd; 2018;675–707. [Google Scholar]
- 32. Dueber DM. Bifactor Indices Calculator: A Microsoft Excel-based tool to calculate various indices relevant to bifactor CFA models. 2017. https://uknowledge.uky.edu/edp_tools/1/ (accessed February 20, 2020).
- 33. Bonifay WE, Reise SP, Scheines R, Meijer RR. When are multidimensional data unidimensional enough for structural equation modeling? An evaluation of the DETECT multidimensionality index. Struct Equ Model A Multidiscip J. 2015;22(4):504–516. [Google Scholar]
- 34. Reise SP, Scheines R, Widaman KF, Haviland MG. Multidimensionality and structural coefficient bias in structural equation modeling: A bifactor perspective. Educ Psychol Meas. 2013;73(1):5–26. [Google Scholar]
- 35. Rodriguez A, Reise SP, Haviland MG. Evaluating bifactor models: Calculating and interpreting statistical indices. Psychol Methods. 2016;21(2):137. [DOI] [PubMed] [Google Scholar]
- 36. Hosmer Jr DW, Lemeshow S, Sturdivant RX. Applied Logistic Regression. Vol. 398: New Jersey: John Wiley & Sons; 2013. [Google Scholar]
- 37. Brouwer D, Meijer RR, Zevalkink J. On the factor structure of the Beck Depression Inventory-II: G is the key. Psychol Assess. 2013;25(1):136. [DOI] [PubMed] [Google Scholar]
- 38. Milton D, Bracher M. Autistics speak but are they heard. J BSA Medsoc Group. 2013;7:61–69. [Google Scholar]
- 39. Pellicano L, Dinsmore A, Charman T. A Future Made Together: Shaping Autism Research in the UK. London: Institute of Education; 2013. [Google Scholar]