Anxiety Disorders Interview Schedule – Autism Addendum: Reliability and Validity in Children with Autism Spectrum Disorder

Abstract

Objective

Assessing anxiety in autism spectrum disorder (ASD) is inherently challenging due to overlapping (e.g., social avoidance) and ambiguous symptoms (e.g., fears of change). An ASD addendum to the Anxiety Disorders Interview Schedule–Child/Parent, Parent Version (ADIS/ASA) was developed to provide a systematic approach for differentiating traditional anxiety disorders from symptoms of ASD and more ambiguous, ASD-related anxiety symptoms.

Method

Inter-rater reliability and convergent and discriminant validity were examined in a sample of 69 youth with ASD (8–13 years, 75% male, IQ:68–143) seeking treatment for anxiety. The parents of participants completed the ADIS/ASA and a battery of behavioral measures. A second rater independently observed and scored recordings of the original interviews.

Results

Findings suggest reliable measurement of comorbid (ICC=0.85–0.98; κ =0.67–0.91) as well as ambiguous anxiety-like symptoms (ICC=0.87–95, κ=0.77–0.90) in children with ASD. Convergent and discriminant validity were supported for the traditional anxiety symptoms on the ADIS/ASA, whereas convergent and discriminant validity were partially supported for the ambiguous anxiety-like symptoms.

Conclusions

Results provide evidence for the reliability and validity of the ADIS/ASA as a measure of traditional anxiety categories in youth with ASD, with partial support for the validity of the ambiguous anxiety-like categories. Unlike other measures, the ADIS/ASA differentiates comorbid anxiety disorders from overlapping and ambiguous anxiety-like symptoms in ASD, allowing for more precise measurement and clinical conceptualization. Ambiguous anxiety-like symptoms appear phenomenologically distinct from comorbid anxiety disorders and may reflect either symptoms of ASD or a novel variant of anxiety in ASD.

Introduction

Anxiety disorders are estimated to occur in 11 – 84% of youth with autism spectrum disorder (ASD) and are associated with additional functional disability (Kerns, Kendall et al., 2015; White et al., 2009). Efforts to understand the phenomenology of anxiety in this cohort have yielded inconsistent findings regarding characteristics (e.g., cognitive ability, age, ASD severity) associated with anxiety (Van Steensel, Bögels, & Perrin, 2011). This variability is partially attributable to differences in sampling methods, but also likely reflects issues surrounding the measurement and presentation of anxiety in youth with ASD (Kerns & Kendall, 2012).

Assessing anxiety in youth with ASD is inherently challenging due to overlapping symptoms (e.g., social avoidance, compulsive behaviors, perseveration v. rumination) and the limited ability of some youth to report their internal experiences, which are commonly relied upon to differentiate behaviorally similar symptoms. For example, whereas social avoidance may reflect a fear of negative evaluation or rejection in those with anxiety, in youth with ASD, social avoidance may reflect a general disinterest in social interaction (a symptom of ASD), a fear of negative evaluation (a symptom of comorbid social anxiety) or anxiety related to social confusion, rather than evaluation fears (an ambiguous anxiety-like symptom that may reflect an intersection of anxiety and ASD symptoms; Kerns & Kendall, 2012; Kerns et al., 2016a). In addition to overlapping symptoms, youth with ASD present with a number of such ambiguous anxiety-like symptoms, including anticipatory worries about change, negative reactions to change, fears surrounding restricted interests (e.g., worries about getting access to a preferred activity), and phobic responses to unusual stimuli, such as men with beards, specific sounds, or toilets (Kerns et al., 2014). Odd and intense fears are disproportionately represented in children with ASD versus children without ASD, children with language disorder, children with learning disabilities, and children with intellectual disabilities (Mayes, Calhoun, Mayes, & Molitoris, 2012). Though few studies have focused on these distinct presentations of anxiety, Mayes et al. (2012) found that parents reported unusual fears (e.g., toilets, vacuum cleaners, visual media, steps/escalators, wind) in 41% of a large sample of children with ASD and varied intellectual abilities (N=1033). In addition, several qualitative studies (Ozsivadjian, Knott, & Magiati, 2012; Trembath, Germano, Johanson, & Dissanayake, 2012) suggest distinct sources of anxiety in individuals with ASD, including changes in routine, the lack of clearly defined rules in social situations and interactions, and anxiety around special or obsessive interests and specific sensory experiences (e.g., loud noises, crowds, smells).

The correct diagnostic classification of these ambiguous anxiety-like symptoms is unclear, which presents a problem for both researchers attempting to measure and better understand co-occurring anxiety in ASD and clinicians seeking to determine whether co-occurring anxiety is present and deserving of targeted intervention in an individual with ASD (Kerns & Kendall, 2012; Kerns et al., 2016b). Sensory-related or special-interest related fears, anxiety about change, and difficulties with the unpredictable nature of social interactions have long been associated with ASD (Kanner, 1943; Lord, Rutter, & Le Couteur, 1994), but may also reflect symptoms of anxiety, altered in presentation by its co-occurrence with ASD (Kerns & Kendall, 2012; Wood & Gadow, 2010). Whether such ambiguous symptoms reflect the ASD phenotype or a co-occurring anxiety disorder is a difficult to resolve question without a systematic approach to differentiating and measuring these symptoms. The present article evaluated an adapted measure, the Anxiety Disorders Interview Schedule – Child/Parent, Parent Version (Silverman & Albano, 1996) with Autism Spectrum Addendum (ADIS/ASA; Kerns et al., 2014), designed to address this challenge and provide a systematic approach to operationalizing and assessing comorbid anxiety disorders as well as ambiguous anxiety-like symptoms in ASD.

Thus far, research on anxiety in ASD has largely depended on measures designed and validated for youth without ASD that may have limitations (diminished accuracy and consistency and distinct factor structures) in ASD samples (Grondhuis & Aman, 2012; Kerns et al., 2015; Lecavalier et al., 2014; Mazefsky, Kao, & Oswald, 2011; White et al., 2015). In studies employing semistructured diagnostic interviews, estimations of different anxiety disorders fluctuate widely depending on how overlapping and ambiguous symptoms are operationalized. For example, studies report OCD in 6 to 37% of youth with ASD depending upon whether or not parents may infer their child’s mental states during ritualistic behaviors (de Bruin, Ferdinand, Meester, deNijis, & Verheji, 2007; Leyfer et al., 2006; see also Kerns & Kendall, 2012).

As these studies illustrate, whether overlapping and ambiguous symptoms are attributed to Diagnostic and Statistical Manual of Mental Disorders (5^th ed.; DSM-5; American Psychiatric Association, 2013) anxiety disorders or solely ASD may have a substantial influence on the rate of anxiety reported, the characteristics associated with anxiety in ASD and the rates of response to anxiety-focused treatments. Nonetheless, diagnostic approaches are not consistent across studies. Further, efforts to modify and test existing semistructured interviews of psychological disorders for youth with ASD have been extremely limited. The Autism Comorbidities Interview, a modified version of the Kiddie Schedule for Affective Disorders and Schizophrenia, was piloted in 109 youth with ASD; however, inter-rater reliability, concurrent, and criterion validity were only established for OCD, attention deficit/hyperactivity disorder (ADHD), and major depression, not the anxiety disorders (Leyfer et al., 2006). An instrument designed and validated to differentiate and comprehensively assess symptoms of anxiety in youth with ASD, suspected ASD and ASD-related social and developmental difficulties is needed.

In response to this need, the ADIS/ASA facilitates differential diagnosis of comorbid anxiety disorders (i.e., social phobia, separation anxiety, generalized anxiety, specific phobia) in ASD and also measures ambiguous anxiety-like symptoms, such as fears of change, unusual stimuli (e.g., toilets) and social unpredictability. ASA questions are added to the ADIS parent interview to support differential diagnosis within the interview’s intended semistructured format. In this way, the ASA provides a standardized approach to differential diagnosis rather than relying solely on unstructured follow-up from the assessor to determine the proper conceptualization of overlapping and ambiguous symptoms.

In a recent study, Kerns et al. (2014) tested an initial version of the ADIS/ASA in 59 youth with ASD and borderline to superior intelligence (IQ range = 67 – 158). The sample was relatively “low risk” for anxiety, as youth were not seeking treatment for anxiety or purposefully recruited for an anxiety research study. In this version of the ADIS/ASA, the assessors gathered information on both DSM-consistent anxiety disorders and anxiety-like symptoms that arose in participants but did not fit neatly into DSM categories. Results suggested that the ADIS/ASA exhibited interrater (n = 21; intraclass correlation coefficient [ICC] range = 0.89 – 0.98) and 2-week retest reliability (n = 15; ICC range = 0.77 – 1.00) as well as convergent validity with the Screen for Anxiety and Related Emotional Disorders total score and Behavior Assessment System for Children – Second Edition anxiety subscale and discriminant validity with the behavior Assessment System for Children – Second Edition Activities of Daily Living and Externalizing Subscales (Kerns et al., 2014). This study demonstrated the ability to differentiate and better understand variance attributable to ambiguous anxiety-like symptoms in ASD with an adapted interview; however, results were preliminary given that reliability was assessed in only a subset of cases. Further, representation of each type of anxiety (e.g., DSM social phobia, DSM generalized anxiety disorder, ambiguous fear of change) was limited given that the sample was not selected for anxiety problems.

Findings regarding the quality of overlapping and ambiguous anxiety-like symptoms from Kerns et al. (2014) informed the refinement of the ADIS/ASA. This included the creation of specific prompts to assess ambiguous symptoms (e.g., fear of change, special-interest fears, unusual phobias) and assist in differential diagnosis of ASD and anxiety symptoms (e.g., bullying history, social motivation). The present study assessed the ability of this refined ADIS/ASA to provide more precise measurement of comorbid anxiety disorders and ambiguous anxiety-like symptoms in ASD. Specifically, it builds on Kerns et al.’s (2014) preliminary findings by evaluating the interrater reliability, convergent and discriminant validity of the refined ADIS/ASA in a larger sample of cognitively able youth with ASD seeking treatment for anxiety. By reliably differentiating overlapping and ambiguous anxiety-like symptoms from comorbid anxiety disorders in ASD, this approach to measurement has the potential to shed light on inconsistent findings in research (e.g., inconsistent estimates of anxiety prevalence and associated characteristics in ASD). We hypothesized that both traditional anxiety clinician severity rating (CSR) scores and ambiguous anxiety-like CSR scores on the ADIS/ASA would exhibit acceptable interrater reliability and a pattern of convergent and discriminant validity signifying that traditional anxiety CSR scores primarily converge with other measures of traditional anxiety (but not other measures), whereas ambiguous anxiety-like CSR scores converge with other measures of traditional anxiety, as well as ASD symptoms, but not other areas of psychiatric symptomatology.

Method

Participants

Participants were 69 youth, 8 – 13 years of age (M = 10.70, SD =1.64; 75% male) with a community diagnosis of autism spectrum disorder and their parents. Youth were randomly selected from a larger pool of participants (N = 92) recruited as part of a randomized controlled trial (RCT) comparing therapies for anxiety for youth with ASD across three sites. Flyers, local presentations, and online advertisements facilitated recruitment of verbally fluent youth between the ages of 8 – 13 years seeking treatment for anxiety from three research clinics housed within universities in urban areas of the United States. Phone screens were conducted with interested families to assess initial inclusion/exclusion criteria for the trial. Inclusion criteria included (a) community diagnosis of ASD, (b) child age of 8 – 13 years, (c) verbal proficiency in the English language (for children and parents), and (d) child’s anxiety considered problematic for child by parents. Exclusion criteria (and rationale) included (a) a diagnosed or suspected intellectual disability (may struggle to understand cognitive-behavioral therapy concepts); (b) a diagnosis of bipolar or schizophrenia spectrum disorder or suicidality in the past 6 months (requires higher level of care than could be provided); (c) receiving concurrent therapy targeting anxiety, social skills, or behavioral interventions and unwilling to cease this treatment during the RCT (may confound experimental treatment); (d) family unwilling or unable to keep child’s medication regimen consistent during the RCT. Although additional, more stringent inclusion/exclusion criteria were assessed at the screening interview (e.g., confirmed clinically significant anxiety and ASD symptoms, specific IQ range) to determine eligibility for the RCT (see Kerns et al., 2016b), these criteria were not required for participants in the present study, who consisted of all those completing the screening interview. For example, though a verbal comprehension IQ greater than 70 was required for entry to the RCT, youth who completed the screening interview were enrolled in the present study as long as parents did not report a diagnosed or suspected intellectual disability during the initial phone screening. As such, the IQ range, as well as the range of anxiety and ASD symptom severity and presentation, was more varied for the present study than in the ultimate RCT. This allowed the ADIS/ASA to be tested in a broader range of youth with symptoms, but not always a diagnosis, of ASD or anxiety disorder. Ten youth were included in the present analyses but did not advance to the randomized clinical trial because they did not present with a primary anxiety disorder (n = 4), did not meet full criteria on ASD assessment measures (n = 4) or scored below the verbal proficiency requirement for the trial (Full-Scale IQ > 70; n = 2). This “at-risk” sample is akin to that seen in the clinical and research settings in which the ADIS/ASA is most likely to be used, for example, as a diagnostic measure for youth with suspected anxiety or ASD who require a comprehensive evaluation and treatment plan or for studies of co-occurring psychopathology in which precise phenotyping is needed. See Figure 1 for a flow chart of study recruitment and enrollment.

Figure 1.

Flow chart of study recruitment and enrollment. Note: RCT = randomized controlled trial; ASD = autism spectrum disorder.

As part of the in-person screening interview, ASD presentation was assessed by the Autism Diagnostic Observation Schedule – Second Edition by research reliable evaluators (ADOS-2; Lord et al., 2012) and the Child Autism Rating Scale (Schopler, Reichler, & Renner, 2002). Sixty-six of the 69 youth met full diagnostic criteria for ASD on the ADOS-2 (the remaining three youth demonstrated symptoms of ASD but not enough to meet clinical cutoff on the ADOS-2). Participant Full-Scale IQ, estimated via the Matrix Reasoning and Vocabulary subscales of the Wechsler Intelligence Scale for Children–IV, also completed during the screening interview, ranged from 68 to 143 (M = 102.09, SD = 15.31). Participants were predominantly white (70%) but also Black/African America (10%), Asian (9%), multiracial (7%), and American Indian (1%; 3% of participants did not specify race). Sixteen percent of the sample identified as Hispanic/Latino. Paternal and maternal education level was varied: graduate training (45% mothers, 29% fathers), standard college degree (26% mothers, 23% fathers), some college (19% mothers, 30% fathers), high school graduate (7% mothers, 13% fathers), less than 7^th grade completed (3% mothers, 2% fathers) and unspecified (3% fathers).

Measures

Anxiety Disorders Interview Schedule C/P – Parent Version (ADIS C/P; Silverman & Albano, 1996) with the Autism Spectrum Addendum (Kerns, 2015)

The ADIS-C/P is a semistructured interview for assessing DSM-IV (American Psychiatric Association, 1994) psychiatric disorders in children. Parents and youth are interviewed separately regarding the presence of current (last 3 months) specific phobias, generalized anxiety disorder (GAD), social phobia, separation anxiety disorder (SAD), and OCD. Modules for other disorders (e.g., panic disorder, agoraphobia, post-traumatic stress disorder) are also included but were not administered in the present study in order to reduce participant burden. Prior research suggests a low incidence of these disorders in children with ASD (Leyfer et al., 2006). By comparison, ADIS-C/P modules for ADHD, major depression, dysthymia, and oppositional defiant disorder were administered (without adaptations) given the relative prevalence of these conditions in ASD (Leyfer et al., 2006; Simonoff et al., 2008). Interviewers provide CSRs for each disorder category ranging from 0 (not impairing) to 8 (debilitating), with a CSR of 4 representing the cutoff for a clinical diagnosis. The ADIS-C/P has favorable psychometric properties in youth without ASD (Lyneham, Abbott, & Rapee, 2007; Silverman, Saavedra, & Pina, 2001; Wood, Paicentini, Bergman, McCracken, & Barrios, 2002) with preliminary evidence of acceptable convergent validity and interrater agreement in youth with ASD (Reaven, Blakeley-Smith, Culhane-Shelburne, & Hepburn, 2012; Storch, Wood, et al., 2012a; Wood et al., 2009). Only the parent version of the ADIS-C/P was administered in this study. Studies of the ADIS-C/P in children with ASD suggest that parent and child agreement about anxiety diagnoses is limited and that clinicians most often base their diagnostic impressions on parent report (Storch, Ehrenreich-May, et al., 2012b). As such, several studies utilizing the ADIS-C/P with ASD populations have relied solely on the parent-report tool (Kernd et al., 2016b; Reaven et al., 2012).

The ADIS/ASA offers a series of additional clinical guidelines and queries that are woven into the semistructured format of the ADIS-C/P, parent version for the traditional specific phobia, social phobia, OCD, GAD, and SAD sections. ASA items add to rather than replace the original items of the ADIS and provide additional information and guidelines to aid in the differential diagnosis of ASD and anxiety symptoms. The emergence and continuity of anxiety symptoms can be assessed within the context of differential diagnosis items. For example, whether a child’s social disinterest preceded or followed the emergence of social anxiety concerns is assessed as part of the Social Phobia module. However, past diagnoses are not routinely queried. Prompts include six additional items (Friendships, Social Motivation, Bullying/Social Rejection, Social Awareness, Hypersensitivity and Perseveration) that are asked of all participants and scored on a 0-to-3 Likert scale (with higher scores indicating more impairment). These additional items provide information on the social motivation, awareness, and experiences (i.e., ability to establish friendships, history of bullying) of each youth to inform the assessment of social anxiety disorder. Information on a child’s sensory sensitivities and perseverative interests or cognitive style is also gathered so that these aspects of ASD can be differentiated from phobias and GAD. These prompts and guidelines were informed by data gathered via an initial version of the ADIS/ASA (see Kerns et al., 2014) evaluated in a separate sample of nontreatment seeking youth with ASD (8–17 years of age). This initial version of the ADIS/ASA differentiated and recorded DSM-consistent (e.g., social phobia) and inconsistent (e.g., fear of change, social fear without fear of evaluation) symptoms of anxiety, but did not provide specific prompts to assess these symptoms or gather information important to differential diagnosis (e.g., bullying history, social interest). See Appendix A for a sample of ADIS/ASA items.

The ADIS/ASA also allows diagnosticians to provide separate CSRs for ambiguous anxiety-like symptoms in five categories: The first is Fears of Change – anticipatory anxiety and worry about novelty, the unknown and potential changes in routine, rules, and the environment. Fear is differentiated from simple rigidity and poor coping with change by requiring parents to provide clear behavioral examples of anticipatory worry about potential changes to schedule, routine, or novelty and avoidance of activities or reduced functioning due to this anticipation. Children who lack such anticipatory worry and avoidance but who respond with extreme distress and worry to changes in routines, rules, or the environment are instead coded under the second category, Negative Reactions to Change. Notably, difficulties with change and novelty can arise within the context of GAD as well as amongst children with behaviorally inhibited temperaments. In the ADIS/ASA these symptoms are measured separately, rather than being collapsed under GAD, in order to assess whether such fears and negative reactions to change occur predominantly when GAD is also present for children with ASD (suggesting they may be an aspect of GAD) or if they are more often apparent in the absence of other generalized concerns (suggesting an ASD-related presentation). The third category is Social Fear – worry and anxiety in social situations in youth who do not fear or seem aware of negative social evaluation, but instead worry about the unpredictability of social environments. The fourth is Unusual Phobias – phobias with a highly unusual focus (e.g., fear of men with beards, toilets, wind)- and the firth is Special Interest Fears – excessive and circumscribed worry related to a special or obsessive interest, for example, anxiety and fear about getting access to or losing a special interest, such as cartoon periodicals. Clinicians query specifically for impairment related to anticipatory worries about loss of access to the special interest and do not code disruptive behavior in response to the child not getting their way, loss of a privilege, or the removal of preferred objects.

Rather than subsuming these symptoms under the available DSM categories (an approach that may overestimate anxiety symptoms) or attributing them solely to ASD (an approach that may underestimate anxiety symptoms), these ambiguous symptom categories receive unique CSRs that can then be analyzed to inform scientific understanding of their correct taxonomy, as well as their unique contribution to a child’s clinical profile. For discussion of the framework that informs the ADIS/ASA, see Kerns et al. (2016a). See also Appendix A for a sample of ADIS/ASA prompts for ambiguous symptoms. Preliminary evidence supports the interrater reliability and retest reliability of both traditional and ambiguous anxiety symptoms as measured by an initial version of the ADIS/ASA (Kerns et al., 2014).

Note that no modifications were made to the attention deficit, major depression, dysthymia, and oppositional defiant disorder sections. In addition, though the ADIS/ASA provides guidelines for differential diagnosis of OCD and ASD and allows clinicians to rate “ambiguous symptoms of OCD” with a CSR, these behaviors were not described at a clinical level for any participant in the present study. As such psychometric data on this module is not included in the present manuscript.

Pediatric Anxiety Rating Scale (PARS; Research Units on Pediatric Psychopharmacology Anxiety Study, 2002)

The PARS is a clinician-rated interview that assesses anxiety symptoms over the past week, as well as associated severity and impairment. The PARS has demonstrated interrater reliability (.86), high test–retest reliability (.83), and convergent validity in anxious youth with ASD (Storch, Wood, et al., 2012a). Interrater reliability of the five-item total, assessed in a randomly selected 20% of participants in the present sample, was strong (ICC = 0.95), confidence interval [0.85, 0.99].

Autism Diagnostic Observation Schedule – Second Edition (ADOS-2; Lord et al., 2012)

The ADOS-2 is a semistructured observational assessment administered directly to the participant to elicit social, communicative, and restricted or repetitive behaviors associated with ASD. The ADOS-2, Module 3, is designed for verbally fluent individuals, has appropriate sensitivity (0.91) and specificity (0.84; Gotham, Risi, Pickles, & Lord, 2007), and has a continuous comparison score that can be used to estimate the severity of ASD symptoms (Gotham, Pickles, & Lord, 2009). The ADOS-2 was administered by research reliable evaluators (trained to percent agreement > .80) in the present study.

Child Behavior Checklist/6–18 Years, Parent Version (CBCL; Achenbach & Ruffle, 2000)

The CBCL is a parent-report questionnaire of positive and problem behaviors in children with established psychometric properties (Achenbach & Ruffle, 2000) that has shown a similar factor structure and acceptable internal consistency in several studies of school-aged youth with ASD (Duarte, Bordin, de Oliveira, & Bird, 2003; Mazefsky et al., 2011; Pandolfi et al., 2009) The CBCL includes subscales for DSM-oriented anxiety problems, attention, and aggression that were used to explore the convergent and discriminant validity of the ADIS/ASA in this study.

Procedure

This study was approved by the Institutional Review Board at each research site. All participants completed informed consent (parent) and assent (child) before completing any study measures or procedures. Youth with ASD and their parents who expressed interest in the intervention trial were scheduled for an initial screening interview, which included the ADIS/ASA, CBCL, PARS, Wechsler Intelligence Scale for Children-IV subscales, Child Autism Rating Scale, and ADOS-2. This protocol was completed for all youth regardless of whether they met full inclusion/exclusion criteria for the clinical trial. Participants were compensated $25 for participating in the screening interview.

The ADIS/ASA and PARS were administered to parents by seven trained evaluators across the research sites. Evaluators were doctoral level behavioral health clinicians and doctoral level psychology students who participated in a day-long training and initial consensus check led by the first author prior to the trial’s initiation. Interviewers also participated in weekly supervision with the first author to discuss questions regarding administration and scoring throughout the trial. ADIS/ASA interviews were audio-recorded with permission from participants.

Convergent and Discriminant Validity

The clinician-administered ADOS-2 and PARS and the parent-report CBCL attention, aggression, and DSM-oriented Anxiety Problems subscales were used to estimate discriminant and convergent validity of the ADIS/ASA CSRs. As youth could present with multiple forms of both traditional and ambiguous anxiety symptoms, the CSRs for traditional and ambiguous anxiety symptoms included in these analyses were represented by the most severe CSR given within each traditional and ambiguous category.

Inter-rater Reliability

Recording of the ADIS/ASA interviews were randomly selected from the first total 92 screening evaluations conducted. Selected interviews were then watched and scored by six independent, reliably trained raters. Raters were doctoral-level psychologists or students in psychology from the three different research sites. Though some raters (n = 3) were also trained evaluators, conducting the ADIS/ASA for the trial, no interviewers reviewed or rated their own recordings. Assignment of recordings to the raters was random (if assigned to their own recordings, raters were re-assigned to the next randomly selected interview). Before beginning their independent rating of tapes, raters participated in a day-long training on the ADIS/ASA as well as weekly conference calls over 2 months in which they watched and discussed their ratings for six unique ADIS/ASA administrations (not included in the present sample) with the first author to establish their expertise. Raters reviewed and scored 10–12 recordings each and submitted their independent ratings for reliability analyses. Rater agreement regarding both the severity of anxiety difficulties within each category (ICC) and the presence of meaningful symptoms (CSR ≥ 4) within each category (Cohen’s Kappa) was calculated. Agreement for CSRs was operationalized as matching within 1 point for each symptom category, with the exception of CSRs of 3 and 4, for which exact agreement was required given that a CSR of 4 signifies the cutoff for a diagnosis and clinically meaningful impairment.

Results

Descriptive Analyses

There were no significant differences in age, sex, or anxiety and ASD severity between the study sample (N = 69) and those not randomly selected ( N = 23). Table 1 displays the frequency and mean CSRs for all anxiety categories assessed by the ADIS/ASA in the sample. In addition, means and standard deviations of the ADIS/ASA items, as well as the PARS five-item Total Score, CBCL attention, aggression, and DSM-oriented anxiety problems subscales and ADOS-2 comparison scores, are presented. Data was missing at random for three CBCL questionnaires. No significant relationship between anxiety severity or type and age and sex were found with the exception of SAD, which was associated with younger child age (M = 9.58, SD = 1.51) as compared to youth without SAD (M = 10.93, SD = 1.58), t(67) = 2.70, p < 0.01, Cohen’s d = .87. Children with negative reactions to change were also, on average, younger (M = 9.20, SD = 0.84) than children without this symptom (M = 10.81, SD = 1.63), t(67) = 3.78, p = .01, Cohen’s d = 1.24. Fifty-three percent (n = 37) of youth had two or more traditional anxiety disorders and 4% (n = 3) presented with two or more types of meaningful (CSR ≥ 4) ambiguous anxiety-like symptoms. Fifty-five percent of the sample presented with co-occurring ADHD (combined or inattentive type), 22% with oppositional defiant disorder, and 9% presented with dysthymia or major depressive disorder as assessed by the ADIS/ASA.

Table 1.

Characteristics of Sample (N=69)

ADIS/ASA Traditional Anxiety (CSR Range: 0 – 8)	N (%) (CSR≥4)	CSR M (SD)
Separation Anxiety Disorder	12 (17%)	1.03 (1.79)
Social Phobia	27 (39%)	2.20 (2.20)
Specific Phobia	23 (33%)	2.04 (2.09)
GAD	44 (63%)	3.79 (1.89)
OCD	8 (11%)	0.87 (1.82)
Principal Traditional Anxiety	-	4.90 (1.21)
ADIS/ASA Ambiguous Anxiety-Like Symptoms (CSR Range: 0 – 7)	N (%) (CSR≥4)	CSR M (SD)
Social Fear	7 (10%)	0.49 (1.44)
Uncommon Phobia	6 (9%)	0.47 (1.47)
Special Interest Fear	2 (3%)	0.18 (0.84)
Fear of Change	20 (29%)	1.67 (2.23)
Negative Reaction to Change	5 (7%)	0.48 (1.18)
Principal Ambiguous Anxiety	-	2.63 (2.23)
Other Behavioral Measures	M (SD)	Range
PARS 5-item Total Score	16.54 (2.83)	5 to 23
ADOS-2 Comparison Score	7.25 (2.09)	1 to 10
CBCL Anxiety Subscale	6.87 (2.15)	2 to 12
CBCL Attention Subscale	11.18 (3.86)	3 to 18
CBCL Aggression Subscale	12.48 (7.55)	0 to 30

Note: N = 69. CSR = clinician severity rating from ADIS/A. ADIS/ASA = Anxiety Disorders Interview Schedule with Autism Addendum; Traditional Anxiety= Disorders consistent with Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition, Revised (DSM-IV-TR)) criteria and included in the original ADIS; Ambiguous Anxiety-Like Symptoms = categories added to the ADIS as part of the ASA in order to capture anxiety-like symptoms that do not fit traditional DSM categories; GAD = Generalized Anxiety Disorder; OCD = Obsessive Compulsive Disorder; PARS = Pediatric Anxiety Rating Scale; ADOS-2 = Autism Diagnostic Observation Schedule, 2nd Edition; CBCL = Child Behavior Checklist.

^aCSR range = 0–8.

^bCSR range = 0–7.

In a random selection of 20% of the sample, average administration of the ADIS/ASA was less than 2 hours (M = 105.36 mins, SD = 30.32). Ninety-four percent of the sample met criteria for one or more anxiety disorders per the ADIS/ASA. This included 46% with traditional anxiety disorders alone, 46% with both traditional anxiety disorders and CSR equal to or more than four ambiguous anxiety symptoms, and 2% with ambiguous anxiety alone. Among the traditional anxiety disorders, GAD was the most common (64%) followed by social phobia (39%) and specific phobia (33%). Among ambiguous anxiety, fears of change (29%) were most common followed by social fears (10%) and unusual phobias (e.g., fear of men with beards; 9%). The most common principal anxiety concern (traditional or ambiguous) was GAD (46.4%) and the mean CSR for primary diagnoses was 5.10 (SD = 1.06).

Interrater Reliability

Table 2 displays the diagnostic agreement (Cohen’s Kappa) between raters regarding principal diagnoses and the presence of meaningful (CSR ≥ 4) traditional and ambiguous anxiety symptoms as well as rater agreement regarding the CSRs (ICC) for each anxiety symptom type and the six additional ADIS/ASA differential diagnosis items (Friendships, Social Motivation, Bullying/Social Rejection, Social Awareness, Hypersensitivity, Perseveration). Interrater agreement for specific ASA items (ICC = 0.82–0.96), traditional anxiety (ICC = 0.85–0.98), and ambiguous anxiety severity ratings (ICC = 0.87–0.95) was excellent. Agreement was also good to excellent regarding principal diagnoses (κ = 0.82), the presence of traditional anxiety (κ = 0.67–.91), and ambiguous anxiety (κ = 0.77–0.90). Agreement regarding the presence of special interest fears, which occurred infrequently in the sample (n = 2) was lower (κ = 0.49); however, this was likely due to Kappa being an overly conservative estimate of agreement for low base rate phenomena. Given this limitation, two additional measures of rater agreement, less influenced by base rate, were analyzed and suggest high interrater agreement regarding special interest fears: percentage exact agreement = 0.97; prevalence and bias adjusted Kappa = 0.94.

Table 2.

Interrater reliability of ADIS/A anxiety categories (Cohen’s K), CSRs (ICC) and additional items (ICC)

Traditional Anxiety	ICC (CI)	κ	Ambiguous Anxiety	ICC (CI)	κ
SAD	0.95 (0.92–0.97)	0.84	Social Fear	0.88 (0.82–0.93)	0.78
Social Phobia	0.95 (0.91–0.97)	0.91	Uncommon Phobia	0.87 (0.77–0.93)	0.79
Specific Phobia	0.85 (0.77–0.91)	0.67	Special Interest Fear	0.89 (0.82–0.93)	0.49a
GAD	0.93 (.089–0.96)	0.70	Fear of Change	0.94 (0.90–0.96)	0.90
OCD	0.98 (0.97–0.99)	1.00	Negative Rxn Change	0.95 (0.82–0.93)	0.88
Additional Items	ICC (CI)	M (SD)	Additional Items	ICC (CI)	M (SD)
Friends	0.83 (0.72–0.89)	0.96 (0.98)	Social Awareness	0.82 (0.71–0.89)	1.36 (0.69)
Social Motivation	0.82 (0.71–0.89)	0.76 (0.74)	Hypersensitivity	0.85 (0.74–0.91)	1.87 (0.74)
Bullying/Rejection	0.89 (0.83–0.93)	1.23 (0.86)	Perseverative	0.80 (0.66–0.87)	1.28 (0.65)

Note. ADIS/A = Anxiety Disorders Interview Schedule with Autism Addendum; CSR = clinical severity rating from ADIS/A; ICC = intraclass correlation coefficient; Traditional Anxiety= Disorders consistent with Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition, Revised (DSM-IV-TR)) criteria and included in the original ADIS; Ambiguous Anxiety-Like Symptoms = categories added to the ADIS as part of the ASA in order to capture anxiety-like symptoms that do not fit traditional DSM categories; GAD = generalized anxiety disorder; OCD = obsessive compulsive disorder; Rxn = Reaction.

^aGiven that low base rates are likely to result in overly conservative Cohen’s Kappa estimates, percentage agreement (0.97) and a prevalence and bias adjusted Kappa (.94) were also calculated to provide an estimate of rater agreement for the infrequently observed (n = 2) Special Interest Fear.

Convergent & Discriminant Validity

Table 3 displays the correlations among the study measures and the ADIS/ASA CSRs for traditional and ambiguous anxiety-like symptoms (represented by the most severe CSR provided within each category). Whereas the PARS 5-item Total Score, and CBCL DSM anxiety subscale were significantly and positively correlated with ADIS/ASA traditional anxiety CSRs, they were not significantly correlated with the ADIS/ASA CSR for ambiguous anxiety. By comparison, whereas ambiguous anxiety was significantly correlated with ADOS-2 scores, traditional anxiety was not. Neither traditional nor ambiguous anxiety symptoms were correlated with the CBCL Attention and Aggression subscales.

Table 3.

Pearson correlations between ADIS/A most severe CSR and study measures

Measure	Construct	Source	Traditional Anxiety CSR	Ambiguous Anxiety CSR	Fisher R-to-Z
PARS 5-item Total	Anxiety	Diagnostician	0.40**	0.11	Z = 1.79, p = 0.7
CBCL Anxiety	Anxiety	Parent	0.30*	0.11	Z = 1.11, p = .26
CBCL Attention	Attention	Parent	0.02	0.09	Z = −0.4, p = .69
CBCL Aggression 0.11	Aggression	Parent	−0.16		Z = −1.54, p = .12
ADOS-2 Comparison	Autism	Diagnostician	−0.01	0.30*	Z = −1.84, p = 0.7

Note: Fisher R-to-Z transformation was used to assess differences between the correlations of traditional and ambiguous CSR with each measure. Traditional Anxiety= Disorders consistent with DSM criteria and included in the original ADIS. Ambiguous Anxiety-Like Symptoms=Categories added to the ADIS as part of the ASA in order to capture anxiety-like symptoms that do not fit traditional DSM categories. PARS=Pediatric Anxiety Rating Scale; CBCL = Child Behavior Checklist.

^*p < .05.

^**p < .01.

Discussion

There is a need for a psychometrically sound measure to systematically differentiate and assess comorbid anxiety disorders and other, more ambiguous anxiety-like behaviors in children with ASD. The ADIS/ASA expands the ADIS-C/P, a commonly used semistructured diagnostic interview for childhood anxiety disorders, to address this need. The present findings provide supportive data on the interrater reliability, and convergent and discriminant validity of the ADIS/ASA, in a well-characterized sample of children with ASD seeking treatment for anxiety. By differentiating overlapping and ambiguous anxiety-like behaviors (e.g., fears of change, fears related to social unpredictability, sensory and special-interest related fears) from comorbid anxiety disorders (e.g., separation, social, generalized anxiety), the ADIS/ASA may facilitate a more precise and clearly defined diagnostic process for both researchers and clinicians alike. It may help researchers parse out variability associated with comorbid versus ambiguous anxiety symptoms and support practitioners in recognizing symptoms of anxiety in ASD that warrant a comorbid diagnosis (e.g., traditional anxiety symptoms) and may benefit from anxiety-specific interventions, such as cognitive-behavioral therapy. Moreover, the ADIS/ASA may improve understanding and treatment of ambiguous anxiety-like behaviors in ASD, which have been relatively understudied and lack a clear diagnostic “home” (Kerns & Kendall, 2012).

Data support the ADIS/ASA as a psychometrically sound measure for assessing the prevalence and severity of comorbid anxiety disorders, including specific phobia, GAD, SAD, and social phobia, as well as OCD, in children with ASD. Findings suggest that the ADIS/ASA provides consistent measurement of DSM anxiety diagnoses in youth with ASD across different raters. Interrater agreement on the ADIS/ASA was good to excellent regarding both the presence of traditional anxiety disorders and the perceived severity of these disorders when this tailored and standardized approach to differential diagnosis was applied. With regard to construct validity, data support the convergence of the ADIS/ASA traditional anxiety severity ratings with other clinician (PARS) and parent-report measures of anxiety (CBCL Anxiety/Depression subscale). By comparison, ADIS/ASA traditional anxiety severity ratings did not significantly correlate with measures of ASD (ADOS-2), attention difficulties (CBCL Attention subscale) or aggression (CBCL Aggression subscale), supporting the discriminant validity of the tool. The ADIS/ASA differs from the ADIS-C/P and other available semistructured interviews in that it provides specific guidelines for differentiating ASD and anxiety symptoms and parses out ambiguous symptoms that may contribute to both the overcounting (e.g., social avoidance due to ASD is misinterpreted as social anxiety) and undercounting of anxiety symptoms (e.g., social anxiety is missed because social difficulties are attributed solely to ASD; a phobia is dismissed as a ASD-related sensory difficulty).

Results also supported the interrater reliability and partially supported the convergent and discriminant validity of the ADIS/ASA in measuring the ambiguous anxiety-like symptoms, such as fears of change, fears related to special interests, unusual phobias (e.g., fear of men with beards, toilets), and social fears in individuals with limited awareness of negative evaluation. These symptoms have long been associated with a diagnosis of ASD (Gotham et al., 2013; Kanner, 1943; Lord et al., 1994; Mayes et al., 2013), but their correct classification as core or co-occurring features of the disorder has been debated (Kerns & Kendall, 2012). Interrater agreement regarding the presence of ambiguous anxiety symptoms in the present study was as strong as for DSM-consistent syndromes. Further, consistent with prior studies (Gotham et al., 2013; Kerns et al., 2014) and with our hypotheses, results suggest that ambiguous symptoms were moderately associated with symptoms of ASD. However, in contrast to our hypotheses, ambiguous symptoms of anxiety were not significantly related to more traditional, DSM-consistent manifestations of anxiety in our sample. This finding could mean that ambiguous anxiety-like symptoms primarily represent core symptoms of ASD. However, the restricted, severe range of anxiety in the present study may have obscured the true relationship of anxiety and these ambiguous symptoms (i.e., through attenuation of correlation). In a sample of youth with ASD not recruited for having a risk of anxiety (in contrast to the present sample), Kerns et al. (2014) found ambiguous symptoms to be associated with both ASD and anxiety severity. Further, Gotham et al. (2013) found that neither Insistence on Sameness (IS) behaviors – which share many similarities with the Fear and Negative Reaction to Change categories measured by the ADIS/ASA – nor more traditional anxiety symptoms were associated with ASD symptoms in a large sample (N = 1429) of youth with ASD (5 – 18 years). Although IS behaviors and anxiety were associated in the Gotham et al. (2013) study, they appeared phenomenologically distinct: Only anxiety symptoms, not IS behaviors, were associated with other psychiatric and behavioral difficulties in ASD. Another noteworthy contrast: Kerns et al. (2014) found that 46% of their sample presented with ambiguous anxiety-like behaviors, including 15% for whom ambiguous anxiety was their only significant anxiety symptom, only 2% of the present sample presented with only ambiguous anxiety, whereas most (46%) presented with both ambiguous and comorbid anxiety symptoms. These variable results (i.e., 15% vs. 2% presenting with only ambiguous anxiety) likely reflect the specific recruitment of children with suspected anxiety difficulties in the present sample. Whereas this recruitment approach was ideal for assessing the interrater reliability of the ADIS/ASA symptom categories, which requires a sample saturated with different anxiety presentations, it does not offer an accurate representation of the actual prevalence of ambiguous anxiety-like behaviors in the population. Given these limitations, future study of ambiguous anxiety-like behavior in more representative samples of children with ASD is warranted to clarify the prevalence and precise relationship of these behaviors to ASD and comorbid anxiety.

In support of the discriminant validity of the ambiguous anxiety-like symptoms on the ADIS/ASA tool, these symptoms were not significantly associated with other behavioral symptoms such as attention and aggression problems. This suggests that ambiguous anxiety-like symptoms as measured by the ADIS/ASA are not simply a reflection of poorer overall functioning or greater psychiatric illness in youth with ASD, but rather a separate construct.

Strengths of the study include a well-characterized sample of youth with ASD and a varied range of comorbid and ambiguous anxiety-like symptoms as well as a robust test of the ADIS/ASA tool by examining its reliability amongst a large number of raters selected from multiple, geographically distant and diverse sites. The consistent measurement of anxiety facilitated by the ADIS/ASA across so many raters and research centers supports the generalizability of the present findings and the likelihood that these results will be replicable in future research. Limitations of the study include the restricted racial diversity; age (8 – 13 years) and intellectual functioning (IQ range = 68 – 143) of participants, which may limit the generalizability of results; the low base rate of Special Interest Fears, which resulted in varied reliability estimates for this symptom; and the focus on youth seeking treatment for anxiety, which provided the greater range and base rate of different anxiety symptoms needed to sufficiently assess interrater reliability but also, consequently, excluded low-risk youth (e.g., youth without suspected anxiety difficulties). Encouragingly, similarly promising psychometric results for an early version of the ADIS/ASA were reported by Kerns et al. (2014) in a smaller, but more varied (in terms of age and anxiety severity), sample of youth with ASD. Nevertheless, future research with more representative samples will clarify the range of this instrument and determine its performance in individuals with more diverse sociodemographic profiles and characteristics. Further adaptation and validation of the ADIS/ASA will likely be required to accurately capture the distinct presentation of anxiety in youth with ASD and moderate to severe intellectual disability. Special Interest Fears, which may be a low occurrence symptom, will also require further validation, given that only two youth presented with this concern in this study.

Although the ASA maintains the original items and structure of the ADIS, the addition of questions likely alters the administration (i.e., potential ordering effects) of the interview. Exact parameters of this influence are difficult to assess without a direct comparison of the ADIS-C/P and ADIS/ASA interviews; however, results support the feasibility and utility of the ADIS/ASA interview. In addition to the promising reliability and convergent and discriminant validity of the instrument, results suggest that average ADIS/ASA administration time is between 1.5 and 2 hours. This duration is within the typical administration range for a comprehensive diagnostic evaluation (Grecco & Morris, 2004) and suggests that the ASA does not add substantial additional time to the ADIS interview. Notably, the guidance and structure of the ADIS may improve the efficiency of the interview for evaluators by providing them with a specific approach to differential diagnosis that reduces the need for unstructured follow-up questions. Like most comprehensive diagnostic interviews, the ADIS/ASA requires a larger investment of clinician (for training and administration) and parent time (for interview completion) than brief parent report measures or symptom screeners. For this reason, the ADIS/ASA is likely to be most useful when precise behavioral characterization or phenotyping is critical to a research question or when the clinical goal is comprehensive evaluation and differential diagnosis, following an initial positive screen for anxiety concerns. Promisingly, results suggest that evaluators in this study were able to establish and maintain reliability across multiple sites after a single, day-long in-person training with follow-up phone supervision to ensure clinician comfort and fluency with the tool. This training model can be replicated in research and clinical settings alike, particularly given the growing ability to offer trainings and supervision remotely via web-based platforms. Moreover, follow-up supervision may be most indicated for research studies in which highly reliable and precise phenotyping of anxiety and ASD symptoms is paramount. Even without this level of research reliability, the ADIS/ASA may be useful to clinicians who wish to use it as a tool or element of evaluation that can be combined with clinical judgment and other sources of information to guide diagnostic decisions and treatment planning.

The ADIS/ASA stands apart from other anxiety measures in its assessment of ambiguous anxiety-like behaviors. What then are the implications of measuring these behaviors for clinical research and practice? Ambiguous anxiety-like features may reflect ASD-specific variants of anxiety or may be aspects of the ASD diathesis that vary among individuals with ASD (Kerns & Kendall, 2012; Wood & Gadow, 2010). With regard to the latter possibility, this could mean that the anxiety-like features measured by the ADIS/ASA are indicators of social-cognitive and self-regulatory deficits characterizing ASD rather than a co-occurring anxiety disorder. The reliable and valid measurement of ambiguous symptoms provided by the ADIS/ASA will inform these questions, allow researchers to reduce variability attributable to ambiguous symptoms in their research, and help practitioners flag these features, for which diagnostic labels and treatment recommendations are less clear. Notably, given the close tethering of ambiguous and core ASD symptoms, giving an anxiety diagnosis for these symptoms may over-pathologize a behavioral presentation that could be more parsimoniously captured under a single diagnosis (e.g., ASD). On the other hand, recognizing that these behaviors may be related to anxiety and somewhat distinct from core ASD symptoms, as current and prior research suggests (Gotham et al., 2013; Kerns et al., 2014), may afford novel supports and intervention approaches (e.g., relaxation strategies, affective labeling, graded exposure) to youth with ASD and their families that might otherwise be overlooked. With this in mind, Kerns et al. (2016a) provide guidelines for practitioners regarding when a diagnosis of Other Specified Anxiety Disorder might be considered to aid case conceptualization and intervention in youth with ASD; however, these guidelines will require further study. Important to note, the reliable measurement of these ambiguous behaviors afforded by the ADIS/ASA will help to determine best practices for these symptoms and their relationship to the functioning and outcomes of children with ASD in the future. Finally, it is plausible that ambiguous features are not unique to ASD. Ambiguous anxiety-like symptoms likely also arise in youth who share certain characteristics with youth with ASD, such as specific executive functioning or social cognition deficits or sensory sensitivities (Ollendick & White, 2012). As such, the ADIS/ASA may prove useful for a variety of clinical groups.

In sum, the ADIS/ASA may improve consistency and explain discrepancies across research studies of anxiety in ASD that might otherwise be hindered by the lack of a systematic and standardized assessment approach. Studies are needed, however, to determine how precisely differentiating anxiety, ASD, and ambiguous symptoms with the ADIS/ASA may influence the estimated prevalence of co-occurring anxiety disorders in ASD and inform our understanding of the developmental trajectory, treatment response, and underlying neurobiological mechanisms of these disorders.

Appendix A

Table A1.

Sample of ADIS/ASA Differential Diagnosis and Ambiguous Anxiety Items

Differential Diagnosis Items (0 – 3 Likert Rating)
Example 1. Social Motivation Does your child seem interested in being around others? Does your child watch and appear interested in joining in with others, even if she doesn’t know how? Has your child shown a clear desire to make friends? Has his/her interest changed overtime? What do you think caused this change? Sample Anchors: 0 = Child displays clear desire to engage socially and develop friendship even if s/he has not been successful; 3 = Child seeks social interaction only as a means to an end (e.g., to obtain a need or want). Example 2. Bullying/ Peer Rejection Has your child been rejected by peers, picked on or bullied? How often? What happened? Has she also had positive interactions or relationships? Sample Anchors: 0 = No bullying/rejection; 3 = Severe. Significant rejection and/or bullying, resulting in child’s refusal to attend school or other activities or parents removal of child from school/activities due to concern for their welfare.
Ambiguous Anxiety-Like Symptoms (0 – 8 Clinician Severity Rating)
Example 1. Fear of Change Does your child worry excessively about changes in his/her routine (e.g., substitute teacher, being dropped off by different person) or personal surroundings? Or about changes in routine or appearance of others? Does your child worry about the time or about changes in rules? Does your child worry ahead of time about potential changes? How does your child react if the change is positive (e.g., getting out of school early)? Sample Anchors: 2 = worry ahead of time about changes in routine/environment, but can typically recover and continue to function. No accommodation needed; 4 = worries excessively about potential changes in routine/environment and/or novelty such that child or family’s functioning is limited. 6 = anticipatory worries about change or novelty cause substantial impairment to child or family functioning (e.g., child may not leave house at times or may refuse activities due to difficulties coping with unexpected events).