Measuring Anxiety as a Treatment Endpoint in Youth with Autism Spectrum Disorder

Abstract

Despite the high rate of anxiety in individuals with autism spectrum disorder (ASD), measuring anxiety in ASD is fraught with uncertainty. This is due, in part, to incomplete consensus on the manifestations of anxiety in this population. Autism Speaks assembled a panel of experts to conduct a systematic review of available measures for anxiety in youth with ASD. To complete the review, the panel held monthly conference calls and two face-to-face meetings over a fourteen-month period. Thirty eight published studies were reviewed and ten assessment measures were examined: four were deemed appropriate for use in clinical trials, although with conditions; three were judged to be potentially appropriate, while three were considered not useful for clinical trials assessing anxiety. Despite recent advances, additional relevant, reliable and valid outcome measures are needed to evaluate treatments for anxiety in ASD.

Introduction

Autism Spectrum Disorder (ASD) is a neurodevelopmental condition characterized by impairments in social communication coupled with repetitive patterns of behaviors and interests (American Psychiatric Association, 2013). ASD may affect as many as 11 per 1000 children (Centers for Disease Control, 2012) with a male to female ratio of 4:1. This estimate reflects a steady increase in the detected prevalence over the past two decades (Fombonne, 2009). Not surprisingly, this increase in the detected prevalence has sparked media attention and an increased demand for services. In addition to the core features of ASD, children and adolescents with ASD may also exhibit concomitant problems such as tantrums, aggression, hyperactivity, and anxiety (Lecavalier, 2006; White et al., 2009). Although the relationship of these problems to the defining features of ASD is not completely clear, they can complicate the clinical picture and contribute to the overall disability of the disorder. Whether these problems are part of ASD or independent from ASD, they may be appropriate targets for psychosocial or pharmacological treatment (Research Units on Pediatric Psychopharmacology (RUPP) Autism Network, 2002; RUPP Autism Network, 2005; White et al., 2009; Wood et al., 2009a). Progress in treatment development, however, relies on the availability of relevant, reliable and valid outcome measures that are sensitive to change with treatment in the ASD population.

To date, two medications have been approved by the U.S. Food and Drug Administration (FDA) for the treatment of children with autism as defined in DSM-IV (APA, 2000). The antipsychotic drugs, risperidone and aripiprazole, have demonstrated efficacy in reducing tantrums, aggression and self-injury (Marcus et al., 2009; RUPP Autism Network, 2002; RUPP Autism Network, 2005), as measured on the Irritability subscale of the Aberrant Behavior Checklist (ABC; Aman, et al., 1985). The successful approval of these two drugs offers a precedent for drug development in ASD for a specific set of symptoms. Other frequent targets of interest include social disability, repetitive behavior, hyperactivity, sleep disturbance and anxiety. Here, we focus on anxiety, which is common in children with ASD, contributes to overall impairment, and is overdue as a treatment focus (White et al., 2009; Kerns and Kendall, 2012).

Estimates of impairing anxiety range from 11–84% in school-aged children with ASD and as many as 40% meet criteria for an anxiety disorder (White et al., 2009; van Steensel et al., 2011 Kerns and Kendall, 2012). These reviews note a wide range of estimates attributable to differences in the sample source, sample size and assessment methods employed. van Steensel et al. (2011) reported that the most common anxiety disorders are specific phobia (30%), OCD (17%) and social anxiety (17%). These rates of anxiety disorders in youth with ASD are nearly two-fold higher than current estimates in typically developing children (Costello et al., 2005). Other reports indicate that Separation Anxiety (Leyfer et al., 2006) and Generalized Anxiety (Gadow et al., 2005) also occur at higher than expected rates in youth with ASD. Although the majority of previous studies involved clinic-based samples, community-based studies also indicate that children with ASD are at greater risk of anxiety (Simonoff et al., 2008).

Despite the high prevalence of anxiety and identifiable anxiety disorders in children with ASD, consensus on how to measure anxiety in this population is uncertain. According to a recent review, as many as 36 different measures, including parent-, self-, clinician- and teacher-rated instruments (Grondhuis and Aman, 2012) have been used to measure anxiety in children with ASD. With few exceptions (Gadow et al., 2005; Hallett et al., in press; Hallett et al., 2009; Leyfer et al. 2006; Storch et al., 2012a,b), most instruments used to measure anxiety in children with ASD have not been validated in this population.

Several challenges confront the measurement of anxiety in ASD. First, symptoms of anxiety may be difficult to disentangle from symptoms of ASD (White et al., 2009; Wood & Gadow, 2010; Kerns and Kendall, 2012). For example, avoidance of social situations is observed in children with social anxiety and those with ASD and may be intrinsic to the diagnostic criteria for ASD. It may not be clear in children with an ASD whether protest on separation from the parent is due to separation anxiety, the child’s adherence to a routine or both. Similarly, repetitive behavior is a defining feature of both ASD and OCD. Difficulties with sleep are also common across ASD and anxiety disorders. Other anxiety disorders, such as simple phobia, may be more common in children with ASD with no difference in manifestations compared to typically developing children. For example, Evans and colleagues (2005) reported that children with ASD had more specific fears (e.g., riding the school bus, bathrooms, medical procedures) compared to typically developing controls and children with Down syndrome. In this study, anxiety in children with ASD was also more strongly associated with disruptive behavior than in the comparison groups.

A second measurement challenge concerns the high proportion of children with ASD with low levels of cognitive functioning (IQ < 70), which is currently estimated to range between 38 to 50% for ASD (Centers for Disease Control, 2012; Fombonne, 2009). To report anxiety, expressive language skills and the ability to identify internal states such as worry and fear are required. As a result, self-report measures that are reliable in typically developing children may have limited utility in cognitively lower-functioning children with ASD. For this reason, primary caregivers are often considered more reliable informants of anxiety. However, in the absence of direct verbal expression from the child, even parents may find it difficult to infer which behaviors are driven by anxiety and which are due to ASD. In addition to expressive verbal ability, this problem of attribution is likely to be influenced by the child’s IQ. Using parent-rated dimensional measures, some studies have shown increased anxiety in children with higher IQ compared to children with intellectual disability (Weisbrot, Gadow, DeVincent, & Pomeroy, 2005; Hallett et al., 2013). The vulnerability of higher-functioning children to anxiety may stem from increased awareness of their social disability, alongside greater motivation to fit in with peers. It may also reflect the greater ability of these children to express their concerns, making it easier for parents to detect anxiety symptoms. In contrast, the meta-analysis by van Steensel and colleagues (2011) reported higher rates of anxiety disorders in children with lower levels of intellectual functioning. This suggests that children with lower IQ do experience anxiety and exhibit anxiety-driven behaviors even if the anxiety is not expressed verbally. For example, lower-functioning children may demand to follow daily routines as a way to reduce anxiety caused by unpredictability in everyday life.

The scope and format of scale items also warrant consideration when evaluating anxiety outcome measures in ASD. In addition to being reliable and valid, an anxiety outcome measure should be sensitive to change over time and have adequate coverage of commonly observed symptoms as well as at least some less common symptoms. At the same time, a measure with a long list of items may be time consuming and unfeasible for repeated measurement in large-scale randomized trials. Thus, the ideal measure will achieve a balance between full coverage and practicality to ensure a range of scores without undue subject burden. For example, available data suggest that Panic Disorder and Post Traumatic Stress Disorder are uncommon in children with ASD. If confirmed by additional study, items reflecting these disorders could be excluded from an outcome measure without sacrificing the relevance of the measure to ASD. By contrast, specific fears, alterations in arousal, social anxiety, and separation anxiety are more common and deserve comprehensive coverage (White et al., 2009; van Steensel et al., 2011; Kerns and Kendall, 2012).

A third challenge for measuring anxiety in youth with ASD emerges from the possibility that manifestations of anxiety may be unique or particularly salient in children with ASD. For example, a child’s insistence on following specific routines may be driven by anxiety. The child’s distress and over-reaction to transitions or minor disappointments may reflect the degree of anxiety. If ASD-specific manifestations of anxiety are identified, these pertinent items warrant inclusion in an anxiety measure.

Fourth and finally, even in typically developing children, boundaries between anxiety disorders may not be precisely demarcated, as children often have symptoms from more than one anxiety disorder (Walkup et al., 2008). This observation suggests that a dimensional approach to measurement may be preferred over a categorical approach to identify children at high risk for anxiety disorders, to establish a severity threshold for eligibility into clinical trials and to monitor outcome.

This review examines the strengths and weaknesses of available instruments for measuring anxiety in youth with ASD. The aims of this review are to describe the state of the field for measuring anxiety in ASD with the aim of offering guidance on the selection of primary outcome measure in clinical trials focused on treatment of anxiety. The impetus for the review is the emerging consensus that anxiety is common in children with ASD and is a clinically meaningful target for treatment.

Methods

Workgroup

In 2011, Autism Speaks empanelled work groups to evaluate outcome measures for clinically meaningful targets, including social communication deficits, repetitive behaviors and anxiety. The workgroup (which consisted of experts in clinical trials in individuals with developmental disabilities, experts in the assessment of anxiety in ASD, and Autism Speaks program staff) held monthly conference calls and two face-to-face meetings over the course of 14 months. The charge of the work group was to review the strengths and weaknesses of available instruments for measuring anxiety in youth with ASD and to rank them for readiness for use in clinical trials. The process of rank ordering the available instruments also provided a platform for exploring whether anxiety symptoms co-occur as a separate problem in youth with ASD or whether anxiety blends imperceptibly with ASD. The issue of whether anxiety or the manifestations of anxiety are different in children with ASD compared to typically developing children is intellectually compelling, practically important for measurement and may have regulatory implications (e.g., FDA review; see section on pseudospecificity, below).

Identification of Measures

The measures evaluated by the working group for this report were identified in PubMed, Web of Knowledge, Google Scholar, and clinicaltrials.gov using search terms: “autism and anxiety and clinical trial” and “autism and anxiety and treatment” “autism and anxiety and intervention” and “autism and anxiety” for the years 2005 through 2012. Additional measures were considered for evaluation based on knowledge of workgroup members and examination of recently published textbooks. Whenever possible, the instrument manuals were obtained and reviewed for additional information.

Evaluation of Measures

Identified measures were categorized by type (questionnaire, direct observation, clinical interview) and respondent type (clinician, parent, teacher, patient). The workgroup examined (a) the relevance and symptom coverage of the measure for anxiety and (b) evidence of prior use in ASD or other developmentally delayed populations and overall subject burden. The assessments also evaluated psychometric indices such as construct validity, internal consistency, and test-retest reliability and, for clinician-rated instruments, inter-rater reliability. We also examined the evidence on sensitivity to change. The age range of subjects in published reports and whether the measure was available in languages other than English were also considered. The review on reliability and validity statistics was guided by principles enumerated in standard textbooks (Nunnally and Bernstein, 1994; Fleiss, Levin and Paik, 2003). Because our interest was to identify the best available instruments, however, we did not impose strict benchmarks for reliability and validity indices. Thus, using all available information, the workgroup ranked measures by consensus (see Appendix 1 for details of evaluative criteria).

Based on these evaluative criteria, instruments were classified as follows: (a) Appropriate, (b) Appropriate with Conditions, (c) Potentially Appropriate, (d) Unproven, or (e) Not Appropriate (see Table 1). The measures classified as Appropriate, Appropriate with Conditions, or Potentially Appropriate were considered clinically relevant with supportive evidence for reliability and validity. The rating of Unproven or Not Appropriate reflected a judgment that the measure was not adequately studied, was not relevant to the assessment of anxiety, or that available evidence suggested clear threats to the reliability and validity.

Table 1.

Definitions used to classify measures

	Clinical relevance	Psychometric properties
Appropriate	Measures a clinically relevant outcome	Good to excellent reliability and validity in ASD with information available on all relevant indices of reliability and validity.
Appropriate with Conditions	Measures a clinically relevant outcome	Good to excellent reliability and validity with information on several, but not all, pertinent indices (e.g., limited information on test-retest reliability or convergent validity). At least one drawback such as only certain subscales are related to anxiety, limited coverage or available data in a narrow age group.
Potentially Appropriate/Promising	Measures a clinically relevant outcome	Emerging or inconsistent data on reliability and validity in ASD (e.g. good to excellent reliability and validity in some – but not all – indices) or evidence of reliability and validity in typically developing children, but limited data in ASD
Unproven	Measures a clinically relevant outcome	Inadequate data on reliability and validity (e.g. studies covering one aspect of reliability or validity)
Not Appropriate	Items are not relevant to measurement of anxiety	May have solid reliability and validity, but is unlikely to be useful as a measure of anxiety (e.g., a reliable and valid diagnostic measure such as the ADOS or a measure focused on repetitive behavior), or evidence that the measure has inadequate reliability or validity.

Results

The review identified 38 published papers describing 12 instruments. Following initial review, 10 instruments were evaluated in detail. Of these, four were considered Appropriate with Conditions. Three measures were considered Potentially Appropriate and three were rated Not Appropriate (i.e., not likely to be useful for measuring anxiety in clinical trials). Table 2 summarizes the evaluation of each measure considered in this review (for a detailed description see Appendix 1). Based on the evaluative criteria, the following measures were recommended as Appropriate with Conditions for use in clinical trials to evaluate outcome.

Table 2.

a. Descriptive information, evaluative criteria for anxiety measures considered Appropriate with Conditions for youth with ASD
Instrume nt	Type/ Informant	Ages and need for verbal skills	Clinical ly Releva nt	Psychometric properties for anxiety		Sensitivi ty to Change	Used in a DD/ASD populati on	Burde n	Available in other languages*	Comment
				Reliabl e	Valid
Child and Adolescent Symptom Inventory (CASI)	Caregiver, teacher	5–18 years, yes	Yes	Yes	Yes	Not shown	Yes	Low	Yes	a, b, d
Multi- dimension al Anxiety Scale for Children (MASC)	Self, caregiver	8–19 years, yes	yes	Yes	Yes	Yes	Yes	Low	Yes	a, c, e
Anxiety Disorders Interview Schedule (ADIS)	Clinician interview of parent & child	6–18 years, yes	yes	Yes	Yes	Yes	Yes	High	Yes	c, e, f
Pediatric Anxiety Rating Scale (PARS)	Clinician interview of parent & child	6–17 years, yes	Yes	Limited	Yes	Yes	Yes	High	Yes	a,c,f

b. Descriptive information, evaluative criteria for anxiety measures considered Potentially Appropriate with Conditions for youth with ASD
Instrume nt	Type/ Informant	Ages and need for verbal skills	Releva nt	Psychometric properties for anxiety		Sensitivi ty to Change	Used in a DD/ASD populati on	Burde n	Available in other languages*	Comment
				Reliabl e	Valid
Anxiety, Depressio n and Mood Scale (ADAMS)	Caregiver	10–80 years, unknown	Yes	Yes	Yes	Not shown	Yes	Low	Yes	b
Revised Child Anxiety and Depressio n Scale (RCADS)	Self, Caregiver	9–18 years, yes	Yes	Yes	Yes	Not shown	Yes	Low	Yes	a, b, c
Screen for Child Anxiety Related Disorders (SCARED)	Self, Caregiver	6–17 years, yes	Yes	Yes	Yes	Yes	Limited	Low	yes	a, b, c

^*Measure has been translated into one or more non-English language, but reliability and validity of translation may vary; There is limited information on translations;

^a=incomplete information on convergent validity;

^b= sensitivity to change not yet demonstrated;

^c= available data on higher functioning youth only;

^d= low coverage on social anxiety;

^e=high reliance on language;

^f= requires training for clinicians and involves high time burden for respondents

^*Measure has been translated into one or more non-English language, but reliability and validity of translation may vary;

^a=limited information in ASD population;

^b=includes anxiety and mood items with limited coverage of anxiety;

^c=high reliance on language

Appropriate with Conditions

Child and Adolescent Symptom Inventory-4^th Edition Revised (CASI-4R, Gadow and Sprafkin, 2002; Hallett et al., 2013)

The CASI-4R is an informant-completed (parent or teacher) scale based on the DSM-IV for children between 5–18 years of age. Items are rated on a 4-point Likert scale (0 to 3, with higher scores indicating greater severity). The parent version contains 132 items and the teacher version contains 105 items. The parent-rated CASI-4R has been used as a psychiatric screening measure in a number of clinical trials in children with ASD (RUPP Autism Network, 2002; RUPP Autism Network, 2005; King et al., 2009).

Nested within the CASI-4R are 26 DSM-IV-based anxiety items across several anxiety disorders. Sukhodolsky et al. (2008) showed that a subset of 20 derived from the 26 original anxiety items is a reliable scale with evidence of divergent validity from measures of irritability, repetitive behavior and hyperactivity. In a larger sample of 415 children with ASD ages 5 to 17 (roughly half below IQ of 70), Hallett et al. (2013) showed similar results and identified 4 factors: Generalized Anxiety, Separation Anxiety, Social Anxiety, and Over-arousal. The total score on the 20-item scale (range 0 to 50; mean of 14.2 ± 9.4) showed an orderly distribution from low to high scores suggesting that parents are able to detect anxiety symptoms in children with ASD, including those functioining in the range of intellectual disability (ID). Internal consistency for the total score was 0.87 with no differences by age or gender. Three of the 20 items were rarely endorsed by parents, presumably because these items are highly reliant on verbal expression and require awareness of internal state. In the group of children with IQ below 70, four additional items were endorsed by parents for less than 5% of the subjects. All four of these items also relied on the child’s verbal expression (e.g., items beginning with complains or worries). Not surprisingly, mean total scores were lower in the ID group. To date, the 20-item CASI Anxiety scale has not demonstrated sensitivity to change. The investigators noted that social anxiety is under-represented on the current version of the CASI Anxiety scale and that additional work is needed to cover anxiety symptoms in lower-functioning children.

In the studies by Sukhodolsky et al. (2008) and Hallett et al. (2013), the 20-item CASI Anxiety scale was extracted from the 132-item instrument. White and colleagues (2013) used the 20-item CASI Anxiety scale as a free-standing measure in a sample of 30 adolescents (age 12 to 17 years; 23 males and 7 females) with high-functioning ASD (verbal IQ > 70) and at least one anxiety disorder. Although the results of this clinical trial are not yet published, this preliminary report focused on baseline data of subjects enrolled in a randomized trial of cognitive behavioral treatment. At baseline, the mean score on the CASI Anxiety scale was 18.5 ± 9.1; Cronbach’s alpha was 0.85. The correlation with the parent-rated Multidimensional Anxiety Scale for Children (see below) was 0.78, suggesting strong convergent validity.

The Multidimensional Anxiety Scale for Children (MASC; March et al., 1997)

The MASC is a 39-item scale scored from 0 to 3 that can be rated by parents or completed as a self-report. There is evidence for reliability and validity in typically developing children age 8 and older (March et al., 1997; Thayler, Kazemi, & Wood, 2010) and preliminary evidence supporting the use of the parent-rated version in 72 higher-functioning (Full Scale and Verbal Comprehension IQ > 70) youth with ASD between the ages of 7 to 17 years (Storch et al., 2012a). Items are distributed on several subscales: Physical Symptoms Scale, Social Anxiety Scale, Harm Avoidance Scale, and Separation/Panic Scale, as well as a Total Anxiety scale (sum of all items). The Total score showed a correlation of 0.4 with the Pediatric Anxiety Rating Scale, suggesting only modest convergent validity (see below). A review of the items indicates that the MASC is highly dependent on language, which may limit its use in ASD to higher-functioning children. This limitation may be especially true for the child self-report version. Although the MASC includes subscale scores, the total score of the parent-rated version has shown promise as an outcome measure in two pilot trials of cognitive behavioral therapy in high functioning children with ASD (Wood et al., 2009a; Storch et al., 2013).

The Pediatric Anxiety Rating Scale (PARS; Ginsburg et al. 2011; RUPP Anxiety Study Group, 2002)

The PARS is a clinician-rated measure developed by the RUPP Anxiety Study Group (2002) to rate the current severity of anxiety in typically developing children 6–17 years of age with anxiety disorders. The interview usually involves the parent and the child together using a semi-structured format. In some cases, however, the parent and the child are interviewed separately. The PARS begins with a checklist of possible anxiety symptoms across a range of anxiety disorders. The endorsed symptoms are then rated by the clinician on seven dimensions of severity on a 0-to-5 scale for each dimension. These dimensions include Number, Frequency, Distress, Level of Physical Symptoms, Avoidance, Interference at Home and Interference outside the Home. The PARS total score is the sum of the seven severity items from 0–35. Although the PARS includes seven dimensional scales, the two largest trials in typically developing children with anxiety disorders dropped the Number and Level of Physical Symptom dimensions and used only five dimensions (RUPP Anxiety Study Group, 2001; Walkup et al., 2008).

The psychometric properties of the PARS were examined in a study of 72 high-functioning children with ASD (Full Scale and Verbal Comprehension IQ > 70; age 7-to-17 years) (Storch et al., 2012a). The internal consistency was low at .59. The study showed that the PARS was reliable across raters with an intraclass correlation (ICC) of .86, though the ICC varied widely across the seven dimensions. The test-retest reliability was also excellent (ICC = .83) for the total score and consistent across the seven dimensions. Correlations with other anxiety measures such as the MASC was moderately low (r = 0.40) and similar in magnitude to parent-rated measures of disruptive behavior (r = 0.37 on the Aggression dimension of the Child Behavior Checklist). These results only partially support the convergent and divergent validity of the PARS as a measure of anxiety in ASD. In addition, the child-interview portion of the PARS requires fluent language, which may limit its use to higher-functioning children and adolescents with ASD. This potential limitation might be addressed by testing the validity of using the parent as the primary informant for the PARS across the full range of IQ.

Anxiety Diagnostic Interview Scale for DSM-IV (Parent and Child)

The ADIS (Grills et al., 2003; Silverman et al., 2001; Silverman & Albano, 1996; Wood et al., 2002) is a reliable and valid, clinician-administered, semi-structured interview for assessing the presence and severity of DSM-IV anxiety disorders as well as Dysthymia and Major Depression, ADHD, Conduct Disorder, and Oppositional Defiant Disorder. Each diagnosis is assigned a clinician’s severity rating (CSR), which is a 0–8 rating of symptom severity and functional impairment. A minimum CSR of 4 is required to assign a particular diagnosis. By convention, the primary diagnosis is the one with the highest CSR. The ADIS is often conducted jointly with parent and child, but may be conducted separately. If conducted separately, the clinician integrates the responses to derive composite diagnoses (Silverman & Albano, 1996).

The ADIS has been used as a diagnostic interview in several studies in typically developing children with anxiety disorders (Walkup et al., 2008). It also has evidence of reliability and validity in high-functioning children and adolescents with ASD (Renno & Wood, in press; White et al., 2013; Storch et al. 2012b). In typically developing children and youth with ASD, it has been used as an outcome measure to document the rate of remission – i.e., the proportion of subjects who no longer meet diagnostic criteria (Ginsburg et al., 2011; Storch et al., 2013; Wood et al., 2009b). Whether future clinical trials in children with ASD will focus on discreet anxiety disorders or more dimensional indices of anxiety is not clear. Nevertheless, the ADIS is likely to be useful in the characterization of clinical trial participants and perhaps as an outcome measurement in the context of early treatment development. The extended time required to conduct the interview, however, makes it an unlikely choice as a repeat measure in a large-scale randomized clinical trial. In the high-functioning sample of 85 children with ASD, Storch and colleagues (2013) noted only fair agreement between parents and children, with parents often reporting more symptoms than children. This disparity in parental and child reporting of anxiety symptoms would likely be accentuated in lower-functioning children with ASD.

Instruments Judged Potentially Appropriate as an Outcome Measure for Anxiety in ASD

Anxiety, Depression and Mood Scale (ADAMS, Esbensen et al., 2003)

The ADAMS is a 28-item informant-rated scale scored from 0 to 3 (from “not a problem” to “severe problem”) designed to assess mood and anxiety symptoms. Five factors emerged from a series of assessments conducted by Esbensen et al. (2003): Depressed Mood; Social Avoidance; Generalized Anxiety; Manic/Hyperactive; Compulsive Behavior. Social Anxiety and Generalized Anxiety include a total of 14 items. As presented, the Social Anxiety and Generalized Anxiety subscales are separate factors. How the scale might perform as a single index for total anxiety is not clear. The internal consistency of the Generalized Anxiety subscale was .83 and test-retest reliability was .78. There are no items for separation anxiety, which may limit its use in school-age children. It is a potentially useful instrument because of its brevity and behavioral basis (most items can be observed by a caretaker). However, the reliability and validity of the 28-item ADAMS have only been evaluated in a sample of individuals (ages 10 to 79 years) with ID, and the extent of participation by individuals with ASD is unknown. Thus it is unclear how well it would perform in youth with ASD in the average IQ range.

The Revised Child Anxiety and Depression Scale (RCADS; Chorpita et al., 2000; 2005; 2008)

The RCADS is a 47-item parent or self-report questionnaire for children 9-to-18 years of age. Thirty-six items (rated on a 0-to 3-scale) focus on anxiety in the following subscales: Separation Anxiety Disorder, Social Phobia, Generalized Anxiety Disorder, Panic Disorder, and Obsessive-compulsive Disorder. The RCADS has solid indices of reliability and validity in typically developing children. To date, only one study has evaluated the RCADS in children with ASD (Hallett et al., in press). In that study, 129 twin pairs with at least one co-twin affected by ASD was compared to 80 control twin pairs. The sample (age 10 to 15 years) was recruited from a longitudinal community twin sample in England and Wales. Participants were assessed with the RCADS (parent and child self-report version), the Autism Diagnostic Interview-Revised (ADI-R), the Autism Diagnostic Observation Schedule – Generic (ADOS) and an IQ test consistent with the child’s ability. Following the assessment children were classified as ASD, subclinical ASD, unaffected co-twins and controls. The internal consistency of the total anxiety score on the parent-rated RCADS was 0.93. The ASD group exceeded unaffected co-twins and controls on parent-rated total anxiety scores and exceeded controls on all RCADS subscales. These trends were evident in the child self-reported scores on the RCADS, but the differences between ASD subjects and controls were not statistically significant. Children with ASD, with IQ in the normal range had higher scores on the Social Anxiety scale compared to those with ID. These results suggest that the RCADS is a potentially useful scale for children with ASD. However, test-retest reliability, convergent and divergent validity and sensitivity to change in the ASD population have not been evaluated.

The Screen for Child Anxiety Related Disorders (SCARED; Birmaher et al., 1997; 1999; Su et al., 2008)

The SCARED contains 41 items rated on a 3-point scale (0–2) that can be completed by parents or as a child self-report form. Items are distributed on five subscales: Somatic/panic, Social Phobia, Separation Anxiety, Generalized Anxiety, and School Phobia (Birmaher et al., 1999). Data in typically developing children support its reliability and validity. Although it has five subscales, studies in typically developing children have used the total score, which has shown sensitivity to change with treatment (Cartwright-Hatton et al., 2011). There is limited information on the use of this measure in children with ASD. The SCARED was used in a CBT trial in a sample of high function children with ASD (Reaven et al., 2009). In that study, 10 children (age 8 to 14 years) with ASD and at least one anxiety disorder were enrolled in a structured, 12-week group-CBT program. The outcomes for these 10 subjects were compared to outcomes for 23 children on a 3-month waitlist. The subjects on the waitlist were not randomly assigned to that treatment condition – but joined the trial after the first group of 10 children were in the CBT program. The total score on the parent-rated SCARED declined 34% in the CBT group, which was significantly better than the 6% reduction in the control group. These results provide partial support for the use of the SCARED as an outcome measure in high functioning children with ASD. Given the small sample size in the absence of random assignment, however, these results warrant considerable caution. In addition, as with other measures cited in this review, the reliance on language with the SCARED may limit its use in ASD.

Instruments Judged Not Appropriate as an Outcome Measure for Anxiety in ASD

Revised Children’s Manifest Anxiety Scale (RCMAS; Reynolds and Richmond, 1978)

The RCMAS is a 37- item child self-report scale that uses a “yes-no” format. It consists of three subscales (Physiological Symptoms, Worry and Oversensitivity, and Social Concerns) as well as a Total score. It also includes nine items that comprise a lie scale. The RCMAS has been used in several psychotherapy trials in typically children with anxiety disorders (Barmish and Kendall, 2005 for a review). Most of these studies were conducted in the 1990s when there were limited choices for measuring effects of treatment on anxiety in children with anxiety disorders. During this same period, however, investigators began to question the specificity of the RCMAS as a measure of anxiety versus a measure of general distress that over-lapped with depression (Curry and Craighead, 1990; Perrin and Last, 1992). In a classroom survey of 632 children, Dierker et al. (2001) showed that the RCMAS did not discriminate between anxiety and depression. The study also included the MASC, which was able to discriminate between these two symptom domains.

The RCMAS was used as an outcome measure in a psychotherapy trial in children with high-functioning ASD. In this sample of 47 children (35 boys; mean age 10.8 years), subjects were randomly assigned to 12 sessions of CBT or to waitlist (Chalfant, Rapee and Carroll, 2007). After 12 weeks of treatment, there was a 69% drop in the RCMAS Total score compared to no change in the waitlist group. Despite these results, the RCMAS has several drawbacks as an outcome measure. First, the yes-no format offers a count, but little insight into severity of symptoms. Second, it may be completed by higher-functioning children, as in the trial by Chalfant and colleagues, but would not likely be useful for lower functioning subjects. Third, as described above, children tend to under-estimate their anxiety symptoms, suggesting that parent ratings may be more informative. Fourth, the RCMAS has not been used as an outcome measure in many randomized trials, even among typically developing children with anxiety disorders. Although the RCMAS may be useful as a screening measure in higher-functioning children, the CASI, RCADS, SCARED or MASC provide wider scoring range making them more suitable as outcome measures‥

The Nisonger Child Behavior Rating Form (NCBRF, Aman et al., 1996; Lecavalier et al., 2004)

The NCBRF is a factor-analytically derived scale with items rated from not true (0) to completely or always true (3). There are two versions of the NCBRF: a parent and teacher version. The 60-item parent version and the 62-item teacher version are distributed on six subscales: (1) Conduct Problem, (2) Insecure/Anxious, (3) Hyperactive, (4) Self-Injury/Stereotypic, (5) Self-Isolated/Ritualistic, and (6) Overly Sensitive (parent version) or Irritable (teacher version). With the exception of the Overly Sensitive/Irritable subscales, both versions share similar subscale content. Raters are instructed to consider both the rate of occurrence and the degree to which the behavior was a problem over the previous month. The Conduct subscale of the NCBRF has been used as an outcome measure in placebo controlled trials of children with mild developmental disabilities (Aman et al., 2002; Snyder et al., 2002). It has also been used to characterize a large sample of children with ASD (Lecavalier, 2006). The two subscales (Insecure/Anxious; Overly Sensitive) germane to this review contain a limited number of items relevant to anxiety. The low coverage of anxiety suggests that the NCBRF is not likely to be as useful a dimensional measure of anxiety in children with ASD as several other tools identified earlier.

The Child Behavior Checklist (CBCL; Achenbach, 2005; Achenbach and Rescorla, 1991; Achenbach and Rescorla, 2001; Dutra et al., 2004)

The CBCL is a 120-item parent-rated instrument designed to measure behavioral and emotional problems and social competencies in typically developing children 4–18 years of age. Few rating instruments in the child mental health field have the level of empirical support for reliability and validity as the CBCL. The behavior problem items, which are scored 0 to 2, are classified in one of eight empirically-derived narrow dimensions and can also be mapped to DSM-IV-oriented subscales (Ebesutani et al., 2010). However, the Anxious/Depressed dimension has only 14 items including both anxiety and mood. The CBCL has been used in a number of studies with children with ASD, primarily for characterization (Fischbach and Lord, 2011). The relatively low number of anxiety items raises questions about the trade-off between informant burden and a measure with inadequate coverage.

Discussion

This review illustrates the challenges of measuring anxiety symptoms in children with ASD. Among the several measures that were reviewed, only four measures were rated as “Appropriate with Condition” for use in clinical trials focused on anxiety . These included the parent-rated 20-item CASI, parent-rated MASC, PARS and ADIS. These measures differ in format, time of administration, and need for trained raters. Informant-based measures (e.g., 20-item CASI, MASC) have the advantage of relative brevity and have emerging support for reliability and validity in children and adolescents with ASD. In contrast to double-blind placebo-controlled trials, however, informant-based reports are not blinded in psychosocial intervention trials. Therefore, the use of ratings conducted by a clinician blind to treatment condition may reduce potential bias in outcome measurement.

Although the 20-item CASI Anxiety Scale and MASC, ADIS, and PARS each have an emerging track record for application in clinical trials in children with ASD, each measure has caveats for use in this population. The parent-rated 20-item CASI Anxiety scale showed a distribution of scores from low to high in a sample of 415 children with ASD across a wide range of intellectual and verbal ability (Hallett et al., 2013). This information could be used as a screen for anxiety on children with ASD or for setting a criterion score for entry in a clinical trial focused on anxiety symptoms. In its current form, however, the 20-item CASI does not have adequate coverage of social anxiety. Given the state quality of the items and the broad distribution of scores in children with ASD, this scale is likely to be sensitive to change, but this has not been yet been clearly demonstrated. The parent-rated MASC appears to perform adequately in higher functioning children with ASD, but several items require the child’s verbal expression of worries and fears. The clinician-rated ADIS and PARS have been used to evaluate change with treatment in higher-functioning (IQ > 70) adolescents with ASD in pilot CBT trials (Reaven et al., 2012; Storch et al., 2013; White et al., 2013; Wood et al., 2009a). In its usual interview format, the PARS incorporates information obtained from the child and the parent. Even if the parent is the primary informant, the PARS symptom checklist and the severity dimensions rely on parental awareness of verbal expression of anxieties by the child. Thus, in current form, the PARS may be useful only in high-functioning children and adolescents with ASD. The ADIS, which was developed as a diagnostic interview, has been used to document remission of anxiety disorders in higher functioning youth with ASD. Given its length, however, the added cost and subject burden could limit the feasibility of using the ADIS in a large-scale trial.

The results of this review raise questions about the underlying relationship of anxiety and ASD. First, anxiety disorders may be independent of ASD and reflect a co-occurring condition (Gotham et al., 2012; Kerns and Kendall, 2012). Second, anxiety symptoms may be inextricably linked to core features of ASD. In this model, anxiety symptoms may be distributed from low to high in children with ASD just as other aspects of ASD such as language delay (Hallett et al., in press; White et al., 2009). Third, there may be certain genetic or environmental influences that elevate the risk for ASD and anxiety in some children. In this model, anxiety and ASD may be separate, but not independent such that the presence of one amplifies the other (Hallett et al., 2009).

Anxiety as a separate co-occurring problem in children with ASD

In this view, anxiety disorders in children with ASD are essentially the same as anxiety disorders in typically developing children. If anxiety disorders are simply superimposed on a subgroup of children with ASD, treatments would presumably be similar to those used in typically developing children. For example, high-functioning youth with ASD with anxiety disorders have shown a positive response to CBT in several pilot trials (Reaven et al., 2012; Storch et al., 2013; White et al., 2013; Wood et al., 2009a).

A drug development program built on the notion that anxiety disorders are independent from ASD and not different from anxiety disorders in typically developing children would likely encounter the regulatory issue referred to as pseudospecificity (Laughren & Levin, 2006). This term reflects the FDA’s concern about a drug approval built on a narrow indication. For example, a drug development program focused on separation anxiety in children with ASD would face questions about whether separation anxiety is somehow different in ASD than in typically developing children. In the absence of empirical support for this proposal, the FDA might insist that a sponsor test the new drug for safety and efficacy in children with separation anxiety generally – rather than the more narrowly defined population of children with ASD (Laughren, 2003).

Anxiety as an aspect of ASD

The clinical characteristic “insistence on sameness” so often observed in children with ASD has been proposed as a manifestation of anxiety in children with ASD (White et al., 2009; Gotham et al., 2012). In this view, the tendency toward strict adherence to daily routines and the over-reaction to alteration in the routine may be driven by anxiety or may induce anxiety. In some children, their concern about the daily routine may give rise to vigilance about the upcoming sequence of events in order to detect any deviation from the routine. Other observable behaviors such as protest on separation, phobic avoidance of group situations or noisy environments (e.g., school cafeteria) may also be expressions of anxiety in children with ASD. On the other hand, although this strict adherence to routines and phobic avoidance are often observed children with ASD, these behaviors are not true for all children with ASD. Similarly, the over-reaction to change in routine is not uniformly observed in children with ASD. Thus, given our current state of knowledge, it may be difficult to identify the cluster of observable behaviors that uniquely reflect anxiety in children with ASD. To be generally applicable in clinical trials, this cluster of behaviors would have to be detectable in children across the full range of IQ and language ability.

Gotham and colleagues (2012) examined the association between anxiety and ASD in a sample of 1,400 well-characterized children with ASD (ages 5 to 18 years). Anxiety symptoms were only weakly correlated with a measure labeled insistence on sameness. Moreover, anxiety and insistence on sameness were not associated with core features of ASD. The severity of anxiety symptoms was associated with other behavioral problems such as inattention, aggression and irritability. In a sample of 88 high-functioning youth with ASD, Renno and Wood (in press) also observed that anxiety was independent of autism symptom severity. On the other hand, the population twin study by Hallett and colleagues (in press) observed that unaffected controls of ASD probands had higher mean scores on several anxiety subscales compared to control twins. The authors suggested that this observation may reflect “overlap” of anxiety and ASD traits.

Anxiety as separate but not independent of ASD

Current models of anxiety in the general population focus on the balance of two interrelated neural processes: threat appraisal mediated by the limbic system and executive functioning mediated by the prefrontal cortex (LeDoux, 1998). In this model, individuals with high anxiety tend to view ambiguous stimuli as threatening, which may compromise executive functioning and promote anxiety symptoms (e.g., excessive worry, sympathetic arousal, avoidance). Deficiencies in executive functioning may render children with ASD less able to manage emotional arousal in the context of even mildly threatening external stimuli (e.g., change in routine, new situations) giving rise to anxiety. In a longitudinal, population-based study of approximately 6,000 twin pairs, Hallett and colleagues (2010) observed a small, but significant correlation of internalizing traits (anxiety and depression) and autistic traits at Time 1 (subjects were 8 years-old) and Time 2 (subjects were 12 years-old). The longitudinal design permitted examination of the association of autistic traits and internalizing traits at Time 1 with the same measurements at Time 2. Internalizing traits measured at Time 1 and Time 2 were strongly associated. Not surprisingly, autistic traits were also strongly associated across the two time points. In a sophisticated cross-lagged model, the results also showed that internalizing traits at Time 1 were modestly associated with autistic traits at Time 2. However, autistic traits at Time 1 were more strongly associated with internalizing traits at Time 2, suggesting an interaction between autistic traits at Time 1 and internalizing traits at Time 2.

Given the limited ability of youth with ASD to communicate emotions to caregivers and the presumed deficit in managing emotions, youth with ASD may be at increased risk for anxiety. Anxiety may be manifested by a range of maladaptive behaviors including tantrums, noncompliance, social avoidance, repetitive behaviors, aggression or self-injury. Because these behaviors may also occur for other reasons, however, it may be difficult to claim that these behaviors uniquely reflect anxiety in children with ASD. Nevertheless, if the manifestations of anxiety in children with ASD are fundamentally different than the manifestations of anxiety in typically developing children, measures of anxiety will require modification for the ASD population. The ensuing challenge for drug development is to build a reliable and valid outcome measure that captures these unique manifestations of anxiety in children with ASD.

Limitations

The limitations of this review reflect the current state of the art for measuring anxiety in youth with ASD. Prior to conducting the review, we identified various indices of reliability (internal consistency, test-retest, inter-rater) and validity (construct, content, divergent, convergent). Although we intended to apply benchmarks for these relevant indices, reports varied in their methodological rigor and in their examination of the psychometric landscape. For example, a study may show low-to-medium correlations with measures of mood or disruptive behavior (supporting divergent validity), but fail to report higher correlation with other measures of anxiety (convergent validity). In addition, available studies varied in sample size and sample composition, often with limited participation of ethnic and racial minorities. Given the few treatment studies focused on anxiety in youth with ASD, there was also limited information on sensitivity to change for currently available instruments. Consequently, the review could not identify a clear front-runner for measuring anxiety symptoms in youth with ASD. Thus, we did not use independent reviewers, followed by assessment of their agreement and consensus based on a full list of psychometric properties. Indeed, the entire committee reviewed all measures, discussed the findings and came to consensus through the comparison of one measure with the others.

In summary, anxiety symptoms are common in children with ASD and may contribute to greater impairment. Applying outcome measures used in typically developing children with anxiety disorders may be less than satisfactory, especially in lower-functioning children with ASD. Future studies could consult with parents to flesh out the manifestations of anxiety in children with ASD. The use of neuroimaging, eye tracking, startle paradigms, heart rate variability and galvanic skin response may be useful in treatment development as proof of concept and as surrogate outcome measures. New measures or modifications of existing measures will require attention to reliability, validity, sensitivity to change and subject burden.

Appendix #1: Descriptive and Evaluative Criteria for Measures Reviewed

1. Respondent type indicates who reports on behavior, including self, caregiver, clinician, or other person (sibling, classmate).

2. Age and requirement of verbal skills reflects the degree to which the measure requires verbal skills by the person with ASD and the extent to which it can be used across a range of developmental level and chronological age. To be reliable and valid, some measures may rely less on language and more on other communication skills. For example, individuals may have language but may not have the pragmatic skills to meet the demands of some tests. Other measures may require the individual to respond within a specific time period or require a motoric response. These tests may require adequate receptive language skills. Abilities and knowledge in these areas may vary for children with ASD. The notation in this category indicates whether verbal skills are required by the instrument to assess the domain of interest.

3. Comment on the use in ASD, DD population or both follows from the unique challenges of assessing behavior in youth with ASD. A “yes” in this column indicates that the instrument has been studied in one or more samples of individuals with ASD (e.g., community sample, clinical sample, an intervention trial).

4. Clinical relevance reflects the degree to which the measure captures dimensions of anxiety that are applicable to youth with ASD.

5. Reliability reflects the extent to which the measure assesses anxiety consistently. Measures of reliability include internal consistency (items or dimensions contribute evenly to the total score), test-retest reliability (correlation of score over brief periods of time), inter-rater reliability (level of agreement across raters). Inter-rater reliability is particularly important in clinician-rated instruments (e.g., Pediatric Anxiety Rating Scale). The demand for inter-rater reliability may be lower for informant-based measures (e.g., parents and teachers may not be in agreement about a child’s behavior given the differences in setting). The following rating system was adapted from the Centre for Childhood Disability Research (2004) for this review.

   • Excellent (e.g. publication of more than 3 studies with reliability statistics in the excellent range plus use of the measure in at least two clinical trials with adequate statistical power.
   • Good (e.g. 2 reliability studies completed with adequate to excellent reliability values plus use of at least one clinical trial with adequate statistical power.
   • Adequate (e.g. 1 reliability study completed with adequate to excellent reliability values)
   • Poor (e.g. reliability studies poorly completed, or reliability studies showing inadequate levels of reliability)
   • No evidence available

6. Validity reflects the degree to which the instrument measures what it purports to measure. The review considered (a) content validity, i.e., The degree to which the measure adequately reflects anxiety; (b) construct validity, i.e., the degree to which the measure positively correlates with another measure of anxiety (convergent validity) and the degree to which it does not correlate with a measure presumed to measure a construct separate from anxiety (divergent validity). The following ratings were used:

   Content Validity

   • Excellent: (e.g. expert judgment or statistical method (e.g. factor analysis) was used to determine that the measure adequately covers the domain of interest without inclusion of unrelated material;
   • Adequate: Has reported content validity but no specific method was used
   • Poor: Instrument is not comprehensive
   • No evidence available
   Construct Validity

   • Excellent: (e.g. more than 2 independent, experimental studies have shown adequate agreement with a gold standard measure, stronger for overall evaluation if determination of both convergent and divergent validity reported)
   • Adequate: (e.g. 1 to 2 studies demonstrate confirmation of theoretical formulation; determination of either convergent or divergent validity reported)
   • Poor: (e.g. construct validation poorly documented or construct validity not supported by the study)
   • No evidence available
7. Sensitivity to change. Measures with sufficient assay sensitivity are typically used as the primary endpoint in pivotal clinical trials in support of regulatory approval for a new drug or device, or new indications for existing drug or device. Validity and reliability are pre-requisites for assay sensitivity, but high levels of validity and reliability do not guarantee sensitivity to change. Measures of assay sensitivity include effect sizes and receiver operating characteristic curves (Liang, 2000), among others (Evans et al., 2002). The assessment of assay sensitivity may include review of design elements (e.g., entry criteria, length of trial, and selection of control condition) and properties of the measure itself (scoring range, number of items). Although sensitivity to change is essential for a measure to be useful as a primary endpoint, failure of a given measure to demonstrate sensitivity to change may reflect misapplication of the measure, design problems in the trial or lack of efficacy for the intervention. Therefore, in this review, failure of a measure to demonstrate sensitivity to change would not result in discounting the potential for the measure as a primary endpoint. Ratings of demonstrated sensitivity to change were defined as follows:

   • Excellent: More than 2 experimental trials have demonstrated that measure captures changes in outcome (improvement or decline), in a clear and consistent manner
   • Adequate: 1 – 2 studies (experimental or quasi-experimental) have demonstrated that the measure captures changes in outcome, but evidence is incomplete or inconclusive. For example, evidence is limited to only one or two studies; evidence is available for more than two studies, but findings across studies are not consistent. An assessment of adequate could also be given if available evidence is inconclusive due to study design limitations making it difficult for the instrument to detect change due to variability. For example, a relatively small sample with unclear entry criteria resulting in large variability on the primary outcome.
   • Poor: Available studies have failed to show the capacity of the measure to differentiate active from a control condition.

   No evidence available or limited evidence due to type of study (open label, small pilot trial, randomized trial with design limitations, single dose study only).
8. Burden reflects the time, demand structure or difficulty associated with collection of the measure. The review considered: (1) the individual with autism, (2) other persons who would complete the measure, such as parents, teachers, and clinicians, and (3) the investigator. For the individual with ASD, considerations included whether the test involves an unusually high level of demand, such as sitting motionless for an extended period of time, or an expectation that a person with ASD might find uncomfortable or particularly challenging, such as interacting with groups of people, being touched, or required to comprehend complex, verbal instructions. The length of a measure and frequency of measurement could also affect burden. For informants such as a parent, clinician, or teacher, the review also included length and frequency of the test, level of literacy required, or features of the test that might make the informant uncomfortable, such as inquiring about highly personal matters. For clinician-rated interviews, considerations include the level of training or specific qualifications required to conduct the measure, need for detailed supervision following collection of the measure, whether specialized and/or costly equipment is needed, high cost of administration (e.g., need for complex coding of behavior before analyzing data) and time required to complete the rating. Ratings of burden were classified as follows:

   • Low or none: The participant is observed in a natural environment for 15 minutes or less or takes 15–30 minutes to complete the measure by a trained observer. The measure is relatively brief (< 20 minutes), includes current observable behaviors (e.g., past month or less) and requires simple scale (e.g., 0 to 3). There is little or no distressing content. Measure is easily downloadable, free of charge, and simple to score and interpret. Test is available and has been validated in more than one other language other than English.
   • Medium: The participant is observed or engaged with an unfamiliar interviewer/reporter, for 15–30 minutes or informant-based rating that takes > 20 minutes to complete. An informant rating includes a long list of questions (e.g., > 60) or assessment may have < 60 questions, but requires complex directions or responses. There is low to moderate levels of verbal or pragmatic comprehension required. There may be some distressing content. It uses a Likert or Likert type scale, and the rater is required to interact and observe with participant while simultaneously recording responses. The rating may require a manual; however it does not require extensive training. Measure is not free of charge. Test is offered in at least one other language than English, but may not be validated for use in non-English languages. Need for conversion of raw to standardized scores to be calculated by an additional program at a cost.
   • High: The participant is required to spend 30 minutes or more performing tasks that require sustained attention or the assessment requires extensive preparation (e.g., fMRI) or participant training or high levels of verbal/pragmatic comprehension that may involve complex or distressing responses. The rater is required to interact and observe with participant while simultaneously recording responses using extended semi-structured or non-standardized measures. The measure requires extensive training, must be administered by clinician with at least master’s level and the measure requires > 45 minutes to administer. Measure may not require extended time to administer, but it may involve distressing content or evoke distressing responses. Use of the tool is costly and administration would require investigator to provide additional training to personnel. Test is only available in English and requires use of translator.
9. Overall rating involved the inclusion of each category Most weight was placed on clinical relevance, psychometric soundness, and use in ASD population. However, sensitivity to change and burden were also considered when determining rating as follows:

   • Appropriate: Measure achieved high ratings on clinical relevance; good to excellent reliability and validity with information available on all pertinent indices (e.g., test-retest, internal consistency and construct validity) across a range of youth with ASD or developmentally disabled population. On clinician-rated measures, high priority was given to inter-rater reliability (consistency across informant ratings such as parents and teachers was less important). If sensitivity to change has been demonstrated in an ASD or developmentally disabled population, this enhanced the assessment of the rating. The instrument has low to medium burden to individual, parent, and clinician.
   • Appropriate with Conditions: Measure was considered appropriate with one or more qualifications. The psychometric properties have been evaluated in an ASD or developmentally disabled population, but available information is restricted to a specific age range or a specific subgroup such as high functioning ASD. Measure was considered relevant with good to excellent reliability and validity with information on several, but not all, pertinent indices (e.g., limited information on test-retest reliability or convergent validity). On clinician-rated measures, high priority was given to inter-rater reliability (consistency across informant ratings such as parents and teachers was less important). However, it may not be available in multiple languages and translation would be needed for a large-scale international study. Measure has medium or high level of burden and some data on sensitivity to change.
   • Potentially Appropriate: The instrument measures a clinically relevant outcome; however, there is emerging or inconsistent reliability and validity in ASD or developmentally disabled populations. The measure may have demonstrated validity for screening or diagnosis, but no evidence supporting its use as an outcome measure. Measure may be appropriate for a subgroup in the ASD population, but more study is needed to establish the reliability, validity and usefulness as an outcome measure.
   • Unproven: The instrument measures a clinically relevant outcome. However, reliability and validity have been inadequately evaluated or not evaluated. Measure may have evidence for use in other pediatric populations, but has little or no evidence supporting its use in an ASD or developmentally disabled populations.
   • Not Appropriate: Measure is not relevant to the outcome of interest or was developed for screening purposes rather than as a change measure. Available information indicates poor reliability, validity or both.