Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Feb 1.
Published in final edited form as: Autism. 2019 Aug 7;24(2):400–410. doi: 10.1177/1362361319866561

Anxiety in 3 to 7-year old children with Autism Spectrum Disorder seeking treatment for disruptive behavior

Denis G Sukhodolsky 1,*, Luc Lecavalier 2, Cynthia Johnson 3, Tristram Smith 4, Naomi Swiezy 5, Karen Bearss 6, Carla B Kalvin 1, Lawrence Scahill 7
PMCID: PMC6982590  NIHMSID: NIHMS1534280  PMID: 31390873

Abstract

Background:

Anxiety is a common and impairing problem in children with ASD, but little is known about it in preschool children with ASD. This paper reports on the characteristics of anxiety symptoms in young children with ASD using a parent-completed rating scale.

Methods:

180 children (ages 3 to 7) participated in a clinical trial of parent training for disruptive behaviors. Anxiety was measured as part of pre-treatment subject characterization with 16 items from the Early Childhood Inventory (ECI), a parent-completed scale on child psychiatric symptoms. Parents also completed other measures of behavioral problems.

Results:

Sixty seven percent of children were rated by their parents as having two or more clinically significant symptoms of anxiety. There were no differences in the ECI anxiety severity scores of children with IQ<70 and those≥70. Higher levels of anxiety were associated with severity of oppositional defiant behavior and social disability.

Conclusions:

Anxiety symptoms are common in preschoolers with ASD. These findings are consistent with earlier work in school-age children with ASD. There were no differences in anxiety between children with IQ below 70 and those with IQ of 70 and above. Social withdrawal and oppositional behavior were associated with anxiety in young children with ASD.

Keywords: Autism, Autism Spectrum Disorder, Anxiety, Young Children, Early Childhood Inventory

Introduction

Fears and anxious worries are common in the general population of children and can be detected in children as young as two years of age (Costello, Egger, & Angold, 2005). Prevalence estimates of anxiety disorders in school-age children and adolescents range from 6.5% to 32% (Costello et al., 2005; Kessler et al., 2012), though several studies fall in the range of 14 to 18% (Polanczyk et al., 2015). The differences across prevalence studies in youth are due to differences in sample size, source of sample, age of the sample, the diagnostic methods and the severity threshold used to set the diagnosis. For example, studies using a lifetime diagnosis and older age groups yield the highest estimates. By contrast, the prevalence of anxiety disorders in young children are less common with estimates ranging from 1.5% to 6.5% (Lavigne, LeBailly, Hopkins, Gouze, & Binns, 2009; Wichstrom et al., 2012). Advances in detection of anxiety in younger children have relied on the availability of valid and reliable assessment tools.

Despite higher prevalence rates of anxiety disorders in older children, the preschool period may be particularly salient for the development of anxiety. Anxiety symptoms, which have been linked to temperamental characteristics appearing as early as the first year of life, often manifest during the preschool years (Paulus, Backes, Sander, Weber, & von Gontard, 2015; Whalen, Sylvester, & Luby, 2017). In addition, anxiety disorders of early childhood, such as separation anxiety, can interfere with the development of age-appropriate autonomy and elevate risk for future psychopathology (Battaglia et al., 2016).

The co-occurrence of anxiety disorders and autism spectrum disorder (ASD) is an area of increased attention (Keen, Adams, Simpson, den Houting, & Roberts, 2019; Scahill et al., in press; van Steensel & Heeman, 2017). Of interest here is whether anxiety disorders in youth with ASD are the same as anxiety disorders in the general pediatric population, whether ASD increases the risk for anxiety disorder, or whether there is fundamental melding of anxiety symptoms and ASD. Although these proposals may not be settled at present, they have clinical implications. First, when considered dimensionally or categorically, it is clear that not all children with ASD have anxiety symptoms or anxiety disorders (Hallett et al., 2013; Keen et al., 2019; Scahill et al., in press; van Steensel, Bogels, & Perrin, 2011; van Steensel & Heeman, 2017). Second, social anxiety may evolve in children with ASD as they find themselves unable to negotiate the complexities of the social milieu. For such children social interaction may become unpleasant and something to avoid (Moriuchi, Klin, & Jones, 2017). Third, in a series of focus groups with parents of children with ASD, we heard from several parents that core features of ASD such as insistence on routines fostered high levels of vigilance in some children with ASD. This vigilance may reflect the child’s worry about the impending deviation from the routine and, as reported by some parents, may contribute to the child’s over-reaction when the routine is violated (Bearss, 2016). Fourth, anxiety in youth with ASD may also be conceptualized as a deficit in emotion regulation. Interactions between emotion regulation deficits and impaired ability to read social and environmental cues in ASD may contribute to the emergence of anxiety (Mazefsky et al., 2013; White et al., 2014). Thus, although not all youth with ASD are anxious, there appear to be various pathways for the emergence of anxiety in this population. Examination of anxiety symptoms in young children with ASD may provide insight into these developmental pathways.

The relevance of early childhood for the development of anxiety in children with ASD is also supported by temperament research. Similar to young children in the general population at risk for anxiety disorders, young children with ASD show higher levels of temperamental traits associated with anxiety symptomology. Macari and colleagues (2017) found that, compared to typically developing toddlers, toddlers with ASD demonstrated lower levels of effortful control and higher levels of negative emotionality (i.e., soothability), the combination of which have been linked to the development of anxiety disorders (Lonigan, Vasey, Phillips, & Hazen, 2004). Moreover, the persistence of temperamental vulnerabilities over time predicted ASD symptom severity (Macari et al., 2017).

To date, most evaluations of anxiety in ASD have been conducted in school-age children, adolescents and adults (van Steensel & Heeman, 2017; White, Oswald, Ollendick, & Scahill, 2009). Because only a few studies have been conducted in young children with ASD, little is known about anxiety in this population. In a study of anxiety in young children with ASD (age three to 5 years old) parent- and teacher-ratings on the Early Childhood Inventory (ECI), a 108-item rating scale of child psychiatric symptoms, was used to explore anxiety symptoms in a sample of 172 clinically referred children. The study showed that preschool children with ASD had higher levels of anxiety than community controls (Gadow, Devincent, Pomeroy, & Azizian, 2004). Salazar and colleagues (2015) used the Preschool Age Psychiatric Assessment (PAPA) to examine the occurrence of anxiety disorders in a sample of 101 children (age 4.5 to 9.8 years) with ASD. The PAPA is a semi structured, parent-reported interview for preschool children (Egger & Angold, 2004). The sample was ascertained from children residing in one of two London health districts born between 2000 and 2004. Sixty-six percent of children met criteria for generalized anxiety disorder, 52% for specific phobias, 15% for social phobia, and 18% for separation anxiety. Johnson and colleagues (Johnson, DeMand, & Shui, 2015) used parent-rated CBCL anxiety problems scale in a sample of 118 children ranging from 2 to 6 years. Subjects were recruited from five of the 17 Autism Speaks Autism Treatment Network (AS-ATN) sites that participated in a study of diet and nutrition. In that study, 19% were above the clinical cutoff (T score > 65) on the CBCL anxious problems scale. Anxiety was associated with sleep and feeding problems. Using the 24-item, Anxiety Scale for Children with Autism Spectrum Disorder (ASC-ASD), Keen and colleagues (2019) examined anxiety symptoms in a sample of 5- and 6-year old children (N=95) with ASD. Intolerance of uncertainty and separation anxiety were the most commonly reported symptoms.

These previous studies of young children with ASD highlight the relevance of examining anxiety in this population. In this report we use data from a multi-site trial of parent training in a well-characterized sample of young children with ASD (age 3 to 7 years) and disruptive behavior (Bearss et al., 2015; Scahill et al., 2016). The purpose of this study is to explore the frequency and distribution of anxiety symptoms in young children with ASD. Given the exploratory nature of the study, we did not have a-priori hypotheses. This study advances previous research by employing item-level analysis of anxiety symptoms on the ECI and evaluating associations of anxiety with cooccurring psychopathology, as well as with core symptoms of ASD, IQ, and adaptive functioning.

METHODS

Participants

The sample of 180 children (age 3 to 7 years) were participants in a six-month randomized trial of parent training compared to parent education (Bearss et al., 2015; Scahill et al., 2016). In addition to an ASD diagnosis, eligible participants had a score ≥ 15 on the Irritability subscale of the Aberrant Behavior Checklist (described below). Children on stable psychiatric medication (i.e., stable for 6 weeks with no planned changes for the course of the study) were allowed to participate. Children with a serious medical condition, another psychiatric condition in need of immediate treatment, current or past enrollment in structured parent training for noncompliance or receptive language skills ≤ 18 months were excluded. The diagnosis of ASD was made according to DSM-IV-TR criteria based on clinical assessment and corroborated by the Autism Diagnostic Interview-Revised (ADI-R) (Rutter, Le Couteur, & Lord, 2003) and Autism Diagnostic Observation Schedule (ADOS) (Lord et al., 2000). Lhe study was approved by the institutional review boards at each site (Emory University, Indiana University, Ohio State University, University of Pittsburgh, University of Rochester, and Yale University). Informed consent was obtained from all study participants prior to data collection. Parents received compensation to cover travel costs for each assessment and therapy visit.

Measures Used in the Current Study

Early Childhood Inventory (ECI) (Sprafkin, Gadow, Salisbury, Schneider, & Loney, 2002) is a 108-item, DSM-IV referenced parent report on child psychiatric conditions. Lhe items are scored on a 4-point scale with response categories of 0=never, l=sometimes, 2=often, and 3=very often. Lhe scale can be scored in two different ways: as symptom severity (sum of the scores for the subscale) or as a symptom count (number of items rated 2 or 3). Lhe scale has been shown to be valid and reliable in young children with ASD (Lecavalier, Gadow, Devincent, Houts, & Edwards, 2011). Subscales of interest in the current study included: Anxiety (16 items), attention-deficit/hyperactivity disorder (ADHD; 18 items), oppositional defiant disorder (ODD; 8 items), and autism spectrum disorder (ASD; 12 items). Lhe 16 anxiety items include 8 items on separation anxiety, 3 on generalized anxiety, 2 on social anxiety, 1 on obsessive compulsive disorder, and 1 on specific phobias. Lhe large number of separation anxiety items reflects the relatively high prevalence of separation anxiety disorder in children under 12 years of age (American Psychiatric Association, 2013).

Aberrant Behavior Checklist (ABC) (Aman, Singh, Stewart, & Field, 1985) is a 58-item parent-rated scale. Each item is rated from 0 (not a problem) to 3 (severe in degree). Lhe ABC contains five subscales: Irritability (15 items), Social Withdrawal (16 items), Stereotypic Behavior (7 items), Hyperactivity/Noncompliance (16 items), and Inappropriate Speech (4 items). It is reliable and valid in young children with ASD (Kaat, Lecavalier, & Aman, 2013). The 15-item Irritability subscale includes questions about tantrums, aggression, and unstable mood. The Irritability subscale score ranges from 0 to 45, with higher scores indicating greater severity. In large sample of 1,893 children who participated in the Autism Treatment Network study, the mean (SD) Irritability scale scores were 13.8 (9.8) in the 3 to 6-year-old age group.

Children’s Yale-Brown Obsessive-Compulsive Scales-ASD (CYBOCS-ASD) (Scahill et al., 2006) is a modified version of the CYBOCS developed for measuring symptom severity in children with Obsessive-Compulsive Disorder (Scahill et al. 1997). The modified version is a semi-structured clinician-rated scale designed to rate the current severity of repetitive behavior in children with ASD. Current repetitive behaviors are rated on 5 dimensions: Time Spent, Interference, Distress, Resistance, and Control. Each dimension is scored on a 5-point scale from 0 (least symptomatic) to 4 (most symptomatic), yielding a Total score from 0 to 20.

Parent-rated Vineland Adaptive Behavior Scales-Second Edition (Vineland II) (Sparrow, Balia, & Cicchetti, 2005) provides standard scores for Daily Living, Communication, and Socialization Domains. Research coordinators followed a semi-structured script to show parents how to score the Vineland II. The Vineland II asks parents to consider the child’s acquired skills and actual independent performance of the behavior (0 =behavior not performed; 1= performed sometimes or partially; 2=performed on a regular basis). Higher scores indicate better adaptive functioning. Standard scores have a population mean of 100 ± 15 for each domain.

Developmental/Cognitive functioning.

The Abbreviated Stanford-Binet Fifth Edition (SB-V) (Roid, 2003) or the Mullen’s Scale of Early Learning (MSEL) (Mullen, 1995) were used according to the developmental level of the child. The SB-V was attempted with all children. The Mullen was administered to children who were unable to complete the SB-V. Because different tests were employed, children were classified IQ < 70 and IQ ≥ 70.

Statistical Analyses

We examined the distributions of the ECI anxiety total score and the frequency of participants who met or exceeded symptom count cutoff score on each item. Unpaired t-tests were used to compare subgroups by sex and age above and below the median of 4.56 years. To evaluate scale items, we examined the number of children scoring at a clinically significant level [score of two (often) or three (very often)]. We then compared endorsement of each item between younger and older children (by median split) usingχ2 test (or Fisher’s exact test when cell sizes were less than 5) for proportions of children endorsing an item in a clinically significant range. Internal consistency of the 16-item anxiety scale was examined with Cronbach’s alpha. Pearson’s correlation was used to examine the association of the anxiety scores with other continuous variables. Because these analyses were exploratory, we set alpha levels at 0.05 and report without correction for multiple comparisons (Feise, 2002; Rothman, 1990). Logistic regression analyses were used to predict membership in the upper quartile of ECI anxiety using unique contributions of the dichotomously coded cognitive functioning (IQ < 70 was coded as 0 and IQ ≥ 70 as 1), and scores on continuous variables of social impairment (ABC Social Withdrawal subscale), and disruptive behavior (ECI ODD and ADHD scales and ABC Irritability subscale). These variables were selected for logistic regression because they showed significant difference between groups with high and low anxiety (see Table 1). SPSS version 24.0 (SPSS, Inc., Chicago) was used for all data analyses.

Table 1.

Demographic and clinical characteristics of the study sample and subjects with low and high anxiety defined by the upper quartile on the ECI anxiety total score. Significance tests compare participants with high versus low anxiety

Full sample (n= 177) Low anxiety (n=127) High Anxiety (n=50) P value
Demographic characteristics N (%)
Mean age (SD) 4.70 (1.04) 4.63 (1.01) 4.86 (1.12) .19
Sex (% males) 156 (88.1%) 114 (89.8%) 42 (84.0%) .29
IQ (% below 70) 43 (24.3%) 31 (24.4%) 12 (24.0%) .99
Race .68
 White 154 (87%) 110 (86.6%) 44 (88.0%)
 Black 14 (7.9%) 10 (7.9%) 4 (8.0%)
 Asian 8 (4.5%) 6 (4.7%) 2 (4.0%)
Ethnicity .21
 Hispanic 26 (14.7%) 16 (12.6%) 10 (20.0%)
 Non-Hispanic 151 (85.3%) 111 (87.4%) 40 (80.0%)
School Program .61
 Regular class 81 (45.8%) 57 (44.9%) 24 (48.0%)
 Special education class 68 (38.4%) 49 (38.6%) 19 (38.0%)
 Special education school 23 (13.0%) 17 (13.4%) 6 (12.0%)
 Home instruction 5 (2.8%) 4 (3.1%) 1 (2.0%)
Family composition .85
 Two parent family 157 (88.7%) 113 (89.0%) 44 (88.0%)
Maternal education .61
 College or advanced degree 108 (61.0%) 79 (62.2%) 29 (58.0%)
Clinical characteristics
Aberrant Behavior Checklist
 Irritability 23.77 (6.31) 23.10 (5.99) 25.46 (6.83) .03*
 Social Withdrawal 12.89 (8.23) 12.04 (7.79) 15.04 (8.98) .03*
 Stereotypy 6.4 (4.91) 6.19 (5.11) 6.92 (4.37) .38
 Hyperactivity 30.35 (9.32) 30.12 (9.63) 30.94 (8.55) .60
 Inappropriate Speech 5.75 (3.15) 5.59 (3.21) 6.14 (2.98) .30
Vineland Adaptive Behavior Scales
 Communication 81.39
(15.39)		 82.10 (15.00) 79.60 (16.34) .33
 Daily Living Skills 78.23
(13.65)		 78.28 (13.17) 78.10 (14.96) .94
 Socialization 72.17
(10.98)		 72.72 (10.46) 70.78 (12.21) .29
CY-BOCS-ASD Total Score 13.32 (3.53) 12.98 (3.73) 14.20 (2.80) .04*
Early Child Inventory
 ADHD subscale 32.71 (8.91) 32.26 (8.80) 33.86 (9.19) .28
 ODD subscale 11.6 (5.65) 10.51 (5.34) 14.36 (5.51) .001**
 ASD subscale 20.53 (6.72) 20.13 (6.65) 21.54 (6.86) .21
Open in a new tab
*

p< .05;

**

p< .01.

RESULTS

The ECI was missing for three participants who were not included in any analyses; IQ data were not available for 2 participants who were not included in analysis with IQ variable. Thus, the study sample consisted of 177 subjects (156 boys; 21 girls) with the mean age = 4.70, SD = 1.04. Lable 1 summarizes demographic and clinical characteristic of the sample as well as subgroups of children with low and high anxiety (i.e., the upper quartile on the ECI anxiety subscale). Anxiety scores, computed as a sum of responses to 16 items, ranged from 0 to 34 (mean=l1.3, SD=6.8). Figure 1 shows frequency distribution of the ECI total anxiety scores. The total score was normally distributed, skewness =1.11 and kurtosis = 1.49. The internal consistency of the 16-item ECI anxiety scale was Cronbach’s alpha=0.79. The same analyses were repeated for the subsamples of children with IQ ≥ 70 (n=132) and IQ < 70 (n=43), and yielded nearly identical results with alpha coefficients of 0.80 and 0.77, respectively. As shown in Lable 1, children in the high anxiety group (n=50, 28%) did not differ from those below the 75th percentile (n=127) on the Vineland domains. However, children in the high anxiety group had higher scores on the ABC Irritability and Social Withdrawal subscales, the ECI ODD subscale and on repetitive behavior as measured on the CY-BOCS-ASD. There were no differences in the total ECI anxiety score in children above and below the median (age 4.57) (t175 = 0.21, p = .834), between boys and girls (t175 = −1.24, p = .216) and between children with IQ < 70 and those with IQ ≥ 70 (t173 = 0.87, p = .585).

Figure 1.

Figure 1.

Open in a new tab

Frequency distribution of the ECI-anxiety symptoms severity scores

Frequency and Distribution of ECI Anxiety Items

Table 2 presents per-item means and standard deviations as well as the number and percentage of participants with scores of 2 (often) and 3 (very often) on each item. The percentage of children rated as often or very often ranged from 3.9% to 58.9% (see Table 2). The total number of items scored in the clinically significant range varied from 0 to 9 (mean=2.59, SD=2.09). Figure 2, shows the frequency distribution of the number of items scored in the clinically significant range. The median number of symptoms rated in the clinically significant range was two and 66.7 % of study participants were rated as having two or more items and 44.7% as having three or more items in the clinically significant range. There were no differences on the anxiety symptom count in subsamples of children with IQ ≥ 70 and IQ < 70, between boys and girls or children above and below median age. The Table 2 also shows the item to total score correlations on the ECI Anxiety subscale (i.e., the correlations of each item with the sum of remaining items) which ranged from 0.16 to 0.59. Three items (“worries that other children can do things better,” “has trouble falling asleep,” and “is excessively shy with peers”) showed item-total correlations below 0.3.

Table 2.

Descriptive statistics and item-total correlations for the ECI anxiety items in young children with ASD

Abbreviated item content Mean SD n (%) above cutoff Item-total corr
Worries that other children can do things better 0.23 0.59 13 (7.2%) 0.16
Worries more than other children 0.71 1.02 36 (20.0%) 0.47
Has trouble falling asleep 1.35 1.06 63 (35.0%) 0.20
Is overly fearful or tries to avoid specific objects or situations 1.11 0.98 51 (28.3%) 0.52
Cannot get distressing thoughts out of his/her mind 0.62 0.92 33 (18.3%) 0.59
Complains about physical aches and pains with no cause 0.44 0.72 16 (8.9%) 0.42
Anxious or withdraws in uncomfortable social situations 1.78 1.00 106 (58.9%) 0.41
Is excessively shy with peers 1.09 1.04 51 (28.3%) 0.21
Upset when child expects to be separated from home or parents 1.16 0.97 49 (27.2%) 0.45
Worries that parents will be hurt or leave and not come back 0.49 0.82 23 (12.8%) 0.47
Worries that some disaster will separate from parents 0.19 0.55 7 (3.9%) 0.46
Tries to avoid going to school to stay with parent 0.47 0.74 14 (7.8%) 0.43
Worries about being left at home alone or with a sitter 0.34 0.65 11 (6.1%) 0.42
Afraid to go to sleep unless near parent 0.97 1.21 52 (28.9%) 0.34
Has nightmares about being separated from parent 0.20 0.64 8 (4.4%) 0.40
Complains about feeling sick if expects separation from parents 0.19 0.54 8 (4.4%) 0.48
Open in a new tab

Figure 2.

Figure 2.

Open in a new tab

Distribution of symptom counts (item scores of 2 or 3)

The five items most commonly answered as 2 or 3 were: 1) “anxious or withdraws in uncomfortable social situations,” 2) “has trouble falling asleep,” 3) “afraid to go to sleep unless near parent,” 4) “is overly fearful or tries to avoid specific objects or situations,” and 5) “is excessively shy with peers.” Five least endorsed items were: 1) “worries that some disaster will separate from parents,” 2) “complains about feeling sick if expects separation from parents,” 3) “has nightmares about being separated from parent,” 4) “worries about being left at home alone or with a sitter,” and 5) worries that other children can do things better. Although there were no age differences in the total score and symptom count by age, only 2 of 90 children (2.3%) under the median age (< 4.57 years) were rated in the clinically significant range on the item: “worries that other children can do things better that s/he,” compared to 11 of 87 children (12.6%) in the above the median age (> 4.5 years) (Fisher’s exact testp < 0.01). By contrast, a larger proportion of younger children (33 of 90; 36.7%) were rated in the clinically significant range on the item “is excessively shy with peers,” compared to 18 of 87 children (20.7%) in the older age group (Chi-Square=5.51, p<0.05).

Pearson correlations were used to examine the associations of the ECI total anxiety scores with the ABC subscales, Vineland domains, ECI ASD, ODD and ADHD subscales, CYBOCS-ASD total score (see Table 3). These analyses were conducted for the total sample and then separately for the two IQ groups. In the full sample, correlations ranged from .15 to .32. In children with IQ below 70, anxiety was significantly and negatively associated with Vineland Socialization (r=−36, p=0.19) (note: on Vineland higher scores = higher adaptive functioning). Four variables: ABC-Irritability, ABC Social Withdrawal, CYBOCS-ASD and ECI ODD that were significantly correlated with the total score on the ECI Anxiety scale also showed a difference between high and low anxiety groups (See Table 1).

Table 3.

Correlations between anxiety and measures of adaptive functioning, ASD severity, repetitive behavior, and behavioral problems in children with ASD by two levels of cognitive functioning.

Total
sample		 IQ≥70 IQ<70
(n=177) (n=132)a (n=43)a
Variable/Measure r r r
Aberrant Behavior Checklist
 Irritability 0.17* 0.16 0.26
 Social Withdrawal 0.24** 0.23** 0.38*
 Stereotypy 0.15* 0.14 0.22
 Hyperactivity 0.07 0.10 −0.01
 Inappropriate Speech 0.18* 0.14 0.27
Vineland Adaptive Behavior Scales
 Communication −0.07 −0.07 −0.29
 Daily Living Skills −0.30 0.01 −0.30
 Socialization −0.14 −0.12 −0.36*
CY-BOCS-ASD
 CY-BOCS-ASD Total Score 0.17* 0.17 0.21
Early Child Inventory
 ADHD subscale 0.20** 0.22* 0.16
 ODD subscale 0.32** 0.29** 0.43**
 ASD subscale 0.20** 0.22* 0.27
Open in a new tab
a

= IQ scores were unavailable for 2 subjects;

*

p < .05;

**

p < .01.

Prediction of membership in the upper quartile of ECI anxiety

Logistic regression analysis explored associations of children in the upper quartile of the total ECI anxiety subscale (total score ≥ 14). As noted above, children with high anxiety had significantly higher scores on the ABC Irritability and Social Withdrawal subscales, CYBOCS-ASD and on ECI ODD scale. Thus, we limited the number of predictors to these four variables (see Table 3). These variables were entered in logistic regression models to examine prediction of membership in the upper quartile on the ECI anxiety total score (see Table 4). Although sex and IQ were not significantly associated with ECI Anxiety score, we included these variables in the first step of regression model to control for their possible effects on the association with higher levels of anxiety (i.e., children in the upper quartile). Predictor variables (ABC Irritability and Social Withdrawal subscales, CYBOCS-ASD and on ECI ODD scale) were entered using empirical forward stepwise procedure with likelihood ratio estimation. We use the stepwise procedure instead of the simultaneous procedure to identify the best subset of significant predictors in the model by eliminating non-significant predictors. The Chi-square values are reported for each step and regression coefficients are reported for the final step. Higher levels of anxiety were associated with severity of oppositional defiant behavior as measured on the ECI ODD subscale (B = 0.128, Wald test = 14.841, p <0.001) and ABC Social Withdrawal subscale (B = 0.046, Wald test = 3.874, p = 0.050). To explore whether social withdrawal and ODD symptoms have differential association with anxiety for children with higher versus lower cognitive functioning, the interaction terms of IQ and the two centered variables for the ECI ODD and ABC Social Withdrawal were included in the third step. To avoid multicollinearity, continuous variables were centered, i.e., converted to deviation scores so that each variable has a mean of zero (Aiken & West, 1991). These interaction terms were not significant. Thus, the most parsimonious model included ABC Social Withdrawal subscale and the ECI ODD scale adjusted for sex and IQ.

Table 4.

Logistic regression analysis of membership in the upper quartile of anxiety as a function of severity of autism symptoms and co-occurring irritability and noncompliance

Chi- p value B SE B Wald p value
Variables Square
Step 1 0.52 .768
IQ (above or below 70) 0.007 0.392 0.001 0.985
Sex 0.371 0.502 0.544 0.461
Step 2a 21.88 <.001
ABC Social Withdrawal 0.046 0.023 3.874 0.050*
ECI ODD scale 0.128 0.033 14.841 0.001**
Open in a new tab
a

= interaction terms of IQ and ABC social withdrawal and ECI ODD scale scores were not significant and not included in the final model. ABC=Aberrant Behavior Checklist; ECI = Early Childhood Inventory

*

p = .05;

**

p < .01

DISCUSSION

This study examined the distribution and correlates of anxiety symptoms in a well-characterized sample of young children with ASD and disruptive behavior. Participants were not selected for anxiety. This study extended our earlier work on Child and Adolescent Symptom Inventory (CASI) for measuring anxiety in school-age children with ASD (Hallett et al., 2013) by focusing on a younger age group and by examining associations of anxiety with co-occurring psychopathology, as well as with core symptoms of ASD, IQ, and adaptive functioning. Similar to our prior work, the severity of anxiety symptoms ranged from mild to severe. Remarkably, the average per item mean scores were the same, 0.71 ± 0.47 in the Hallett et al. sample and 0.71 ± 0.42 in the current sample of young children.

Another striking similarity to the Hallett et al. (2013) report, the five least endorsed items were: “worries that some disaster will separate from parents,” “complains about feeling sick if expects separation from parents,” “has nightmares about being separated from parent,” “worries about being left at home alone or with a sitter,” and “worries that other children can do things better.” These five items are highly reliant on language. In the current sample of young children with ASD, only 24% had an IQ < 70. Nonetheless, given the age and the high likelihood of some language delays in children with ASD, it is not surprising that items that rely on language were rarely endorsed.

By contrast, the five most common parent-endorsed items included “anxious or withdraws in uncomfortable social situations,” “has trouble falling asleep,” “afraid to go to sleep unless near parent,” “is overly fearful or tries to avoid specific objects or situations,” and “is excessively shy with peers.” These items, which are less reliant on language, were endorsed in 28 to 60% of study participants. As shown in Table 2, one of these two items (“has trouble falling asleep”) had a low item-total correlation with the total ECI anxiety score (0.20), suggesting that sleep problems are common in young children with ASD and may be related to problems other than anxiety in this age group.

The low rate of endorsement of language-dependent items suggests that measures used to evaluate anxiety symptoms in the general pediatric population may not be suitable for youth with ASD (Kerns et al., 2014; Lecavalier et al., 2014; Scahill et al., in press; van Steensel & Heeman, 2017). In addition, the assessment of anxiety in children with ASD can be complicated by parental perceptions that the manifestations of anxiety may differ for children with ASD (Bearss et al., 2016; Kerns et al., 2014). For example, in a large multi-site sample of children with ASD (N=870), Magiati and colleagues (2017) reported that confirmatory factor analysis failed to support the original factors of the parent-rated Spence Children’s Anxiety Scale-Parent Version (SCAS-P). Other authors also caution against using the SCAS-P to assess anxiety in children with ASD (Glod et al., 2017).

In response to the lack of assessment tools for measuring anxiety in children with ASD, Scahill and colleagues conducted a mixed qualitative and quantitative study to develop the 25-item Parent-rated Anxiety Scale for ASD (PRAS-ASD). This single factor scale showed excellent internal consistency and test-retest reliability, as well as solid divergent and convergent validity (Scahill et al., in press). As noted above, Rodgers and colleagues developed the Anxiety Scale for Children with Autism Spectrum Disorder, also intended to index anxiety symptoms in children with ASD (Keen et al., 2019; Rodgers et al., 2016).

Although significant, the relatively small correlations of the ECI Anxiety scale with parent ratings of behavioral problems and ratings of core ASD symptom severity suggest that anxiety is distinct from these domains in young children with ASD. There were no differences in the severity of anxiety symptoms in children with IQ below and above 70. In our prior study in school-age children with ASD, higher-functioning youth had significantly higher levels of anxiety (Hallett et al., 2013). The observation of no difference by IQ in the current sample of young children may be due to the lower endorsement of language-dependent items in the entire sample. Alternatively, it may be that the low proportion of children with IQ below 70 reduced the statistical power to detect a group difference.

ABC Irritability, ECI ODD and ECI ADHD symptoms were positively correlated with anxiety in young children with ASD. Here again, the magnitude of these associations was small. However, the membership in the upper quartile of anxiety severity was predicted by the ECI ODD scale and the ABC Social Withdrawal subscale. Several explanations may account for the associations of anxiety with behavioral problems and social impairment. Deficits in emotion regulation may be a key mechanism linking multiple psychiatric problems, including both internalizing (e.g., anxiety) and externalizing (e.g., disruptive behavior) problems (Mazefsky et al., 2013). Alternatively, anxiety may contribute to the manifestation and development of behavioral problems over time. Children with high levels of anxiety may engage in oppositional behavior to avoid or escape from anxiety-provoking situations. For example, a child who is anxious about joining a group activity may refuse to comply with encouragement to join the activity. Similarly, a young child with separation anxiety may appear obstinate when refusing entry to preschool. The ABC Social Withdrawal subscale is a measure of social disability. The predictive value of this subscale for the highest quartile of anxiety suggests that greater social disability increases the likelihood of higher anxiety. As noted by Duvekot and colleagues (2018), however, the interaction may be more complicated. Using cross-lagged models to examine the interaction between core ASD symptoms and anxiety over a two-year period, these investigators reported that anxiety predicted future social impairment.

There are several limitations of this study. First, this is a sample of convenience. Although participants were not selected for anxiety symptoms, the ascertainment of young children with moderate or greater disruptive behavior may not generalize to the wider ASD population. Second, the presence of anxiety was determined based on parent-report only. The low correlations of the ECI anxiety scale with measures of other constructs such as irritability, noncompliance and social withdrawal, suggests that the scale is measuring a unique construct. However, we did not have another measure of anxiety (either from parents or additional informants) and consequently we do not have evidence of convergence. Another potential limitation is that eight of 16 ECI Anxiety scale items inquire about separation anxiety. Separation anxiety is the most common anxiety disorder in this age group in the general pediatric population. In addition, separation anxiety may interfere with the mastery of key developmental tasks and pose greater risk for subsequent problems for young children with ASD. However, it may be argued that other anxiety domains are underrepresented and thus, the observed associations may be driven by separation anxiety.

Lastly, the cross-sectional nature of the study precludes inference on causation. Future studies could employ longitudinal designs to examine the predictors of anxiety in children with ASD. Future studies should also employ novel analyses, such as network analyses (e.g., Montazeri, de Bildt, Dekker, & Anderson, 2018), to explore the associations between ASD and anxiety. Concurrent assessment of anxiety and emotion regulation processes, using measures validated for ASD samples (Mazefsky et al., 2018), may also be informative.

Clinical implications. This study shows that anxiety is present in young children with ASD. Our results suggest that anxiety is a separate construct in this population that appears to be associated with oppositional behaviors and social disability. The high frequency of anxiety symptoms in young children with ASD and the possible contribution of anxiety to overall disability indicates that anxiety should be incorporated into clinical evaluation in this population. For example, the association of anxiety and oppositional behavior suggests that assessment of anxiety may provide insight into presenting behavioral problems and inform treatment planning. Parents of children with ASD might also benefit from careful discussion about the differential diagnosis and potential interaction of anxiety and ASD.

The results of this study also suggest that treatment of anxiety in young children with ASD may reduce overall burden for the child and family as observed in the general pediatric population (Pincus, Santucci, Ehrenreich, & Eyberg, 2008). For example, young children with ASD and anxiety may benefit from family-based CBT which employs developmentally-guided modifications (Cook, Donovan, & Garnett, 2019; Nadeau, Arnold, Selles, Storch, & Lewin, 2015). Behavioral interventions targeting social communication in ASD may reduce symptoms of anxiety in young children with ASD (Lei, Sukhodolsky, Abdullahi, Braconnier, & Ventola, 2017). Interventions focused on targeting specific subtypes of anxiety, such as separation anxiety, in young children (e.g., Pincus et al., 2008), may also be useful when modified for children with ASD. However, to best inform the development of such interventions, continued research on anxiety and ASD in preschool children is needed.

References

  1. Aiken LS, & West SG (1991). Multiple regression: Testing and interpreting interactions’. Thousand Oaks, CA, US: Sage Publications, Inc. [Google Scholar]
  2. Aman MG, Singh NN, Stewart AW, & Field CJ (1985). The Aberrant Behavior Checklist: A behavior rating scale for the assessment of treatment effects. American Journal of Mental Deficiency, 89, 485–491. [PubMed] [Google Scholar]
  3. Association AP (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Association. [Google Scholar]
  4. Battaglia M, Touchette E, Garon-Carrier G, Dionne G, Cote SM, Vitaro F,… Boivin M (2016). Distinct trajectories of separation anxiety in the preschool years: persistence at school entry and early-life associated factors. J Child Psychol Psychiatry, 57, 39–46. doi: 10.111l/jcpp.12424 [DOI] [PubMed] [Google Scholar]
  5. Bearss K, Johnson C, Smith T, Lecavalier L, Swiezy N, Aman M,… Scahill L (2015). Effect of parent training vs parent education on behavioral problems in children with autism spectrum disorder: a randomized clinical trial. JAMA, 313, 1524–1533. doi: 10.1001/jama.2015.315 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bearss K, Taylor CA, Aman MG, Whittemore R, Lecavalier L, Miller J,… Scahill L (2016). Using qualitative methods to guide scale development for anxiety in youth with autism spectrum disorder. Autism, 20, 663–672. doi: 10.1177/1362361315601012 [DOI] [PubMed] [Google Scholar]
  7. Cook JM, Donovan CL, & Garnett MS (2019). Parent-mediated cognitive behavioural therapy for young children with high-functioning autism spectrum disorder and anxiety: a randomized control trial. Early Child Development and Care, 189, 119–134. doi: 10.1080/03004430.2017.1303684 [DOI] [Google Scholar]
  8. Costello EJ, Egger HL, & Angold A (2005). The developmental epidemiology of anxiety disorders: Phenomenology, prevalence, and comorbidity. Child and Adolescent Psychiatric Clinics of North America, 14, 631–648. [DOI] [PubMed] [Google Scholar]
  9. Duvekot J, van der Ende J, Verhulst FC, & Greaves-Lord K (2018). Examining bidirectional effects between the autism spectrum disorder (ASD) core symptom domains and anxiety in children with ASD. J Child Psychol Psychiatry, 59, 277–284. doi: 10.1111/jcpp.12829 [DOI] [PubMed] [Google Scholar]
  10. Egger T, & Angold A (2004). The preschool age psychiatric assessment (PAPA): A structured parent interview for diagnosing psychiatric disorders in preschool children In DelCarmen-Wiggins R & Carter AS (Eds.), Handbook of Infant, Toddler, and Preschool Mental Health Assessment. New York: Oxford Unviersity Press. [Google Scholar]
  11. Feise RJ (2002). Do multiple outcome measures require p-value adjustment? BMC Med Res Methodol, 2, 8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Gadow KD, Devincent CJ, Pomeroy J, & Azizian A (2004). Psychiatric symptoms in preschool children with PDD and clinic and comparison samples. Journal of Autism and Developmental Disorders, 34, 379–393. [DOI] [PubMed] [Google Scholar]
  13. Hallett V, Lecavalier L, Sukhodolsky DG, Cipriano N, Aman MG, McCracken JT,… Scahill L (2013). Exploring the manifestations of anxiety in children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 43, 2341–2352. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Johnson CR, DeMand A, & Shui A (2015). Relationships Between Anxiety and Sleep and Feeding in Young Children with ASD. Journal of Developmental and Physical Disabilities. doi: 10.1007/s10882-015-9419-3 [DOI] [Google Scholar]
  15. Kaat AJ, Lecavalier L, & Aman MG (2013). Validity of the Aberrant Behavior Checklist in Children with Autism Spectrum Disorder. Journal of Autism and Developmental Disorders, 1–14. [DOI] [PubMed] [Google Scholar]
  16. Keen D, Adams D, Simpson K, den Houting J, & Roberts J (2019). Anxiety-related symptomatology in young children on the autism spectrum. Autism, 23, 350–358. doi: 10.1177/1362361317734692 [DOI] [PubMed] [Google Scholar]
  17. Kerns CM, Kendall PC, Berry L, Souders MC, Franklin ME, Schultz RT,…Herrington J (2014). Traditional and Atypical Presentations of Anxiety in Youth with Autism Spectrum Disorder. Journal of Autism and Developmental Disorders, doi: 10.1007/sl0803-014-2141-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Kessler RC, Avenevoli S, McLaughlin KA, Green JG, Lakoma MD, Petukhova M,… Merikangas KR (2012). Lifetime co-morbidity of DSM-IV disorders in the US National Comorbidity Survey Replication Adolescent Supplement (NCS-A). Psychological Medicine, 42, 1997–2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Lavigne JV, LeBailly SA, Hopkins J, Gouze KR, & Binns HJ (2009). The prevalence of ADHD, ODD, depression, and anxiety in a community sample of 4-year-olds. Journal of Clinical Child and Adolescent Psychology, 38, 315–328. doi: 10.1080/15374410902851382 [DOI] [PubMed] [Google Scholar]
  20. Lecavalier L, Gadow KD, Devincent CJ, Houts CR, & Edwards MC (2011). Validity of DSM-IV syndromes in preschoolers with autism spectrum disorders. Autism, 15, 527–543. doi: 10.1177/1362361310391115 [DOI] [PubMed] [Google Scholar]
  21. Lecavalier L, Wood JJ, Halladay AK, Jones NE, Aman MG, Cook EH,… Scahill L (2014). Measuring anxiety as a treatment endpoint in youth with autism spectrum disorder. Journal of Autism and Developmental Disorders, 44, 1128–1143. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Lei J, Sukhodolsky DG, Abdullahi SM, Braconnier ML, & Ventola P (2017). Brief report: Reduced anxiety following Pivotal Response Treatment in young children with Autism Spectrum Disorder. Res Autism Spectr Disord, 43–44,1–7. doi: 10.1016/j.rasd.2017.09.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Lonigan CJ, Vasey MW, Phillips BM, & Hazen RA (2004). Temperament, anxiety, and the processing of threat-relevant stimuli. J Clin Child Adolesc Psychol, 33, 8–20. doi: 10.1207/S15374424JCCP3301_2 [DOI] [PubMed] [Google Scholar]
  24. Lord C, Risi S, Lambrecht L, Cook EH Jr, Leventhal BL, Dilavore PC,… Rutter M (2000). The Autism Diagnostic Observation Schedule-Generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30, 205–223. [PubMed] [Google Scholar]
  25. Macari SL, Koller J, Campbell DJ, & Chawarska K (2017). Temperamental markers in toddlers with autism spectrum disorder. J Child Psychol Psychiatry, 58, 819–828. doi: 10.1111/jcpp.12710 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Magiati I, Lerh JW, Hollocks MJ, Uljarevic M, Rodgers J, McConachie H,… Simonoff E (2017). The measurement properties of the spence children’s anxiety scale-parent version in a large international pooled sample of young people with autism spectrum disorder. Autism Res, 10, 1629–1652. doi: 10.1002/aur.l809 [DOI] [PubMed] [Google Scholar]
  27. Mazefsky CA, Day TN, Siegel M, White SW, Yu L, Pilkonis PA, & Inpatient ADD (2018). Development of the Emotion Dysregulation Inventory: A PROMIS (R) ing Method for Creating Sensitive and Unbiased Questionnaires for Autism Spectrum Disorder. Journal of Autism and Developmental Disorders, 48, 3736–3746. doi: 10.1007/sl0803-016-2907-l [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Mazefsky CA, Herrington J, Siegel M, Scarpa A, Maddox BB, Scahill L, & White SW (2013). The role of emotion regulation in autism spectrum disorder. J Am Acad Child Adolesc Psychiatry, 52, 679–688. doi: 10.1016/j.jaac.2013.05.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Montazeri F, de Bildt A, Dekker V, & Anderson GM (2018). Network Analysis of Anxiety in the Autism Realm. J Autism Dev Disord, doi: 10.1007/sl0803-018-3474-4 [DOI] [PubMed] [Google Scholar]
  30. Moriuchi JM, Klin A, & Jones W (2017). Mechanisms of diminished attention to eyes in Autism. American Journal of Psychiatry, 174, 26–35. doi: 10.1176/appi.ajp.2016.15091222 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Mullen E (1995). Mullen Scales of Early Learning. Circle Pines, MN: American Guidance Service. [Google Scholar]
  32. Nadeau JM, Arnold EB, Selles RR, Storch EA, & Lewin AB (2015). A Cognitive-Behavioral Approach for Anxiety in a Preschool-Aged Child With Autism Spectrum Disorder. Clinical Case Studies, 14, 47–60. doi: 10.1177/1534650114536028 [DOI] [Google Scholar]
  33. Paulus FW, Backes A, Sander CS, Weber M, & von Gontard A (2015). Anxiety disorders and behavioral inhibition in preschool children: a population-based study. Child Psychiatry Hum Dev, 46,150–157. doi: 10.1007/sl0578-014-0460-8 [DOI] [PubMed] [Google Scholar]
  34. Pincus DB, Santucci LC, Ehrenreich JT, & Eyberg SM (2008). The implementation of modified Parent-Child Interaction Therapy for youth with Separation Anxiety Disorder. Cognitive and Behavioral Practice, 15, 118–125. doi: DOI 10.1016/j.cbpra.2007.08.002 [DOI] [Google Scholar]
  35. Rodgers J, Wigham S, McConachie H, Freeston M, Honey E, & Parr JR (2016). Development of the anxiety scale for children with autism spectrum disorder (ASC-ASD). Autism Res, 9, 1205–1215. doi: 10.1002/aur.l603 [DOI] [PubMed] [Google Scholar]
  36. Roid GH (2003). Stanford-Binet Intelligence Scale manual (5th ed). Itasca, IL: Riverside. [Google Scholar]
  37. Rothman KJ (1990). No adjustments are needed for multiple comparisons. Epidemiology, 1,43–46. [PubMed] [Google Scholar]
  38. Rutter M, Le Couteur A, & Lord C (2003). The Autism Diagnostic Interview-revised (ADI-R). Los Angeles, CA: Western Psychological Services. [Google Scholar]
  39. Salazar F, Baird G, Chandler S, Tseng E, O’Sullivan T, Howlin P,… Simonoff E (2015). Co-occurring Psychiatric Disorders in Preschool and Elementary School-Aged Children with Autism Spectrum Disorder. J Autism Dev Disord, 45, 2283–2294. doi: 10.1007/sl0803-015-2361-5 [DOI] [PubMed] [Google Scholar]
  40. Scahill L, Bearss K, Lecavalier L, Smith T, Swiezy N, Aman MG,… Johnson C (2016). Effect of Parent Training on Adaptive Behavior in Children With Autism Spectrum Disorder and Disruptive Behavior: Results of a Randomized Trial. Journal of the American Academy of Child and Adolescent Psychiatry, 55, 602–609.e603. doi: 10.1016/j.jaac.2016.05.001 [DOI] [PubMed] [Google Scholar]
  41. Scahill L, Lecavalier L, Schultz RT, Evans AN, Maddox B, Pritchett J,… Edwards MC (in press). Development of the Parent-rated Anxiety Scale for Youth with Autism Spectrum Disorder. JAmer Acad Child and Adolescent Psychiatry. [DOI] [PubMed] [Google Scholar]
  42. Scahill LM, McDougle CJ, Williams SK, Dimitropoulos AP, Aman MG, McCracken JT,… Vitiello BM (2006). Children’s Yale-Brown Obsessive Compulsive Scale Modified for Pervasive Developmental Disorders. Journal of the American Academy of Child & Adolescent Psychiatry September, 45, 1114–1123. [DOI] [PubMed] [Google Scholar]
  43. Sparrow SS, Balia DA, & Cicchetti DV (2005). Vineland Adaptive Behavior Scales. (2 ed.). Circle Pines, MN: American Guidance Service. [Google Scholar]
  44. Sprafkin J, Gadow KD, Salisbury H, Schneider J, & Loney J (2002). Further Evidence of Reliability and Validity of the Child Symptom Inventory-4: Parent Checklist in Clinically Referred Boys. Journal of Clinical Child and Adolescent Psychology, 31, 513–524. [DOI] [PubMed] [Google Scholar]
  45. van Steensel FJ, Bogels SM, & Perrin S (2011). Anxiety Disorders in Children and Adolescents with Autistic Spectrum Disorders: A Meta-Analysis. Clin Child Fam Psychol Rev. doi: 10.1007/sl0567-011-0097-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. van Steensel FJA, & Heeman EJ (2017). Anxiety Levels in Children with Autism Spectrum Disorder: A Meta-Analysis. Journal of Child and Family Studies, 26, 1753–1767. doi: 10.1007/s10826-017-0687-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Whalen DJ, Sylvester CM, & Luby JL (2017). Depression and Anxiety in Preschoolers: A Review of the Past 7 Years. Child Adolesc Psychiatr Clin N Am, 26, 503–522. doi: 10.1016/j.chc.2017.02.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. White SW, Mazefsky CA, Dichter GS, Chiu PH, Richey JA, & Ollendick TH (2014). Social-cognitive, physiological, and neural mechanisms underlying emotion regulation impairments: understanding anxiety in autism spectrum disorder. Int J Dev Neurosci, 39, 22–36. doi: 10.1016/j.ijdevneu.2014.05.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. White SW, Oswald D, Ollendick T, & Scahill L (2009). Anxiety in children and adolescents with autism spectrum disorders. Clinical Psychology Review, 29, 216–229. doi: 10.1016/j.cpr.2009.01.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Wichstrom L, Berg-Nielsen TS, Angold A, Egger HL, Solheim E, & Sveen TH (2012). Prevalence of psychiatric disorders in preschoolers. Journal of Child Psychology and Psychiatry and Allied Disciplines. [DOI] [PubMed] [Google Scholar]

RESOURCES