Using the BASC-2 parent rating scales to screen for autism spectrum disorder in toddlers and preschool-aged children

Abstract

Early identification of toddlers and preschool-aged children with autism spectrum disorder (ASD) is important for ensuring that these youth receive targeted early intervention services. Identifying young children with ASD is complicated by overlap among symptoms of ASD and other developmental delays. Additionally, youth with ASD have a higher risk of experiencing co-occurring challenging behaviors that are beyond the diagnostic criteria for ASD (e.g., attention difficulties, anxiety). Given this, broadband behavioral assessments that measure symptoms of ASD as well as other behavioral and emotional challenges offer a cost-effective method for screening young children. The present study evaluated the utility of one such assessment, the Behavior Assessment System for Children, Second Edition, Parent Rating Scale-Preschool (BASC-2 PRS-P), for identifying young children with ASD from those with other diagnoses (including other developmental delays) and those without diagnoses. The sample included 224 toddlers and preschoolers (age range: 24-63 months, males n= 153 [68% total sample]) who screened positive on an ASD-specific screening tool. Results demonstrated that the Developmental Social Disorders (DSD) scale on the BASC-2 PRS-P had adequate sensitivity and specificity values when distinguishing youth with ASD from those without any diagnoses, but not when differentiating between youth with ASD and those with other diagnoses. Similar to other multidimensional behavior rating scales, the BASC-2 PRS-P may be most useful for identifying young children who require comprehensive diagnostic evaluations.

Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by deficits in social communication and stereotyped or repetitive interests and behaviors (American Psychiatric Association, 2013). ASD is one of the most common neurodevelopmental disorders, affecting approximately 1 in 68 children (Christensen et al., 2016). The median age of diagnosis for children with ASD is 50 months (Christensen et al., 2016). However, ASD can be reliably diagnosed within the first 36 months of life (Boyd, Odom, Humphreys, & Sam, 2010; Kleinman et al., 2008; Zwaigenbaum, Bryson, & Garon, 2013). Great attention has been given to early identification of young children with ASD because it is clear that early, intensive intervention is key for symptom improvement (Thompson, 2013).

One challenge to early identification is differentiating ASD from other developmental disabilities. Some symptoms of ASD, including delayed speech development, difficulties with peer interactions, and co-occurring challenging behaviors, are common to other conditions such as language disorder and attention deficit/hyperactivity disorder (ADHD; Mandell, Ittenbach, Levy, & Pinto-Martin, 2007) and global developmental delay (Ventola et al., 2007). Further, the severity of core ASD symptoms and presence of co-occurring conditions, such as intellectual disability, vary greatly among affected individuals (Kim, Macari, Koller, & Chawarska, 2016) and may complicate the identification of this disorder in some children (e.g., Lord, 1995). Thus, it is critical that caregivers and service providers are aware of the early warning signs of ASD and that appropriate screening measures are utilized (Johnson & Myers, 2007).

Current best practices for ASD detection include three complementary aspects: ongoing developmental surveillance, broad developmental screening for all children at 9-, 18-, and 24/30- month well-child check-ups, and ASD-specific screening for all children at the 18- and 24-month check-ups (Gupta et al., 2007; Johnson & Myers, 2007). Although a variety of ASD-specific screening tools have been developed, few have been validated in low-risk samples for children younger than 3 years (for a review, see Johnson & Myers, 2007). The most commonly used ASD screeners are the Modified Checklist for Autism in Toddlers (M-CHAT; Robins, Fein, & Barton, 1999) and its revision, the Modified Checklist for Autism in Toddlers, Revised, with Follow-Up (M-CHAT-R/F; Robins, Fein, & Barton, 2009).

Multidimensional behavior rating scales represent an alternative approach to screening for ASD in young children (Sikora, Hall, Hartley, Gerrard-Morris, & Cagle, 2008; Volker et al., 2010). These scales are designed to measure multiple behavioral and emotional problems, including symptoms of ASD. Using a single tool to screen for ASD-specific symptoms and broad emotional and behavioral issues may be more efficient and reduce clinician and family time as well as financial costs, compared to relying on multiple assessments (Sikora et al., 2008). Considering the high rates of co-occurring conditions in children with ASD (Volkmar & Klin, 2005), multidimensional behavior rating scales may be particularly appropriate screening tools for this population. Two commonly used behavior rating scales are the Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2000) and the Behavior Assessment System for Children, Second Edition (BASC-2; Reynolds & Kamphaus, 2004).

The CBCL is a rating scale designed to measure emotional and behavioral problems in youth (Achenbach & Rescorla, 2000). The CBCL/1.5-5, designed for toddlers and preschool children (18 months – 5 years), yields T-scores for seven syndrome scales and five DSM-oriented scales, including a Withdrawn syndrome subscale and a Pervasive Developmental Problems (PDP) DSM-oriented subscale, which measure behaviors associated with ASD. The CBCL/1.5-5 Withdrawn and PDP scales distinguish between youth with ASD and typically developing (TD) children (Muratori et al., 2011; Narzisi et al., 2013; Predescu, Șipos, Dobrean, & Micluția, 2013), with sensitivity and specificity values for a cut-off score of 65 on the PDP scale as high as .98 and .91, respectively (Narzisi et al., 2013). The utility of both scales is less robust when comparing children with ASD to children with other developmental delays and diagnoses (Havdahl, von Tetzchner, Huerta, Lord, & Bishop, 2016; Muratori et al., 2011; Myers, Gross, & McReynolds, 2014; Predescu et al., 2013; Sikora et al., 2008), with specificity values for the PDP scale cut-off score of 65 dropping to .29 (Myers et al., 2014).

The BASC-2 is a multidimensional behavior rating system that assesses clinical and adaptive features of behavior and emotional functioning via Parent Rating Scales (PRS) and Teacher Rating Scales (TRS; Reynolds & Kamphaus, 2004). On the BASC-2 Preschool Form, T-scores are provided for eight clinical scales (Aggression, Anxiety, Attention Problems, Atypicality, Depression, Hyperactivity, Somatization, Withdrawal) and four adaptive scales (Activities of Daily Living, Adaptability, Functional Communication, Social Skills), as well as four composite scales: Externalizing Problems, Internalizing Problems, Behavioral Symptoms Index, and Adaptive Skills. Additionally, seven content scales are available on the PRS/TRS. The content scales contain items belonging to the primary clinical and adaptive scales and additional items not on these scales. These scales combine items from multiple constructs to detect patterns of behavior. One content scale, the Developmental Social Disorders (DSD) scale, measures symptoms associated with ASD, including difficulties with social skills and communication (Reynolds & Kamphaus, 2004).

Multiple studies have examined the utility of the BASC-2 PRS and TRS Child (ages 6–11 years) and Adolescent (ages 12–21 years) versions for identifying children and adolescents with ASD (Goldin, Matson, Konst, & Adams, 2014; Hass, Brown, Brady, & Johnson, 2012; Mahan & Matson, 2011; Volker et al., 2010). On the PRS, school-aged youth with ASD obtained scores indicating significantly more problematic behaviors on the Externalizing Problems Composite, Behavioral Symptoms Index, and Adaptive Skills Composite compared to TD groups (Goldin et al., 2014; Mahan & Matson, 2011; Volker et al., 2010). Volker et al. (2010) determined that a cut-score of 60 (T-score) on the DSD scale resulted in strong sensitivity (.98) and specificity (.95) values for identifying school-aged youth with high-functioning ASD. Hass et al. (2012) also found that on the BASC-2 TRS, children and adolescents receiving special education services under the eligibility category of Autism obtained significantly higher scores on the DSD scale compared to youth without this classification.

Though results for the BASC-2 PRS and TRS in school-aged children suggest that the BASC-2 may be a useful screening measure of ASD, little research is available on the utility of the BASC-2 PRS-Preschool Form (BASC-2 PRS-P). Given the high cost of clinical evaluations, both in the hospital setting and among private practitioners, many children may not be evaluated for possible developmental delays until they reach preschool age or older, and are assessed by their local school system. The BASC-2 is a widely used instrument among school-based practitioners, both at the preschool and school-aged levels. Myers et al. (2014) evaluated the usefulness of the BASC-2 PRS-P as a possible screening tool for ASD. The authors found significant differences between the ASD group and the non-ASD group on the Social Skills and Functional Communication scales, but did not examine the DSD scale. The current study evaluated the clinical utility of the BASC-2 PRS-P, including in the DSD scale, for identifying youth with ASD and may offer important implications for determining if and when further evaluation of ASD is needed both in clinical and school-based settings.

The current study investigated the utility of the BASC-2 PRS-P for identifying young children with ASD from children without an ASD diagnosis (Non-ASD group). The Non-ASD group included children with other developmental delays (e.g., developmental language delay, global developmental delay), children with pre-existing diagnoses (e.g., ADHD, epilepsy), children with no diagnoses who demonstrated specific areas of weakness in their development, and TD children. Importantly, we drew all participants from multisite large-scale early screening studies for developmental delays in predominantly low-risk samples (meaning that children were screened universally, and caregivers of children were not necessarily concerned about their children’s development when entering the study).

We hypothesized that the DSD scale would exhibit high agreement with other ASD measures, including the M-CHAT-R (Robins et al., 1999, 2009), the Childhood Autism Rating Scale (first and second editions; Schopler, Reichler, & Renner, 1988; Schopler, Van Bourgondien, Wellman, & Love, 2010), and the Autism Diagnostic Observation Schedule (first and second editions; Lord, Luyster, Gotham, & Guthrie, 2012; Lord, Rutter et al., 2012; Lord, Rutter, DiLavore, & Risi, 2000), establishing concurrent validity of the DSD scale. We also hypothesized that the DSD scale would demonstrate adequate sensitivity and specificity when distinguishing between youth with ASD and youth with no diagnoses and TD children (No Diagnosis group – a subset of the Non-ASD group) but have weaker sensitivity and specificity when distinguishing between youth with ASD and youth with other diagnoses (e.g., other developmental delays and non-ASD diagnoses; Other Diagnosis group – a subset of the Non-ASD group). Finally, we hypothesized that the ASD group would display significantly more caregiver-reported impairment than the Non-ASD group across the BASC-2 PRS-P clinical, adaptive, and content scales.

Methods

Participants

Data are from multisite early screening studies for developmental delays using the Modified Checklist for Autism in Toddlers (M-CHAT; Robins et al., 1999) and the revised version of the M-CHAT (M-CHAT-R; Robins et al., 2009). The Institutional Review Boards at each university approved study procedures. Informed consent was obtained from legal guardians. Toddlers were recruited from three samples (see Pandey et al., 2008 and Robins et al., 2014 for full recruitment procedures): low-risk screening (screened at well-child pediatric check-ups, n=186), high-risk screening (screened after referral for early intervention or for ASD diagnostic evaluation, n=32), and high-risk siblings (screened by psychologists based on having an older sibling with ASD, n=6). When children screened positive (i.e., elevated risk of developmental delay), their caregivers completed the structured M-CHAT or M-CHAT-R (referred to as M-CHAT(-R) when referencing both measures) Follow-Up over the phone with research staff. Toddlers who continued to show risk for developmental delay or whose pediatricians and/or parents expressed concerns regarding their development were invited to complete diagnostic evaluations. We included children between the ages of 24 and 63 months (n=224) and their primary caregivers, consistent with the age range of the BASC-2 PRS-P (2-5 years). Participants received comprehensive diagnostic evaluations conducted by a clinical team comprised of graduate student clinicians and licensed psychologists/developmental pediatricians in Atlanta, GA (n=173) or Storrs, CT (n=51).

Inclusion criteria were (a) diagnostic evaluation completed as part of M-CHAT(-R) studies and (b) a complete and valid BASC-2 PRS-P. Caregivers completed BASC-2 PRS-P forms at initial evaluations and/or at follow-up evaluations 1-2 years later. When the BASC-2 PRS-P was available for both time points, we included the earlier BASC-2 PRS-P and evaluation data. The sample included 117 children diagnosed with ASD and 107 children in the Non-ASD group. Within the Non-ASD group, 48 were diagnosed with other developmental delays (including global developmental delay and developmental language delay), seven received other diagnoses or had other diagnoses by history (externalizing disorders such as ADHD [n=3], phonological disorder [n=4], and genetic neurological syndromes [n=1]; some participants had multiple diagnoses), 38 received no diagnoses but demonstrated one or more areas of weakness in their development, and 14 were determined to be TD. We combined the first two non-ASD groups to create an Other Diagnosis subgroup and the last two groups of children to create a No Diagnosis subgroup for specific analyses. This allowed us to examine the utility of the BASC-2 DSD scale for identifying young children with ASD from a large heterogeneous sample of children (Non-ASD group) as well as from smaller, more homogeneous samples (Other Diagnosis vs. No Diagnosis groups).

Measures

Behavior Assessment System for Children-Second Edition (BASC-2)

The BASC-2 (Reynolds & Kamphaus, 2004) is a multidimensional assessment system that evaluates clinical and adaptive aspects of behavior and emotional functioning. We used the Parent Rating Scale-Preschool Form (PRS-P), valid for caregivers of children ages 2–5 years. Parents rate behaviors on a four-point frequency scale (i.e., 0=Never, 1=Sometimes, 2=Often, and 3=Almost Always). Item raw scores are summed and converted into standardized T-scores (M=50, SD=10). For clinical scales (e.g., Anxiety, Hyperactivity), higher scores represent more problematic behaviors, with T-scores between 60 and 69 considered at-risk, and T-scores of 70 or above being clinically significant. On adaptive scales (e.g., Adaptability, Social Skills), lower scores are indicative of deficits, with T-scores between 31 and 40 falling in the at-risk range and T-scores equal to or less than 30 considered clinically significant. Authors of the BASC-2 PRS-P report adequate reliability and validity. Individual clinical and adaptive scales of the BASC-2 PRS-P have a median test-retest reliability of .77 (range = .72–.85), and median inter-rater reliability of .74 (range = .53–.88). In addition, T-scores are reported for six clinical content scales (Anger Control, Bullying, Developmental Social Disorders [DSD], Emotional Self-Control, Executive Functioning, Negative Emotionality) and one adaptive content scale (Resiliency). These scales combine items from multiple constructs to detect patterns of behavior. The content scales have a median test-retest reliability of .75 (range = .66–.84) and a median inter-rater reliability of .61 (range = .59–.74). The BASC-2 PRS-P includes four validity indexes that evaluate consistency and bias in caregiver reports; we excluded protocols that had elevations on one or more of these indexes.

The DSD scale (see Table 1) measures behaviors related to deficits in social skills, communication, interests, and activities (Reynolds & Kamphaus, 2004). The scale also includes items that capture difficulties with attention, self-injurious behaviors, emotion regulation, and flexibility. Importantly, the behaviors measured by the DSD scale are seen across multiple neurodevelopmental disorders, including but not limited to ASD. The DSD scale has strong test-retest reliability (.84) and inter-rater reliability (.74). In a study of school-aged children without intellectual disability (Volker et al., 2010), a cut-score of 60 on the DSD scale had excellent sensitivity (.98) and specificity (.95) for identifying youth with ASD (n=62) from those who had no history of developmental, psychiatric, or learning issues (n=62).

Table 1.

BASC-2 PRS-P Items Contributing to DSD Scale^a

Item
4.	Compliments others
6.	Has a short attention span
9.	Has trouble making new friends
30.	Provides full name when asked
43.	Communicates clearly
54.	Makes friends easily
64.	Bangs head
73.	Acts strangely
75.	Encourages others to do their best
78.	Is chosen last by other children for games
97.	Adjusts well to changes in routine
98.	Shows feelings that do not fit the situation
117.	Throws tantrums

^aReprinted with permission from publisher

Autism Diagnostic Observation Schedule, First and Second Editions (ADOS; ADOS-2)

The ADOS (Lord et al., 2000) and ADOS-2 (Lord, Luyster et al., 2012; Lord, Rutter et al., 2012) are semi-structured assessments of communication, play, and social interaction designed to measure the presence of symptoms of ASD in individuals aged 12 months through adulthood. Four (ADOS) or five (ADOS-2) modules are available based on language level and age. The Toddler Module (only ADOS-2) is used with children who are between ages 12–30 months who do not consistently use phrase speech, Module 1 is used with children who are speaking single words, and Module 2 is suitable for children who have acquired phrase speech. The authors of the ADOS and ADOS-2 (referred to as ADOS(-2) when referencing both measures) report adequate reliability estimates for internal consistency, inter-rater reliability, and test-retest reliability (Lord et al., 2000; Lord, Luyster et al., 2012; Lord, Rutter et al., 2012). We used calibrated severity scores (CSS) as an estimate of the overall severity level of ASD symptoms across the modules (Toddler, 1, or 2) and editions (Esler et al., 2015; Hus, Gotham, & Lord, 2014).

Childhood Autism Rating Scale, First and Second Editions (CARS; CARS2-Standard Form [ST])

The CARS (Schopler et al., 1988) and CARS2-ST (Schopler et al., 2010) are standardized measures used to assess symptoms of autism spectrum disorder in children ages 2 years and older. Evaluators incorporate direct observations of behavior along with parent report to rate children on 15 items, including relating to people, imitation, and adaptation to change. Items are scored on a 7-point Likert scale rated from 1 to 4 in half-point increments. Item scores are then summed and classify the child according to severity of symptoms of autism: No-to-Mild, Mild-to-Moderate, and Moderate-to-Severe Symptoms of Autism. The CARS2-ST retained the original content and recommended cutoff values of the CARS. Reliability estimates for the CARS include an internal consistency of .94, test-retest reliability of .88, and inter-rater reliability of .71. Criterion related validity established by correlating total CARS scores and general clinical ratings of autism severity resulted in a correlation of .84. The CARS2-ST has similarly strong psychometric properties (Schopler et al., 2010). For this project, we used total scores from both CARS and CARS2-ST (referred to as CARS(-2) below).

Modified Checklist for Autism in Toddlers, Revised with Follow-Up (M-CHAT-R/F)

The M-CHAT-R/F (Robins et al., 2009) is a 2-stage, 20-item yes/no caregiver report checklist designed to screen for ASD in children ages 16–30 months. It is a revised version of the original Modified Checklist for Autism in Toddlers with Follow-Up (M-CHAT/F; Robins et al., 1999). If children screen positive, caregivers complete structured follow-up questions that ask for specific behavioral examples. The authors report adequate internal consistency for the 2-stage screen (Cronbach’s α=.79; Robins et al., 2014). Using the recommended cut-off of 3 initially and 2 on Follow-Up, the 2-stage M-CHAT-R/F has a positive predictive value of .48 and a negative predictive value of .99 in low-risk samples (Robins et al., 2014). Total scores from the M-CHAT-R/F (but not the M-CHAT/F) were included in specific analyses (detailed below).

Mullen Scales of Early Learning (Mullen)

The Mullen (Mullen, 1995) is a standardized assessment of cognitive and motor development for young children (birth-68 months). The Mullen assesses abilities in four domains, Visual Reception (visual discrimination, matching, categorization, and memory skills), Receptive Language (ability to understand language), Expressive Language (ability to use language to communicate), and Fine Motor (small movements with hands and fingers), which are combined to create the Early Learning Composite (ELC). The author of the Mullen reports strong psychometric properties, including test-retest reliability for the scales (rs from .71–.79), interscorer reliability, construct validity, and concurrent validity (Mullen, 1995).

The diagnostic evaluation included a larger battery of measures not included in this paper, such as parent interviews of ASD symptoms (Autism Diagnostic Interview-Revised, Lord, Rutter, & Le Couteur, 1994; Toddler ASD Symptom Interview, Barton, Boorstein, Herlihy, Dumont-Mathieu, & Fein, 2012), measures of adaptive functioning (Vineland Adaptive Behavior Scales[-II]; Sparrow, Balla, & Cicchetti, 1984; Sparrow, Cicchetti, & Balla, 2005), and a detailed history form.

Data Analyses

We used independent samples t-tests to examine potential age and overall cognitive abilities (Mullen ELC) differences between the ASD and Non-ASD groups and chi-square analyses to examine group differences by sex and ethnicity. To examine ethnicity, we combined all ethnicity groups, excluding Caucasian, into one Non-Caucasian group due to the small number of individuals in each of these groups.

Some of the distributions of continuous variables in the total sample and separately in ASD and Non-ASD groups did not meet assumptions of normality. As such, we used nonparametric Kendall’s τ correlation coefficients to examine relationships among variables in the total sample.

We ran a series of analyses to evaluate the clinical utility of the BASC-2 DSD scale for identifying toddlers and preschool children with ASD. We used Kendall’s τ correlation coefficients between DSD scale scores and M-CHAT-R total scores, CARS(-2) total scores, and ADOS(-2) CSS to examine concurrent validity. We limited the M-CHAT-R and DSD scale analysis to data from children with M-CHAT-R total scores that were collected within three months of their evaluations. We also conducted three receiver operating characteristic (ROC) analyses to determine sensitivity and specificity for scores on the DSD scale differentiating youth with ASD from three groups of children: 1) children without an ASD diagnosis (Non-ASD group, n=107), 2) children with no diagnoses or who were determined to be TD at their evaluations (No Diagnosis subgroup, n=52), and 3) children with other developmental delays or other pre-existing diagnoses (Other Diagnosis subgroup, n=55). These analyses addressed a large heterogeneous sample, replicated methods from previous studies (e.g., Volker et al., 2010), and examined a sample comparable to children who might be seen at a specialty clinic, respectively. For each ROC analysis, we selected the cut-score with the best balance between sensitivity and specificity values. We then calculated positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (LR+), and negative likelihood ratio (LR−) for each cut-score.

Finally, to examine mean score differences on BASC-2 scales between the ASD and Non-ASD groups, we ran three multivariate analyses of covariance (MANCOVAs): one for clinical scales, one for adaptive scales, and one for content scales. We included age and sex as covariates for each MANCOVA.

Results

Demographics

Demographic characteristics are shown in Table 2. The ASD and Non-ASD groups did not differ in regard to the average age of the participants at evaluation, t(222)=−1.49, p=.14, or the proportion of participants who identified as Caucasian and Non-Caucasian, χ²(1, N=224)=2.17, p=.14. The ASD group did, however, demonstrate significantly lower cognitive abilities (Mullen ELC), t(219)= −4.34, p < .001, and a higher ratio of males to females χ²(1, N=224)=4.15, p=.04, compared to the Non-ASD group.

Table 2.

Demographic Characteristics of Diagnostic Groups

	ASD (N=117)		Non-ASD (N=107)		Other Diagnosisa (N=55)		No Diagnosisa (N=52)
	N	M (SD)	N	M(SD)	N	M (SD)	N	M (SD)
Age (Months)	117	37.59 (9.17)	107	39.50 (9.99)	55	38.59 (9.99)	52	40.47 (9.35)
Mullen
ELC	116	67.36 (20.75)	105	80.24 (23.33)	55	63.62 (14.34)	50	98.52 (16.67)
Exp. Language	116	30.15 (11.91)	105	37.04 (13.10)	55	28.76 (8.78)	50	46.14 (10.85)
Rec. Language	117	30.95 (13.88)	105	38.51 (14.33)	55	28.16 (8.37)	50	49.90 (10.32)
Fine Motor	116	30.10 (13.01)	105	36.09 (14.22)	55	28.02 (10.76)	50	44.96 (12.18)
Visual Reception	116	34.91 (16.19)	105	43.73 (17.03)	55	33.29 (13.79)	50	55.22 (12.20)
CARS(-2)	115	32.46 (5.66)	106	20.84 (3.48)	55	22.10 (3.32)	51	19.49 (3.14)
M-CHAT-R	42	5.88 (4.19)	37	3.76 (3.80)	16	5.56 (3.83)	21	2.38 (3.22)
ADOS(-2) CSS	117	6.79 (1.88)	105	1.84 (1.17)	55	2.07 (1.30)	50	1.58 (.95)
BASC-2 DSD Scale	116	65.93 (10.50)	105	57.70 (9.77)	54	60.61 (8.54)	51	54.61 (10.11)
	N (%)		N (%)		N (%)		N (%)
Sex
Female	30 (25.6%)		41 (38.3%)		21 (38.2%)		20 (38.5%)
Male	87 (74.4%)		66 (61.7%)		34 (61.8%)		32 (61.5%)
Ethnicity
Caucasian	53 (45.3%)		59 (55.1%)		23 (41.8%)		36 (69.2%)
African American	37 (31.6%)		31 (29.0%)		22 (40.0%)		9 (17.3%)
Asian or Pacific Islander	7 (6.0%)		1 (0.9%)		0 (0.0%)		1 (1.9%)
Latino/a	3 (2.6%)		5 (4.7%)		2 (3.6%)		3 (5.8%)
Native   American/Native   Alaskan	0 (0.0%)		1 (0.9%)		1 (1.8%)		0 (0.0%)
Multiracial	14 (12.0%)		10 (9.3%)		7 (12.7%)		3 (5.8%)
Not Reported	3 (2.6%)		0 (0.0%)		0 (0.0%)		0 (0.0%)

Note. ELC=Early Learning Composite, presented as Standard Scores. Exp.=Expressive, Rec.=Receptive. All other Mullen scores presented as T-scores. DSD=Developmental Social Disorders. BASC-2 DSD Scale scores presented as T-scores. ADOS(-2) CSS=Calibrated severity score. CSS, CARS(-2) and M-CHAT-R scores presented as raw scores.

^aOther Diagnosis and No Diagnosis groups combined to create the Non-ASD group.

DSD Scale: Concurrent Validity and Relationships with Other Variables

Kendall’s τ correlation coefficients among age, Mullen ELC, CARS(-2) scores, ADOS(-2) CSS, and BASC-2 DSD scale scores in the entire sample are shown in Table 3. We also include M-CHAT-R scores from participants (ASD=42, Non-ASD=37) whose caregivers completed the M-CHAT-R within three months of the evaluations. The DSD scale showed moderate positive relationships with M-CHAT-R scores, τ(77)=.38, p < .001, CARS(-2) scores, τ(219)=.40, p < .001, and ADOS(-2) CSS, τ(220)=.30, p < .001. The DSD scale had significant negative relationships with age, τ(221)= −.10, p=.03 and Mullen ELC, τ(218)= −.30, p < .001.

Table 3.

Correlations Among DSD Scores, ASD Measures, Age, and Mullen ELC in Total Sample^a

Variable	1	2	3	4	5	6
1. Age	1.00 (224)
2. Mullen ELC	.14** (221)	1.00 (221)
3. CARS(-2)	−.16** (221)	−.40** (218)	1.00 (221)
4. ADOS(-2) CSS	−.08 (222)	−.33** (219)	.68** (220)	1.00 (221)
5. M-CHAT-R Total	−.10 (79)	−.33** (78)	.34** (77)	.32** (78)	1.00 (79)
6. BASC-2 DSD	−.10* (221)	−.30** (218)	.40** (219)	.30** (220)	.38** (77)	1.00 (221)

^aNumbers in parentheses equal N. ELC=Early Learning Composite. CSS=Calibrated Severity Score. DSD=Developmental Social Disorders.

^*p≤.05,

^**p≤.001

We also examined relationships among all BASC-2 scales. The DSD scale had significant positive relationships with the other content scales on which higher scores indicate more problematic behaviors (i.e., Anger Control, Bullying, Emotional Self-Control, Executive Functioning, Negative Emotionality; all τ between .30–.56, all ps < .001) and a significant negative relationship with the Resiliency scale, τ(221)= −.56, p < .001. Many of the BASC-2 clinical, adaptive, and content scales were significantly related to one another with small to moderate Kendall’s τ correlation coefficients, and the directions of the relationships were similar to those reported in the BASC-2 manual.

Clinical Utility of BASC-2 DSD Content Scale

For the ASD and Non-ASD groups, the area under the ROC curve for the DSD scale was .73 (see Figure 1). This analysis indicated that a cut-score of 61 produced the best balance of sensitivity and specificity (see Table 4). With this score, the DSD scale accurately detected 72% of children with ASD and 63% of children without an ASD diagnosis. The LR+ of 1.93 indicates that a child with autism is nearly twice as likely to score above this threshold than below.

Fig. 1.

ROC Curve for Cut-Scores on the BASC-2 DSD Content Scale and Area Under the Curve (AUC) for a. ASD and Non-ASD Groups, b. ASD and No Diagnosis Groups, and c. ASD and Other Diagnosis Groups

Table 4.

Psychometrics for Multiple Cut-Scores on the BASC-2 DSD Content Scale

Cut-Score (T-Score)	Sensitivity	Specificity	PPV	NPV	LR+	LR−
ASD (n=116) and Non-ASD (n=105)
60	.76	.58	.67	.69	1.81	.42
61	.72	.63	.68	.67	1.93	.45
62	.72	.63	.68	.67	1.93	.45
63	.62	.70	.69	.62	2.04	.55
64	.60	.70	.69	.61	1.98	.57
65	.58	.73	.71	.61	2.17	.58
66	.56	.73	.70	.60	2.10	.60
67	.52	.80	.74	.60	2.59	.60
68	.51	.81	.75	.60	2.67	.61
69	.41	.88	.78	.57	3.27	.68
70	.28	.90	.74	.53	2.63	.81
ASD (n=116) and No Diagnosis (n=51)
60	.76	.73	.86	.57	2.76	.33
61	.72	.75	.86	.54	2.81	.38
62	.72	.75	.86	.54	2.81	.38
63	.62	.76	.86	.47	2.64	.50
64	.60	.76	.85	.46	2.56	.52
65	.58	.78	.86	.45	2.68	.54
66	.56	.78	.86	.44	2.60	.56
67	.52	.84	.88	.43	3.30	.57
68	.51	.84	.88	.43	3.24	.58
69	.41	.88	.89	.39	3.44	.67
70	.28	.92	.89	.36	3.52	.79
ASD (n=116) vs. Other Diagnosis (n=54)
60	.76	.44	.75	.46	1.37	.54
61	.72	.52	.76	.46	1.49	.55
62	.72	.52	.76	.46	1.49	.55
63	.62	.63	.78	.44	1.68	.60
64	.60	.63	.78	.43	1.63	.63
65	.58	.69	.80	.43	1.83	.62
66	.56	.69	.79	.42	1.78	.64
67	.52	.76	.82	.42	2.15	.64
68	.51	.78	.83	.42	2.29	.63
69	.41	.87	.87	.41	3.13	.68
70	.28	.87	.82	.36	2.13	.83

Note. Bolded text indicates the cut-score with the best balance of sensitivity and specificity

For the ASD and No Diagnosis groups, the area under the ROC curve was .79. This ROC analysis indicated that a cut-score of 60 yielded the best balance of sensitivity and specificity. With this cut-score, the DSD scale accurately detected 76% of toddlers and preschool children with ASD and 73% of young children with no diagnoses. The LR+ indicates that a child with ASD is 2.76 times more likely to score above this threshold than below.

For the ASD and Other Diagnosis groups, the area under the ROC curve was .67. This ROC analysis indicated that a cut-score of 63 yielded the best balance of sensitivity and specificity. With this cut-score, the DSD scale accurately detected 62% of toddlers and preschool children with ASD and 63% of young children with other diagnoses. The LR+ indicates only a slight increase in the likelihood that a child with ASD will screen positive.

Score Profiles of ASD and Non-ASD Groups on BASC-2 Clinical, Adaptive, and Content Scales

Descriptive statistics for BASC-2 clinical, adaptive, and content scales are shown in Table 5. Box’s test indicated that the assumption of equality of covariance matrices was met for all MANCOVAs. The MANCOVA for BASC-2 clinical scales revealed a significant multivariate effect of diagnostic group, Pillai’s trace=.20, F(8,210)=6.73, p < .001, η²=.20. Significant diagnostic group effects were identified for Atypicality, Withdrawal, and Attention Problems (all ps ≤ .001), with the ASD group having more problematic behavior compared to the Non-ASD group across scales. The MANCOVA for BASC-2 adaptive scales also revealed a significant multivariate effect of diagnostic group, Pillai’s trace=.10, F(4,216)=5.76, p < .001, η²=.10. Significant diagnostic group effects were identified for Adaptability, Social Skills, Daily Living Skills, and Functional Communication (all ps ≤.03), with the ASD group demonstrating fewer adaptive behaviors compared to the Non-ASD group across scales. Finally, the MANCOVA for BASC-2 content scales revealed a significant multivariate effect of diagnostic group, Pillai’s trace=.22, F(7,211)=8.55, p < .001, η²=.22. Significant diagnostic group effects were identified for DSD, Emotional Self-Control, and Resiliency (all ps ≤ .04), with the ASD group showing more problematic behavior on the DSD and Emotional Self-Control scales and fewer adaptive behaviors) on the Resiliency scale compared to the Non-ASD group.

Table 5.

Means and Standard Deviations for BASC-2 Clinical, Adaptive, and Content Scale T-Scores and Univariate Test Results with Age and Sex as Covariates for ASD and Non-ASD Groups

Clinical Scales	ASD M (SD)	Non-ASD M (SD)	F	p-value	Partial Eta2
Hyperactivity	54.76 (10.98)	53.05 (12.03)	1.61	.21	.007
Aggression	47.19 (10.11)	48.42 (10.64)	.72	.40	.003
Anxiety	46.99 (10.74)	48.04 (9.63)	.68	.41	.003
Depression	51.57 (9.88)	52.86 (11.79)	.74	.39	.003
Somatization	49.08 (9.10)	51.21 (10.37)	2.46	.12	.011
Atypicality	64.63 (14.49)	56.59 (11.43)	22.60	<.001	.094
Withdrawal	59.74 (11.48)	52.83 (10.93)	20.28	<.001	.085
Attention Problems	60.60 (9.40)	56.16 (9.49)	12.08	.001	.053
Adaptive Scales	ASD M (SD)	Non-ASD M (SD)	F	p-value	Partial Eta2
Adaptability	43.12 (10.75)	46.38 (11.49)	4.97	.03	.022
Social Skills	36.65 (9.99)	43.08 (11.14)	19.62	<.001	.082
Daily Living Skills	37.85 (10.54)	42.81 (12.61)	9.99	.002	.044
Functional Communication	37.38 (7.92)	42.25 (9.15)	19.95	<.001	.084
Content Scales	ASD M (SD)	Non-ASD M (SD)	F	p-value	Partial Eta2
Anger	56.32 (10.38)	54.18 (9.78)	1.99	.16	.009
Bully	51.99 (9.51)	50.38 (9.97)	1.24	.27	.006
Developmental Social Disorders	65.93 (10.50)	57.70 (9.77)	37.02	<.001	.146
Emotional Self-Control	56.47 (12.20)	53.12 (12.35)	4.41	.04	.020
Executive Functioning	57.70 (10.27)	55.63 (11.09)	2.13	.15	.010
Negative Emotionality	55.02 (11.16)	53.72 (11.60)	.82	.37	.004
Resiliency	40.70 (11.56)	46.33 (11.50)	13.52	<.001	.059

Note. Due to missing data, the number of participants used in each MANCOVA is as follows: Clinical Scales ASD N=117, Non-ASD N=104; Adaptive scales ASD N=117, Non-ASD N=106; Content scales ASD N=116, Non-ASD N=105.

Discussion

Findings from the present study suggest that the DSD scale of the BASC-2 PRS-P, a multidimensional rating scale, is a helpful tool for detecting ASD risk in young children. When tested in heterogeneous sample of children with ASD, other non-ASD diagnoses (developmental delay, ADHD), and no diagnoses, the DSD scale accurately detected 72% of children with ASD and 63% of children without an ASD diagnosis; however, screening accuracy was influenced by the composition of the comparison group. Specifically, the ability of the DSD scale to differentiate children with ASD from those without diagnoses (Sensitivity=.76, Specificity=.73) is greater than its ability to differentiate children with ASD from those with other diagnoses (Sensitivity=.62, Specificity=.63). These results suggest that the BASC-2 DSD scale has validity in predicting the presence of ASD relative to typical development. When used for this purpose, the scale exceeds the minimum sensitivity and specificity of .70 recommended for screening tools by the American Academy of Pediatrics (Council on Children with Disabilities, Section on Developmental Behavioral Pediatrics, Bright Futures Steering Committee, & Medical Home Initiatives for Children with Special Needs Project Advisory Committee, 2006). Nonetheless, results also suggest that the scale’s ability to differentiate ASD from other childhood diagnoses is more limited, a finding consistent with other studies of multidimensional screeners, such as the CBCL (Havdahl et al., 2016; Muratori et al., 2011; Predescu et al., 2013). Further, perhaps unsurprisingly, the DSD scale appears to be a less sensitive and specific screen for ASD than the M-CHAT-R/F (Sensitivity=.85, Specificity=.99; Robins et al. 2014). These findings are consistent with and expand upon prior investigations of the DSD scale, which have focused on school-aged children and adolescents but did not examine the relative accuracy of ASD screening in children with a range of diagnoses (i.e., ASD, developmental delays, and other clinical diagnoses). Moreover, findings support the broad assessment goals of the BASC-2 and similar child behavior rating scales, which aim to identify risk across a broad range of behavioral domains (Reynolds & Kamphaus, 2004). Specifically, the use of the BASC-2 DSD scale embedded in the larger instrument holds the potential to save clinician and family time and expense by not only identifying ASD-risk, but also a range of potential comorbidities or alternative causes for symptom presentation that can then be explored through more comprehensive evaluation.

The significant modest, positive correlations between the DSD scale and the CARS(-2) and ADOS(-2) CSS support the conceptualization of the DSD scale as a useful screener for symptoms associated with, but not limited to, ASD. Whereas the CARS(-2) and ADOS(-2) are ASD-specific diagnostic tools, the DSD scale assesses behaviors associated with multiple neurodevelopmental disorders. Furthermore, the DSD scale does not evaluate ASD-specific symptoms at the same level of detail as diagnostic tools. This likely attenuates the relationship between the DSD scale and diagnostic measures because they evaluate overlapping but distinct symptoms. Relatedly, the magnitude of the correlations between the DSD scale and the CARS(-2) and ADOS(-2) CSS (.40 and .30, respectively) is similar to those of the M-CHAT-R/F (.34 and .32, respectively) in this sample. One previous study of the Japanese version of the M-CHAT found that it had a moderate positive relationship with the CARS Tokyo Version, r(23)=.58, p=.002 (Inada, Koyama, Inokuchi, Kuroda, & Kamio, 2011). This suggests that the DSD scale has similar relationships to ASD-specific diagnostic tools as those of the M-CHAT-R/F, despite the fact that the M-CHAT-R/F has stronger sensitivity (.85) and specificity (.99) than the DSD scale. Although the M-CHAT-R/F is a more precise screening tool than the DSD scale, both tools measure a broader range of developmental issues than diagnostic instruments and thus may not be expected to have more than moderate relationships with diagnostic tools.

A secondary aim of the present study was to compare BASC-2 PRS-P clinical and adaptive scores for young children with and without ASD. Consistent with a previous study of the BASC-2 in school-aged children and adolescents (Volker et al., 2010), young children with ASD demonstrated significant differences across the clinical, adaptive and content scales relative to children without ASD, including significantly more atypical behaviors, withdrawal and attention problems, greater difficulties with adaptability, social skills, daily living skills and functional communication, and reduced emotional self-control and resiliency. In contrast, young children with ASD in the current study did not obtain higher scores on the Anxiety and Depression scales when compared to youth without ASD. This difference in the internalizing profiles of older versus younger children with ASD on the BASC-2 may reflect the increasing verbal abilities, social difficulties, and emotional and interpersonal insights of youth with ASD as they grow older. Youth with ASD are more likely to experience teasing and bullying over time (Cappadocia, Weiss, & Pepler, 2012). They are also likely to grow more aware of and sensitive to their social differences and become more articulate about their internal experiences as they mature developmentally, increasing their risk for anxiety and depression symptoms (Ghaziuddin, Ghaziuddin, & Greden, 2002; Kuusiko et al., 2008). Consistent with this reasoning, several studies suggest internalizing symptoms, such as social anxiety and depression, are positively associated with age and self-awareness in youth with ASD (Gotham, Bishop, Brunswasser, & Lord, 2014; Gotham, Brunswasser, & Lord, 2015), a pattern apparent even in young children (Green, Ben-Sasson, Soto, & Carter, 2012). These relationships may explain, in part, why anxiety and depression symptoms were not significantly greater for youth with versus without ASD in the present sample, which was younger and less verbally-advanced than previous studies of cognitively-able, school-aged youth with ASD (Volker et al., 2010). At the same time, the lower Resiliency score of the ASD group observed in this study may speak to the inherent vulnerability of these children to develop psychiatric symptoms over time due to fewer coping skills at a young age.

Implications for Practitioners

Findings from the current study have several implications for practitioners working with toddlers and preschoolers. First, while more research is needed to replicate results and make definitive recommendations, preliminary evidence suggests that the BASC-2 PRS-P may be a parsimonious strategy for educators and clinicians to detect ASD risk in young children, while also evaluating other areas of risk and ability. Practitioners are cautioned against using the BASC-2 and similar brief rating scales in isolation for making diagnostic decisions. Relatedly, the ability of the BASC-2 DSD scale to differentiate ASD from other forms of developmental delay or clinical diagnoses is more limited. Accordingly, the BASC-2 is likely to be most useful to clinicians as a starting point to guide further assessment and, particularly, the selection of more comprehensive diagnostic measures.

Strengths, Limitations, and Directions for Future Research

The current study builds on existing literature by focusing on participants in the toddler and preschool population. This is especially important considering the emphasis on early diagnosis and intervention. Additionally, we used stringent diagnostic criteria to classify the clinical samples. All children in this study (including those diagnosed with other developmental delays) were diagnosed via comprehensive evaluations administered by trained clinicians, including gold-standard measures of ASD such as the ADOS(-2), as well as measures of adaptive behavior (i.e., Vineland-II) and cognition (i.e., Mullen).

Future research should attempt to replicate the results of the current study using larger clinical and control groups; this work is necessary before the BASC-2 DSD scale can be endorsed as a valid screening instrument for ASD. Second, further research comparing BASC-2 profiles of children with ASD to those diagnosed with other disorders (such as disruptive behavior disorders or ADHD) will be valuable in discriminating between challenging behaviors observed in children with ASD versus patterns seen in children with other disorders. Congruence of multiple raters, such as parents and teachers, also may be helpful in examining the optimal strategies for detecting children in need of further evaluation. Finally, research should be conducted with educational and primary care agencies to determine if the use and examination of behavior rating scales such as the BASC-2 result in a difference in diagnostic outcomes and service delivery for young children at risk for ASD.