Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Oct 1.
Published in final edited form as: Autism. 2016 Jul 11;21(7):841–851. doi: 10.1177/1362361316654083

Psychiatric comorbidity in autism spectrum disorder: Correspondence between mental health clinician report and structured parent interview

Nicole Stadnick 1,2, Colby Chlebowski 1,2,3, Mary Baker-Ericzén 1,2,3,4, Margaret Dyson 1,2,4, Ann Garland 2,5, Lauren Brookman-Frazee 1,2,3
PMCID: PMC5226915  NIHMSID: NIHMS815591  PMID: 27407039

Abstract

Publicly funded mental health services are critical in caring for children with autism spectrum disorder. Accurate identification of psychiatric comorbidity is necessary for effective mental health treatment. Little is known about psychiatric diagnosis for this population in routine mental health care. This study (1) examined correspondence between psychiatric diagnoses reported by mental health clinicians and those derived from a structured diagnostic interview and (2) identified predictors of agreement between clinician-reported and diagnostic interview-derived diagnoses in a sample of 197 children aged 4–14 years with autism spectrum disorder receiving mental health services. Data were drawn from a randomized effectiveness trial conducted in publicly funded mental health services. Non–autism spectrum disorder diagnoses were assessed using an adapted version of the Mini-International Neuropsychiatric Interview, parent version. Cohen’s kappa was calculated to examine agreement between Mini-International Neuropsychiatric Interview, parent version and clinician-reported diagnoses of comorbid conditions. Children met criteria for an average of 2.83 (standard deviation = 1.92) Mini-International Neuropsychiatric Interview, parent version diagnoses. Agreement was poor across all diagnostic categories (κ values: 0.06–0.18). Logistic regression identified child gender and clinical characteristics as significant predictors of agreement for specific diagnoses. Results underscore the need for training mental health clinicians in targeted assessment of specific psychiatric disorders and prioritizing treatment development and testing for specific diagnoses to improve care for children with autism spectrum disorder served in publicly funded mental health settings.

Keywords: autism spectrum disorder, community mental health, diagnostic correspondence, psychiatric comorbidity

Introduction

The prevalence of autism spectrum disorder (ASD) is currently estimated to be one in every 68 children (Centers for Disease Control and Prevention, 2014). Children with ASD have complex service needs necessitating long-term treatment (Ganz, 2007; Leslie et al., 2001; Liptak et al., 2006; Mandell et al., 2006; Wang and Leslie, 2010). The costs of caring for this population are high and estimated to be US$268 billion in the United States in 2015 (Leigh and Du, 2015). Costs are particularly high for children receiving publicly funded services compared to private insurance (Wang et al., 2013). Given the rising prevalence rate and high costs associated with caring for individuals with ASD, maximizing the effectiveness and efficiency of routine care services is crucial.

Children with ASD are served in multiple service systems including developmental disability, special education, vocational rehabilitation, and mental health (MH) (Brookman-Frazee et al., 2009; Jacobson and Mulick, 2000; Wood et al., 2015). MH services that are publicly funded through the special education system or state Medicaid programs play a particularly important role in providing treatment for behavior problems and co-occurring psychiatric disorders (Brookman-Frazee et al., 2009; Brookman-Frazee et al., 2012; Semansky et al., 2011). Over 70% of children with ASD meet criteria for at least one comorbid psychiatric disorder when assessed using structured diagnostic interviews (Joshi et al., 2010; Leyfer et al., 2006; Simonoff et al., 2008) and these comorbid problems often persist from childhood to adolescence (Simonoff et al., 2013). Accurate classification of comorbid psychiatric conditions is essential to inform the development, implementation, and appropriate application of evidence-based interventions in MH settings.

Research suggests that co-occurring psychiatric conditions can be reliably diagnosed using structured diagnostic interview measures for children with ASD such as the Schedule for Affective Disorders and Schizophrenia for School-Age Children (K-SADS-E; Orvaschel, 1994; Orvaschel and Puig-Antich, 1987), the Diagnostic Interview Schedule for Children, Version IV (DISC-IV; Shaffer et al., 1998), and the Child and Adolescent Psychiatric Assessment–parent version (CAPA; Angold and Costello, 2000). It is noteworthy that many of these studies adapted the original measures to meet the unique needs of the ASD population (De Bruin et al., 2007; Joshi et al., 2010; Leyfer et al., 2006; Simonoff et al., 2008), and these assessments were typically conducted by highly trained research staff. It is recognized that ASD symptoms (e.g. repetitive behaviors and avoidance of social situations) can present similarly to symptoms of other common psychiatric disorders, rendering diagnostic assessment especially challenging for this population (White et al., 2012).

There is growing understanding of the profile of children with ASD receiving publicly funded MH services. Specifically, these racially and ethnically diverse children typically have average to above average cognitive functioning and demonstrate high rates of challenging behaviors (Brookman-Frazee et al., 2009, 2010; Brookman-Frazee et al., 2012; Joshi et al., 2014; Levy et al., 2010; Mandell et al., 2005, 2007; Williams et al., 2009). Recent research using a structured diagnostic interview with parents of children with ASD to assess comorbid psychiatric conditions indicates that over 90% of children receiving publicly funded MH services meet criteria for at least one other common psychiatric condition with attention deficit hyperactivity disorder (ADHD), being the most prevalent. Furthermore, many children meet criteria for multiple comorbid diagnoses.

Although no published data exist on the correspondence between clinician-assigned diagnoses and those derived from gold standard diagnostic interviews for children with ASD, research on psychiatric diagnosis for a broad population of children receiving publicly funded MH services indicates that there is generally low-to-moderate concordance between MH clinician-assigned diagnoses and those determined by structured diagnostic instruments (Ezpeleta et al., 1997; Haine et al., 2007; Jensen and Weisz, 2002; Jensen-Doss et al., 2014; Lewczyk et al., 2003; Rettew et al., 2009). In particular, Lewczyk et al. (2003) reported that the prevalence of mood disorders was significantly higher based on clinician assignment, whereas the prevalence of ADHD, disruptive behavior disorders, and anxiety disorders were significantly higher based on structured diagnostic interview (DISC-IV). In addition, Lewczyk et al. (2003) identified several child demographic, clinical, and contextual factors that predicted agreement such as child gender, severity of externalizing problems, and psychiatric comorbidity. It is possible that correspondence between clinician-reported diagnoses and diagnoses from structured diagnostic instruments may be even poorer for the ASD population given that MH clinicians typically have limited training in diagnostic assessment and treatment of youth with ASD (Brookman-Frazee et al., 2010, 2012). Diagnostic agreement may be further impacted by the clinical complexity of children with ASD and their unique clinical characteristics.

Furthermore, the low concordance between diagnoses derived from clinician report versus structured diagnostic interview has potentially significant implications for subsequent treatment and child outcomes. Specifically, most treatment approaches, including evidence-based intervention models, are developed for specific diagnoses or diagnostic categories and based on the assumption that the child’s diagnosis is accurate. Possible consequence of missing or overlooking symptoms could result in a treatment approach that is not indicated for the child’s presenting symptomatology. Therefore, it is critical that diagnostic consistency across research and practice settings is improved to maximize the public health impact of evidence-based practice (EBP) implementation efforts (Jensen-Doss et al., 2014).

Understanding potential gaps between gold standard, structured diagnostic assessment, and routine psychiatric diagnostic assessment has the potential to identify targets to improve the quality of standard care. The primary study purpose was to examine the concordance between child diagnoses reported by clinicians and those derived from a structured diagnostic interview, an adapted Mini-International Neuropsychiatric Interview, parent version (MINI-KID-P; Sheehan et al., 1998), in a sample of children with ASD receiving publicly funded MH services. A secondary aim was to identify predictors of agreement between clinician-reported and MINI-KID-P–derived diagnoses. Methods used in the Lewczyk et al. (2003) study guided those applied in this study and were specifically adapted for children with ASD.

Methods

Procedures

Data were drawn from a randomized community effectiveness trial of “An Individualized Mental Health Intervention for ASD” (AIM HI; Brookman-Frazee and Drahota, 2010) conducted in publicly funded community and school-based MH programs. Brookman-Frazee et al. (2012) is a package of evidence-based strategies designed to reduce challenging behaviors in school-aged children with ASD designed for delivery in community MH services. In the parent study, clinicians were recruited for participation from participant programs, and children were recruited from participant clinicians’ caseloads. Children were eligible if they (1) were aged 4–14 years, (2) spoke English or Spanish as their primary language, (3) had an existing ASD diagnosis on record, and (4) met exhibited clinically significant ASD symptoms on a standardized ASD diagnostic measure In all, 93% of children were classified as “Autism” or “ASD” on the Autism Diagnostic Observation Schedule-2 (ADOS-2; Lord et al., 2012), administered by research staff. The remaining children (6%, n = 12) scored in the clinical range on all subscales of the Social Responsiveness Scale-2 and had been previously diagnosed by a community provider specializing in ASD diagnosis and developmental disabilities (e.g. psychologist from California Regional Center). These inclusion criteria were established to represent the population of children with ASD served in publicly funded MH systems.

Data for analyses were derived from baseline assessments of the larger randomized effectiveness trial. These assessments were conducted in person with separate caregiver and child interview assessments lasting between 2 and 3 h (described in measures). Families received a US$40 gift card for completing the assessment. Study procedures were approved by the institution through which this study was conducted.

Participants

Participants included a subset of 197 children from the parent study drawn from the caseloads of 177 clinicians (e.g. marriage and family therapists, social worker, psychologists, and psychiatrists), providing therapist in 28 MH programs in San Diego and Los Angeles Counties. Children were included in the sample if they had available data regarding (1) clinician report of non-ASD Diagnostic and Statistical Manual of Mental Disorders (4th ed., DSM-IV) Axis I diagnoses and (2) the MINI-KID-P. Based on these criteria, five children from the parent study sample (n = 202) were excluded because they were missing clinician report or MINI-KID-P data. See Table 1 for a summary of participant characteristics.

Table 1.

Participant characteristics.

Child sociodemographics and clinical
characteristics		 M (SD); range or n (%)
Gender (boys) 165 (84%)
Age (years) 9.12 (2.45); 4–14 years
Ethnicity: Hispanic/Latino 117 (59%)
Race
  White 147 (77%)
  African American 13 (7%)
  Asian/Pacific Islander 9 (5%)
  American Indian/Alaskan Native 9 (5%)
  Multiracial 13 (7%)
Cognitive Standard Score (derived from WASI-
II or DAS-II)		 88.25 (16.50); 33–140
ADOS-2 classification
  Autism 149 (76%)
  ASD 34 (17%)
  Non-spectrum 12 (6%)
ADOS-2 comparison score 7.07 (2.04); 1–10
SRS total T-score 80.06 (10.86); 41–104
ECBI Intensity T-score 63.12 (10.27); 38–94
MINI-KID-P diagnoses (number for which
criteria met)		 2.83 (1.92); 0–9.00
diagnoses
Parent sociodemographics N = 192
Gender (female) 179 (93%)
Relation to child (biological parent) 175 (91%)
Marital status (married) 98 (51%)
Household income (<US$35,000) 119 (62%)
Maternal education
  Less than high school 37 (20%)
  High school/GED 73 (38%)
  Some college 28 (15%)
  Bachelor’s degree 25 (13%)
  Graduate degree 9 (5%)
  Other (e.g. trade/technical school) 18 (10%)
Therapist sociodemographics and training N = 177
Gender (female) 151 (85%)
Age (years) 34.19 (8.13); 23–66
Ethnicity: Hispanic/Latino 62 (35%)
Race
  White 130 (73%)
  Asian/Pacific Islander 19 (11%)
  African American 5 (3%)
  American Indian/Alaskan Native 3 (2%)
  Multiracial 2 (1%)
Education
  Bachelor’s degree 18 (10%)
  Master’s degree 137 (78%)
  Doctoral degree 16 (9%)
  Other 5 (3%)
Primary discipline
  Marriage family therapy 74 (42%)
  Social work 45 (25%)
  Psychology 28 (16%)
  School psychology 19 (11%)
  Psychiatry 5 (3%)
  Other 6 (3%)
Licensed 50 (28%)
  Marriage family therapy 22 (30%)
  Social work 13 (29%)
  Psychology 6 (21%)
  School psychology 3 (16%)
  Psychiatry 5 (100%)
  Other 3 (50%)
Open in a new tab

SD: standard deviation; WASI-II: Wechsler Abbreviated Scale of Intelligence-II; DAS-II: Differential Ability Scale-II; ADOS-2: Autism Diagnostic Observation Schedule-2; ASD: autism spectrum disorder; SRS: Social Responsiveness Scale; ECBI: Eyberg Child Behavior Inventory; MINI-KID-P: Mini-International Neuropsychiatric Interview, parent version; GED: General Educational Development.

Measures

Outcome measures

Clinician-reported DSM-IV Axis I diagnoses

At baseline, clinicians completed a questionnaire in which they were asked to list a maximum of three current DSM-IV Axis I diagnoses for each child participant. Each individual diagnosis was categorized into one of four mutually exclusive: (1) ADHD, (2) disruptive behavior disorders, (3) anxiety disorders, and (4) mood disorders. The disruptive behavior disorders category included: oppositional defiant disorder (ODD), conduct disorder, disruptive behavior disorder–not otherwise specified, intermittent explosive disorder, and adjustment disorder with conduct. The anxiety disorders category included generalized anxiety disorder (GAD), separation anxiety disorder, social phobia, posttraumatic stress disorder, obsessive compulsive disorder (OCD), panic disorder, specific phobia, and adjustment disorder with anxiety. The mood disorders category included major depressive disorder, dysthymic disorder, bipolar I or II disorder, and adjustment disorder with mood. Although individual diagnoses were categorized into mutually exclusive categories, children could be assigned diagnoses in multiple categories.

MINI-KID-P

The MINI-KID-P (Sheehan et al., 1998) was used to determine the presence of co-occurring psychiatric disorders. The MINI-KID-P is a structured diagnostic interview to assess symptoms of Axis I disorders as listed in the DSM-IV and The International Classification of Diseases, Tenth Edition (ICD-10; World Health Organization (WHO), 1993). It has strong inter-rater and test–retest reliability and established construct validity. Sensitivity (0.61–1.00) and specificity (0.81–1.00) are also strong (Sheehan et al., 2010). The MINI-KID-P was administered in person or via phone to the primary caregiver identified as knowing the child well. The following 12 MINI-KID-P modules were used based on the most common psychiatric comorbid disorders for children with ASD in MH settings (Brookman-Frazee and Drahota, 2010, 2012; Joshi et al., 2010): ADHD, ODD, panic disorder, agoraphobia, separation anxiety disorder, social phobia, specific phobia, OCD, GAD, tic disorders, major depressive episode, dysthymia, manic and hypomanic episodes, bipolar disorder I or II. In examining MINI-KID-P data for this sample, there were only five children who had a therapist-reported non-ASD diagnosis that fell outside of the scope of the administered MINI-KID-P modules (e.g. enuresis and learning/communication disorder). Similar to the clinician-reported diagnoses, MINI-KID-P diagnoses were collapsed into four mutually exclusive categories: (1) ADHD, (2) disruptive behavior disorders, (3) anxiety disorders, and (4) mood disorders. The disruptive behavior disorders category included ODD exclusively. The anxiety disorders category included panic disorder, agoraphobia, separation anxiety disorder, social phobia, specific phobia, OCD, and GAD. The mood disorders category included major depressive episode, manic and hypomanic episodes, and bipolar I and II.

The MINI-KID-P was adapted for an ASD sample by adding symptom explanations and follow-up questions to facilitate differentiation between ASD and other psychiatric symptoms. Specifically, prior to asking about anxiety, parents were provided a brief explanation of the physiological component that occurs with anxiety to reduce potential over-endorsement based on behaviors related to ASD. For most modules, parents were asked to give examples following endorsement of screening items, skip patterns were removed, and additional items were added to the OCD and tic modules to assist in distinguishing between restrictive, repetitive behaviors (characteristic of ASD) compared to compulsions or tics. Finally, the dysthymia module questions were embedded into the major depressive episode module to reduce time and burden. All interviewers were trained to criterion prior to administering the MINI-KID-P by the third author (M.B.E.) who is a licensed clinical psychologist with clinical expertise in both youth MH and ASD and diagnostic assessment. Training involved didactic instruction about the MINI-KID-P and the use of the measure with an ASD population, audio-recording review of MINI-KID-P administration, role-play administration practice, and administration practice. Each MINI-KID-P interview was audio recorded. The audio recordings of the first two MINI-KID-P administrations of each interviewer and a subsequent sample of administered MINI-KID-P interviews were reviewed by the third author (M.B.E.) to control for administration accuracy and drift. Additionally, the research team met regularly to review and discuss MINI-KID-P administration and scoring questions. A total of six trained study personnel administered the MINI-KID-P to caregivers as part of the baseline assessments. Trained personnel included two doctoral-level clinical psychologists, two masters-level clinicians, and two bachelors-level research staff with clinical experience in ASD. Three of the six study personnel were fluent in Spanish. All six study personnel had clinical research experience with children with ASD and MH problems.

For the first half of the enrolled sample (n = 103), all MINI-KID-P protocols were reviewed by M.B.E. or one of the two clinical psychologist research staff (co-authors C.C. and M.D.) to check scoring algorithms and to refine the training guidelines developed for this study’s ASD sample. For the second half of the sample (n = 99), 50% of the MINI-KID-P protocols were independently reviewed and scored by a licensed clinical psychology research staff member for quality monitoring. Inter-rater reliability was calculated on this sub-sample using the kappa statistic. Kappa values ranged from 0.87 to 1.00 for the four diagnostic categories examined in this study.

ASD symptom severity measures

ADOS-2

The ADOS-2 is a semi-structured observational assessment administered by a trained provider to assist in the diagnosis of ASD (Lord et al., 2012). Children are administered one of the five modules based on their language and developmental level. The quality of the child’s social affect, communication, and restricted, repetitive behaviors is rated. An algorithm is applied to the scores and result in a classification of “Autism,” “Autism Spectrum Disorder,” or “Non-Spectrum” based on standardized cutoff values. The overall total score and the child’s chronological age are used to identify the ADOS-2 Comparison Score that ranges from 1 (minimal-to-no-evidence of ASD-related symptoms) to 10 (high level of ASD-related symptoms). The ADOS-2 has well-established psychometric properties including strong reliability and validity across individual items and classification (Lord et al., 2012). For this study, only the ADOS-2 Comparison Score was used to characterize ASD severity.

Social Responsiveness Scale-2

The Social Responsiveness Scale-2 (SRS-2) is a 65-item caregiver report measure of the severity of social communication impairments associated with ASD (Constantino and Gruber, 2012). It is a common standardized assessment tool used to aid in ASD diagnostic assessment. The SRS-2 has strong internal consistency, inter-rater reliability, and diagnostic discrimination in school-aged children (Constantino and Gruber, 2012). The SRS-2 total T-score (M = 50; standard deviation (SD) = 10) was used to characterize the severity of social communication difficulties.

Cognitive functioning measures

Based on child age and/or developmental level, one of the following measures was administered to estimate the child’s cognitive abilities at the baseline study assessment.

Wechsler Abbreviated Scale of Intelligence-II

The Wechsler Abbreviated Scale of Intelligence-II (WASI-II) is a brief standardized assessment of cognitive ability that is administered by a trained provider (Wechsler, 2011). The WASI-II has demonstrated strong internal consistency, with average reliability coefficients ranging from 0.87 to 0.91 for children, and strong convergent and discriminant validity (Wechsler, 2011). Four subtests are administered to yield a full-scale intelligence quotient (FSIQ), Verbal Comprehension Index score, and a Perceptual Reasoning Index score. Each score is represented as a standard score (M = 100; SD = 15).

Differential Ability Scale-II

The Differential Ability Scale-II (DAS-II) is a comprehensive assessment of cognitive ability that is administered by a trained provider (Elliott, 2007). A FSIQ, the General Conceptual Ability (GCA) score, and three core composites (Nonverbal Reasoning Ability, Verbal Ability, and Spatial Ability) are produced. The GCA and three core composites are represented as standard scores (M = 100; SD = 15). The DAS-II has established psychometric properties with strong internal consistency for both the standardization sample and special clinical populations and evidentiary support for convergent and discriminant validity (Elliott, 2007).

For this study, the DAS-II was administered to children aged younger than 6 years or who had suspected or confirmed developmental delays because research has recommended use of the DAS-II for children with special needs (Deutsch and Joseph, 2003). The FSIQ score derived from the WASI-II or DAS-II was used to characterize the child’s global cognitive abilities.

Behavior problem measure

Eyberg Child Behavior Inventory

The Eyberg Child Behavior Inventory (ECBI) is a 36-item caregiver report measure that assesses the frequency and intensity of child disruptive behaviors (Eyberg and Pincus, 1999). Two scores are yielded: an intensity score that represents the frequency of disruptive behaviors and a problem score that represents the total number of behaviors that parents endorsed as being a problem for them. The ECBI has strong test–retest reliability (reliability coefficient of 0.86 for the intensity score) and good construct and concurrent validity (Boggs et al., 1990; Eyberg and Ross, 1978; Robinson et al., 1980). For this study, the intensity T-score (M = 50; SD = 10) was used to characterize the severity of the child’s behavior problems.

Data analytic plan

The analysis plan for this study was informed by the Lewczyk et al. (2003) study given the similar community MH service system context and study aims.

Diagnostic prevalence and agreement

Descriptive statistics were first performed to examine prevalence data for diagnostic categories based on the MINI-KID-P and clinician report. The McNemar test, which compares paired proportions, was used to contrast differences in the proportion of youth who received diagnoses in each category by assessment type. Cohen’s kappa statistic was calculated to examine agreement between MINI-KID-P and clinician-reported diagnoses.

Child correlates of global diagnostic agreement

To probe differences in global agreement (i.e. agreement about the presence or absence of a diagnosis) between MINI-KID-P and clinician diagnoses, subsequent logistic regression analyses were performed to identify child-level predictors of global agreement. The following child-level predictor variables were examined: (1) sociodemographic characteristics: age, gender, ethnicity (Hispanic versus non-Hispanic), (2) ASD symptom severity: ADOS-2 Comparison Score and the SRS-2 Total Score, (3) severity of challenging behaviors: ECBI Intensity T-score, and (4) cognitive functioning: FSIQ from the WASI-II or GCA from the DAS-II. A model was calculated for each diagnostic category with agreement (presence or absence of the diagnosis) between clinician-reported and MINI-KID-P diagnoses as the binary outcome variable.

Child correlates of specific patterns of diagnostic agreement

Significant child correlates from the global agreement analyses were included in logistic regression analyses to examine specific patterns of diagnostic agreement. Specifically, positive agreement was defined as both clinician report and MINI-KID-P concordance on the presence of a diagnosis; negative agreement was defined as both clinician report and MINI-KID-P concordance on the absence of a diagnosis. Variables that were statistically significant at p < 0.05 in the global agreement analyses were included in these models as predictors.

Results

Diagnostic prevalence and global diagnostic agreement

Results characterizing diagnostic prevalence and concordance divided by assessment type are displayed in Table 2. Overall, the same pattern was demonstrated for each diagnostic category with higher prevalence rates for the MINI-KID-P compared to clinician report. Likewise, there was a high proportion of non-ASD Axis I disorder based on clinician report compared to the MINI-KID-P. Specifically, for ADHD, the prevalence was 36% based on clinician report and 78% based on the MINI-KID-P (p < 0.001). For disruptive behavior disorders, the prevalence was 17% based on clinician report compared to 57% based on the MINI-KID-P (p < 0.001). For anxiety disorders, the prevalence was 23% based on clinician report compared to 57% based on the MINI-KID-P (p < 0.001). For mood disorders, the prevalence was 11% based on clinician report compared to 31% based on the MINI-KID-P (p < 0.001). Finally, the prevalence of no diagnosis was 28% based on clinician report compared to 8% based on the MINI-KID-P (p < 0.001).

Table 2.

Diagnostic prevalence and concordance between therapist report and MINI-KID diagnoses (n = 197).

Diagnostic category Therapist
n (%)		 MINI-KID-P
n (%)		 McNemar test
(p value)		 MINI-KID-P+
Therapist+
(%)		 MINI-KID-P−
Therapist−
(%)		 MINI-KID-P−
Therapist+
(%)		 MINI-KID-P+
Therapist−
(%)		 κ
ADHD 70 (36) 154 (78) <0.001 29 16 49 0.06
Any disruptive 33 (17) 113 (57) <0.001 13 39 4 44 0.13
behavior disorder
Any anxiety disorder 45 (23) 112 (57) <0.001 18 38 5 39 0.18
Any mood disorder 22 (11)   60 (31) <0.001   5 64 6 25 0.10
No non-ASD Axis I
disorder		 55 (28)   16 (8) <0.001 – – – – 0.05
Open in a new tab

MINI-KID-P: Mini-International Neuropsychiatric Interview parent interview; +: diagnosis present; −: diagnosis absent; ADHD: attention-deficit/hyperactivity disorder; ASD: autism spectrum disorder.

The McNemar test compares the proportion of children who received each diagnosis between the assessment methods.

Global diagnostic agreement (i.e. agreement on the presence or absence of a specific diagnostic category) as represented by κ values was poor across all diagnostic categories according to standards for agreement beyond chance (κ = 0.4) (Cohen, 1960). The κ values were 0.06 for ADHD, 0.13 for disruptive behavior disorders, 0.18 for anxiety disorders, 0.10 for mood disorders, and 0.05 for no diagnosis.

Child correlates of global diagnostic agreement

Child gender, the severity of social communication difficulties, and the intensity of behavior problems were significantly associated with global agreement between specific clinician report and MINI-KID-P diagnostic categories (Table 3). Specifically, for disruptive behavior disorders, global agreement was significantly less likely for children with higher ECBI Intensity scores (odds ratio (OR) = 0.93, p < 0.001). For anxiety disorders, global agreement was significantly less likely for children with greater SRS-2 total T-scores (OR = 0.96, p < 0.05). For mood disorders, global agreement was over three times greater for females (OR = 3.26, p < 0.05) and significantly less likely for children with greater ECBI Intensity scores (OR = 0.94, p < 0.01). There were no statistically significant child characteristics associated with global diagnostic agreement for ADHD.

Table 3.

Predictors of global agreement between MINI-KID-P and therapist diagnoses (n = 197).

Predictor ADHD odds ratio
(95% CI)		 Disruptive behavior
disorders odds
ratio (95% CI)		 Anxiety disorders
odds ratio (95% CI)		 Mood disorders odds
ratio (95% CI)
Child age 0.89 (0.77, 1.04) 0. 87 (0.74, 1.02) 1.05 (0.90, 1.23) 0.89 (0.74, 1.05)
Child gender (male = 1,
female = 2)		 0.53 (0.22, 1.26) 0.56 (0.22, 1.43) 1.74 (0.72, 4.21) 3.26 (1.27, 8.32)*
Child ethnicity (Hispanic) 1. 37 (0.73, 2.60) 0.85 (0.44, 1.64) 0.90 (0.48, 1.72) 1.58 (0.77, 3.23)
ADOS-2 Comparison
Score		 1.03 (0.65, 1.63) 1.23 (0.77, 1.97) 1.02 (0.65, 1.61) 1.58 (0.96, 2.59)
SRS-2 total T-score 0.99 (0.97, 1.02) 0.99 (0.97, 1.03) 0.96 (0.93, 0.99)* 0.99 (0.96, 1.03)
ECBI Intensity T-score 0.99 (0.95, 1.02) 0.93 (0.89, 0.97)*** 1.02 (0.98, 1.05) 0.94 (0.90, 0.98)**
FSIQ 1.00 (0.98, 1.02) 1.00 (0.97, 1.02) 1.01 (0.99, 1.03) 1.01 (0.98, 1.03)
Open in a new tab

MINI-KID-P: Mini-International Neuropsychiatric Interview, parent interview; CI: confidence interval; ADHD: attention deficit hyperactivity disorder; ADOS-2: Autism Diagnostic Observation Schedule-2; SRS-2: Social Responsiveness Scale-2; ECBI: Eyberg Child Behavior Inventory; FSIQ: full scale intelligence quotient.

*

p < 0.05,

**

p < 0.01, and

***

p < 0.001.

Child correlates of diagnostic agreement for presence/absence of each diagnostic category

Child gender, the severity of social communication difficulties, and the intensity of behavior problems were also significantly associated with specific patterns of agreement (Table 4). Agreement on the presence of ADHD was lower for children with higher SRS-2 total T-scores (OR = 0.97, p < 0.05) but greater for children with higher ECBI Intensity scores (OR = 1.06, p < 0.01). Agreement on the absence of ADHD was lower for females (OR = 0.35, p < 0.05) and for children with greater ECBI Intensity scores (OR = 0.92, p < 0.001).

Table 4.

Predictors of specific patterns of agreement between the MINI-KID-P and therapist diagnoses (n = 197).

Predictor ADHD odds ratio
(95% CI)
 Disruptive behavior disorders odds
ratio (95% CI)
 Anxiety disorders odds ratio
(95% CI)
 Mood disorders odds ratio
(95% CI)
+ Agreement −Agreement + Agreement −Agreement + Agreement −Agreement + Agreement −Agreement
Child gender
(male= 1, female = 2)		 1.03 (0.43, 2.44) 0.35 (0.13, 0.91)* 0.56 (0.19, 1.63) 0.55 (0.21, 1.39) 1.07 (0.39, 2.94) 1.62 (0.66, 3.98) 1.93 (0.23, 15.87) 2.59 (1.17, 5.76)*
SRS-2 total T-score 0.97 (0.94, 1.00)* 1.00 (0.96, 1.04) 0.97 (0.92, 1.01) 1.01 (0.98, 1.05) 1.08 (1.03, 1.12)** 0.92 (0.88, 0.95)*** 1.03 (0.96, 1.10) 0.98 (0.95, 1.01)
ECBI Intensity T-score 1.06 (1.01, 1.09)** 0.92 (0.87, 0.96)*** 1.12 (1.06, 1.18)** 0.86 (0.82, 0.90)*** 0.99 (0.95, 1.03) 1.02 (0.99, 1.06) 1.00 (0.93, 1.07) 0.96 (0.93, 0.99)*
Open in a new tab

MINI-KID-P: Mini-International Neuropsychiatric Interview, parent interview; CI: confidence interval; ADHD: attention deficit hyperactivity disorder; FSIQ: full scale intelligence quotient. SRS-2: Social Responsiveness Scale; ECBI: Eyberg Child Behavior Inventory.

*

p < 0.05,

**

p < 0.01, and

***

p < 0.001.

For disruptive behavior disorders, agreement regarding the presence of any disruptive behavior disorder was greater for children with higher ECBI Intensity scores (OR = 1.12, p < 0.01). Agreement on the absence of disruptive behavior disorders was lower for children with higher ECBI Intensity scores (OR = 0.86, p < 0.001). Finally, for mood disorders, agreement on the absence of a mood disorder was greater for females (OR = 2.59, p < 0.05) but lower for children with higher ECBI Intensity scores (OR = 0.96, p < 0.05).

Discussion

Diagnostic comorbidity based on two distinct sources was high in this sample of 197 school-aged children with ASD receiving publicly funded MH services. ADHD was the most prevalent diagnosis based on both clinician report and structured diagnostic assessment. Yet, diagnostic agreement between clinician report and the MINI-KID-P was poor for this sample. Prevalence rates across diagnostic categories differed significantly based on diagnostic method. Across all diagnostic categories, the prevalence of comorbid diagnoses was significantly greater for MINI-KID-P-derived diagnoses compared to clinician-reported diagnoses, and clinicians were more likely not to report a non-ASD Axis I disorder compared to reports from the MINI-KID-P. These results are generally consistent with previous research (Jensen-Doss et al., 2014; Lewczyk et al., 2003) indicating substantial disagreement between clinician-reported diagnoses and those derived from structured diagnostic assessments with generally higher identification rates across diagnostic categories based on structured assessments, with the exception of a greater prevalence of mood disorders identified by clinicians in the Lewczyk et al. (2003) study.

There are several possible explanations underlying the poor agreement observed. One is methodological differences in diagnostic assessment process and procedure. Clinicians were using the DSM-IV nosology, which guides clinicians to include co-occurring symptoms like executive dysfunction related to ADHD as encompassed within an ASD diagnosis. This diagnostic process is different from the MINI-KID-P, which quantifies these symptoms into unique diagnoses. It is noteworthy that 36% of children in this sample received a clinician-reported diagnosis of ADHD despite the DSM-IV differential diagnosis instructions on subsuming symptoms of ADHD into the ASD diagnosis. This may reflect how usual care clinicians conceptualize diagnoses (i.e. symptoms of ADHD and those of ASD can be distinguished) and supports the changes in the Diagnostic and Statistical Manual of Mental Disorders (5th ed., DSM-5) nosology to permit separate diagnoses.

Another possibility suggested by Jensen-Doss et al. (2014) is that clinicians were influenced by search satisficing, when alternative diagnostic explanations or additional comorbidities are ignored once a plausible diagnosis is determined (Galanter and Patel, 2005). This proclivity may have contributed to comorbid diagnostic underestimates by clinicians. Additionally, only selected modules from the MINI-KID-P were administered to caregivers based on the most common co-occurring psychiatric diagnoses for children with ASD, so the structured interview did not assess all DSM-IV diagnoses that clinicians could assign. Relatedly, clinicians were limited to providing three diagnoses while there was no maximum placed on the number of MINI-KID-P diagnoses. Although this is a potential limitation to the data, the impact is minimized as 81% of the child participants received two or fewer clinician-reported diagnoses. This is consistent with a study by Jensen-Doss et al. (2014) in which MH providers only assigned an average of 1.4 diagnostic categories per child.

Another methodological and larger service delivery factor to consider is that the MINI-KID-P explicitly and comprehensively asks about specific symptoms within a discrete diagnosis whereas clinician diagnostic assessment may be more global and focus primarily on MH problems that are more apparent or functionally impairing. Clinicians in usual care practice settings are typically not required to use structured diagnostic measures, thus resulting in diagnoses assigned based on informal assessment procedures (Bickman, 2000; Garland et al., 2003). Furthermore, child MH clinicians often gather information about family functioning and processes above and beyond sole focus on the child, potentiating oversight or less attention on comprehensive and specific diagnostic assessment (Baker-Ericzén et al., 2015). Alternatively, MINI-KID-P diagnoses are derived from one data point (i.e. caregiver response), whereas clinician diagnoses can incorporate multiple data sources to provide greater contextualization of the child’s clinical presentation. It is important to note that the higher diagnostic prevalence rates derived from the MINI-KID-P are more consistent with prevalence rates documented in research using structured diagnostic interview assessment with children with ASD (Joshi et al., 2010).

In addition to the findings regarding poor diagnostic agreement, specific child sociodemographic and clinical characteristics predicted global and specific patterns of agreement between clinician and MINI-KID-P diagnoses. Specifically, child gender, severity of social communication difficulties, and intensity of behavioral problems were significantly associated with global agreement (i.e. agreement regarding the overall presence or absence of a diagnosis) and specific patterns of agreement. Notably, child age, ethnicity, cognitive abilities, and ASD severity were not predictive of global or specific patterns of agreement. These child sociodemographic and clinical predictors of global and specific patterns of agreement are generally consistent with previous research that has examined diagnostic agreement between clinician report and structured diagnostic interviews for children receiving MH treatment (Jensen-Doss et al., 2014; Lewczyk et al., 2003). Specific comparisons to the Lewczyk et al. (2003) study, which informed the methodology applied in this study, indicate that child gender, age, and behavior problems were shared predictors of diagnostic agreement, but differed based on the diagnostic category, which may be attributed, in part, to methodological differences in measurement of prediction variables and the specific clinical youth populations included.

Several strengths of this study are notable. First, this represents one of the few studies to examine comorbid diagnostic concordance in children with ASD and specifically for children receiving routine MH services. The MH service system plays an important role in caring for children with ASD underscoring the value of examining diagnostic practices in this service setting. In addition, this study included children from diverse racial/ethnic backgrounds, with greater than half of the sample being Hispanic/Latino. Although diagnostic agreement was poor overall, there were no ethnic disparities in agreement identified. Finally, this study is unique in that there was a sufficient sample size and corresponding statistical power to examine correlates of agreement between clinician and MINI-KID-P diagnoses. This methodological strength is rare in ASD services research and allowed us to examine a variety of child sociodemographic and clinical characteristics.

In addition to the methodological factors mentioned that may have influenced diagnostic agreement, this study’s primary limitation is that there is no standard of validity by which to confirm comorbid diagnoses. Although research staff who administered the MINI-KID-P completed standardized training and underwent ongoing checks to ensure administration adherence, the accuracy of these diagnoses cannot be definitely established. Similarly, the specific diagnostic practices that clinicians used were not assessed, so the diagnostic methods and accuracy of clinician diagnoses are also unknown. In fact, given established research on the lack of agreement between diagnostic assessment methods, it is difficult to identify one specific “gold standard” for diagnostic assessment in research or clinical practice.

The lack of diagnostic agreement between a structured diagnostic interview and clinician assessment highlights the complexity of accurate diagnosis of psychiatric comorbidity in children with ASD served in community MH settings. Accurate identification of MH problems is crucial to delivering appropriately tailored intervention for this population. This is particularly challenging because symptoms of ASD may overlap with distinct psychiatric disorders (e.g. disruptive behavior disorders) and non-ASD psychiatric problems (e.g. anxiety) may manifest differently in children with ASD (e.g. increases in repetitive or sensory behaviors) (Leyfer et al., 2006; Zainal et al., 2014). Improved assessment may be guided by the shift to use of DSM-5 nosology, which allows for separate diagnoses to be assigned when symptoms co-occur (e.g. those in ASD and ADHD) to facilitate tailored treatment plans.

Study findings have implications for diagnostic assessment training provided to usual care clinicians. Specifically, results suggest the value of standardized diagnostic instruments or assessment procedures. While it would not be expected that usual care clinicians administer a structured diagnostic interview like the MINI-KID-P due to organizational factors and pragmatic limitations, there may be value in standardized screening assessment procedures. For example, given that a large portion of children with ASD met criteria for ADHD, ODD, and anxiety disorders in this sample, it is recommended that specific attention to symptoms of these disorders be emphasized and explored by community MH clinicians when completing their diagnostic interviews and formulating treatment plans. Results underscore the need for improved diagnostic evaluation and targeted treatment development and testing of specific diagnoses to ultimately inform efforts to improve well-directed care for youth with ASD served in MH settings.

Acknowledgments

Clinical Trial Registry Name: Effectiveness and Implementation of a Mental Health Intervention for ASD (AIM HI); Registry identification number: NCT02416323.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by National Institute of Mental Health grant, R01MH094317 (PI: Brookman-Frazee).

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

  1. Angold A, Costello EJ. The Child and Adolescent Psychiatric Assessment (CAPA) Journal of the American Academy of Child and Adolescent Psychiatry. 2000;39(1):39–48. doi: 10.1097/00004583-200001000-00015. [DOI] [PubMed] [Google Scholar]
  2. Baker-Ericzén MJ, Jenkins MM, Park S, et al. Clinical decision-making in community children’s mental health: using innovative methods to compare clinicians with and without training in evidence-based treatment. Child & Youth Care Forum. 2015;44(1):133–157. doi: 10.1007/s10566-014-9274-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bickman L. Are you satisfied with satisfaction? Mental Health Services Research. 2000;2(3):125–126. [Google Scholar]
  4. Boggs SR, Eyberg S, Reynolds LA. Concurrent validity of the Eyberg Child Behavior Inventory. Journal of Clinical Child Psychology. 1990;19(1):75–78. [Google Scholar]
  5. Brookman-Frazee L, Baker-Ericzen M, Stahmer A, et al. Involvement of youths with autism spectrum disorders or intellectual disabilities in multiple public service systems. Journal of Mental Health Research in Intellectual Disabilities. 2009;2(3):201–219. doi: 10.1080/19315860902741542. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Brookman-Frazee L, Drahota A. An Individualized Mental Health Intervention for Children with Autism Spectrum Disorders (AIM HI): A Model to Address Challenging Behaviors in Children with ASD—A Therapist Manual (Unpublished manual) La Jolla, CA: University of California, San Diego; 2010. [Google Scholar]
  7. Brookman-Frazee L, Drahota A, Stadnick N. Training community mental health therapists to deliver a package of evidence-based practice strategies for school-age children with autism spectrum disorders: a pilot study. Journal of Autism and Developmental Disorders. 2012;42(8):1651–1661. doi: 10.1007/s10803-011-1406-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Brookman-Frazee L, Drahota A, Stadnick N, et al. Therapist perspectives on community mental health services for children with autism spectrum disorders. Administration and Policy in Mental Health and Mental Health Services Research. 2012;39(5):365–373. doi: 10.1007/s10488-011-0355-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Brookman-Frazee LI, Taylor R, Garland AF. Characterizing community-based mental health services for children with autism spectrum disorders and disruptive behavior problems. Journal of Autism and Developmental Disorders. 2010;40(10):1188–1201. doi: 10.1007/s10803-010-0976-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Centers for Disease Control and Prevention. Prevalence of autism spectrum disorder among children aged 8 years— autism and developmental disabilities monitoring network, 11 sites, United States, 2010. Morbidity and Mortality Weekly Report. 2014;63(SS02):1–21. [PubMed] [Google Scholar]
  11. Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement. 1960;20(1):37–46. [Google Scholar]
  12. Constantino JN, Gruber C. Social Responsiveness Scale (SRS-2) Los Angeles, CA: Western Psychological Services; 2012. [Google Scholar]
  13. De Bruin EI, Ferdinand RF, Meester S, et al. High rates of psychiatric co-morbidity in PDD-NOS. Journal of Autism and Developmental Disorders. 2007;37(5):877–886. doi: 10.1007/s10803-006-0215-x. [DOI] [PubMed] [Google Scholar]
  14. Deutsch CK, Joseph RM. Brief report: cognitive correlates of enlarged head circumference in children with autism. Journal of Autism and Developmental Disorders. 2003;33(2):209–215. doi: 10.1023/a:1022903913547. [DOI] [PubMed] [Google Scholar]
  15. Elliott CD. Differential Ability Scales-II (DAS-II) San Antonio, TX: Pearson; 2007. [Google Scholar]
  16. Eyberg SM, Pincus D. Eyberg Child Behavior Inventory and Sutter-Eyberg Student Behavior Inventory-Revised: Professional Manual. Odessa, FL: Psychological Assessment Resources, Inc.; 1999. [Google Scholar]
  17. Eyber SM, Ross AW. Assessment of child behavior problems: the validation of a new inventory. Journal of Clinical Child Psychology. 1978;7(2):113–116. [Google Scholar]
  18. Ezpeleta L, de la Osa N, Domenech JM, et al. Diagnostic agreement between clinicians and the Diagnostic Interview for Children and Adolescents—DICA-R—in an outpatient sample. Journal of Child Psychology and Psychiatry, and Allied Disciplines. 1997;38(4):431–440. doi: 10.1111/j.1469-7610.1997.tb01528.x. [DOI] [PubMed] [Google Scholar]
  19. Galanter CA, Patel VL. Medical decision making: a selective review for child psychiatrists and psychologists. Journal of Child Psychology and Psychiatry. 2005;46(7):675–689. doi: 10.1111/j.1469-7610.2005.01452.x. [DOI] [PubMed] [Google Scholar]
  20. Ganz ML. The lifetime distribution of the incremental societal costs of autism. Archives of Pediatrics and Adolescent Medicine. 2007;161(4):343–349. doi: 10.1001/archpedi.161.4.343. [DOI] [PubMed] [Google Scholar]
  21. Garland AF, Kruse M, Aarons GA. Clinicians and outcome measurement: what’s the use? Journal of Behavioral Health Services and Research. 2003;30(4):393–405. doi: 10.1007/BF02287427. [DOI] [PubMed] [Google Scholar]
  22. Haine RA, Brookman-Frazee L, Tsai KH, et al. Clinician perspectives of diagnosis and perceived client change in “real world” psychotherapy for youth emotional and behavioral disorders. Journal of Child and Family Studies. 2007;16(5):712–728. [Google Scholar]
  23. Jacobson JW, Mulick JA. System and cost research issues in treatments for people with autistic disorders. Journal of Autism and Developmental Disorders. 2000;30(6):585–593. doi: 10.1023/a:1005691411255. [DOI] [PubMed] [Google Scholar]
  24. Jensen AL, Weisz JR. Assessing match and mismatch between practitioner-generated and standardized interview-generated diagnoses for clinic-referred children and adolescents. Journal of Consulting and Clinical Psychology. 2002;70(1):158–168. [PubMed] [Google Scholar]
  25. Jensen-Doss A, Youngstrom EA, Youngstrom JK, et al. Predictors and moderators of agreement between clinical and research diagnoses for children and adolescents. Journal of Consulting and Clinical Psychology. 2014;82(6):1151–1162. doi: 10.1037/a0036657. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Joshi G, Faraone SV, Wozniak J, et al. Examining the clinical correlates of autism spectrum disorder in youth by ascertainment source. Journal of Autism and Developmental Disorders. 2014;44(9):2117–2126. doi: 10.1007/s10803-014-2063-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Joshi G, Petty C, Wozniak J, et al. The heavy burden of psychiatric comorbidity in youth with autism spectrum disorders: a large comparative study of a psychiatrically referred population. Journal of Autism and Developmental Disorders. 2010;40(11):1361–1370. doi: 10.1007/s10803-010-0996-9. [DOI] [PubMed] [Google Scholar]
  28. Leigh JP, Du J. Brief report: forecasting the economic burden of autism in 2015 and 2025 in the United States. Journal of Autism and Developmental Disorders. 2015;45:4135–4139. doi: 10.1007/s10803-015-2521-7. [DOI] [PubMed] [Google Scholar]
  29. Leslie DL, Rosenheck RA, Horwitz SM. Patterns of mental health utilization and costs among children in a privately insured population. Health Services Research. 2001;36(1 Part 1):113–127. [PMC free article] [PubMed] [Google Scholar]
  30. Levy SE, Giarelli E, Lee LC, et al. Autism spectrum disorder and co-occurring developmental, psychiatric, and medical conditions among children in multiple populations of the United States. Journal of Developmental and Behavioral Pediatrics. 2010;31(4):267–275. doi: 10.1097/DBP.0b013e3181d5d03b. [DOI] [PubMed] [Google Scholar]
  31. Lewczyk CM, Garland AF, Hurlburt MS, et al. Comparing DISC-IV and clinician diagnoses among youths receiving public mental health services. Journal of the American Academy of Child and Adolescent Psychiatry. 2003;42(3):349–356. doi: 10.1097/00004583-200303000-00016. [DOI] [PubMed] [Google Scholar]
  32. Leyfer OT, Folstein SE, Bacalman S, et al. Comorbid psychiatric disorders in children with autism: interview development and rates of disorders. Journal of Autism and Developmental Disorders. 2006;36(7):849–861. doi: 10.1007/s10803-006-0123-0. [DOI] [PubMed] [Google Scholar]
  33. Liptak GS, Stuart T, Auinger P. Health care utilization and expenditures for children with autism: data from U.S. national samples. Journal of Autism and Developmental Disorders. 2006;36(7):871–879. doi: 10.1007/s10803-006-0119-9. [DOI] [PubMed] [Google Scholar]
  34. Lord C, Rutter M, DiLavore PC, et al. Autism Diagnostic Observation Schedule: ADOS-2. Los Angeles, CA: Western Psychological Services; 2012. [Google Scholar]
  35. Mandell DS, Cao J, Ittenbach R, et al. Medicaid expenditures for children with autistic spectrum disorders: 1994 to 1999. Journal of Autism and Developmental Disorders. 2006;36(4):475–485. doi: 10.1007/s10803-006-0088-z. [DOI] [PubMed] [Google Scholar]
  36. Mandell DS, Ittenbach RF, Levy SE, et al. Disparities in diagnoses received prior to a diagnosis of autism spectrum disorder. Journal of Autism and Developmental Disorders. 2007;37(9):1795–1802. doi: 10.1007/s10803-006-0314-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Mandell DS, Walrath CM, Manteuffel B, et al. Characteristics of children with autistic spectrum disorders served in comprehensive community-based mental health settings. Journal of Autism and Developmental Disorders. 2005;35(3):313–321. doi: 10.1007/s10803-005-3296-z. [DOI] [PubMed] [Google Scholar]
  38. Orvaschel H. Schedule for Affective Disorder and Schizophrenia for School-Age Children: Epidemiologic Version 5. Ft. Lauderdale, FL: Nova Southeastern University, Center for Psychological Studies; 1994. [Google Scholar]
  39. Orvaschel H, Puig-Antich J. Schedule for Affective Disorder and Schizophrenia for School-Age Children: Epidemiologic Version 4. Ft. Lauderdale, FL: Nova University; 1987. [Google Scholar]
  40. Rettew DC, Lynch AD, Achenbach TM, et al. Meta-analyses of agreement between diagnoses made from clinical evaluations and standardized diagnostic interviews. International Journal of Methods in Psychiatric Research. 2009;18(3):169–184. doi: 10.1002/mpr.289. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Robinson EA, Eyberg SM, Ross AW. The standardization of an inventory of child conduct problem behaviors. Journal of Clinical Child Psychology. 1980;9(1):22–28. [Google Scholar]
  42. Semansky RM, Xie M, Mandell DS. Medicaid’s increasing role in treating youths with autism spectrum disorders. Psychiatric Services. 2011;62(6):588. doi: 10.1176/ps.62.6.pss6206_0588. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Shaffer D, Fisher P, Lucas C. NIMH DISC-IV: Diagnostic Interview Schedule for Children. Parent-Informant. New York: Columbia University; 1998. [Google Scholar]
  44. Sheehan DV, Lecrubier Y, Sheehan KH, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry. 1998;59(Suppl 20):22–33. quiz 34–57. [PubMed] [Google Scholar]
  45. Sheehan DV, Sheehan KH, Shytle RD, et al. Reliability and validity of the Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-KID) Journal of Clinical Psychiatry. 2010;71(3):313–326. doi: 10.4088/JCP.09m05305whi. [DOI] [PubMed] [Google Scholar]
  46. Simonoff E, Jones CRG, Baird G, et al. The persistence and stability of psychiatric problems in adolescents with autism spectrum disorders. Journal of Child Psychology and Psychiatry. 2013;54(2):186–194. doi: 10.1111/j.1469-7610.2012.02606.x. [DOI] [PubMed] [Google Scholar]
  47. Simonoff E, Pickles A, Charman T, et al. Psychiatric disorders in children with autism spectrum disorders: prevalence, comorbidity, and associated factors in a population-derived sample. Journal of the American Academy of Child and Adolescent Psychiatry. 2008;47(8):921–929. doi: 10.1097/CHI.0b013e318179964f. [DOI] [PubMed] [Google Scholar]
  48. Wang L, Leslie DL. Health care expenditures for children with autism spectrum disorders in Medicaid. Journal of the American Academy of Child and Adolescent Psychiatry. 2010;49(11):1165–1171. doi: 10.1016/j.jaac.2010.08.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Wang L, Mandell DS, Lawer L, et al. Healthcare service use and costs for autism spectrum disorder: a Comparison between Medicaid and private insurance. Journal of Autism and Developmental Disorders. 2013;43(5):1057–1064. doi: 10.1007/s10803-012-1649-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Wechsler D. Wechsler Abbreviated Scale of Intelligence (WASI-II) San Antonio, TX: Pearson; 2011. [Google Scholar]
  51. White SW, Bray BC, Ollendick TH. Examining shared and unique aspects of social anxiety disorder and autism spectrum disorder using factor analysis. Journal of Autism and Developmental Disorders. 2012;42(5):874–884. doi: 10.1007/s10803-011-1325-7. [DOI] [PubMed] [Google Scholar]
  52. Williams ME, Atkins M, Soles T. Assessment of autism in community settings: discrepancies in classification. Journal of Autism and Developmental Disorders. 2009;39(4):660–669. doi: 10.1007/s10803-008-0668-1. [DOI] [PubMed] [Google Scholar]
  53. Wood JJ, McLeod BD, Klebanoff S, et al. Toward the implementation of evidence-based interventions for youth with autism spectrum disorders in schools and community agencies. Behavior Therapy. 2015;46(1):83–95. doi: 10.1016/j.beth.2014.07.003. [DOI] [PubMed] [Google Scholar]
  54. World Health Organization (WHO) The ICD-10 Classification of Mental and Behavioural Disorders: Diagnostic Criteria for Research. Geneva: WHO; 1993. [Google Scholar]
  55. Zainal H, Magiati I, Tan JW, et al. A preliminary investigation of the Spence Children’s Anxiety Parent Scale as a screening tool for anxiety in young people with autism spectrum disorders. Journal of Autism and Developmental Disorders. 2014;44(8):1982–1994. doi: 10.1007/s10803-014-2075-0. [DOI] [PubMed] [Google Scholar]

RESOURCES