Caregiver-Teacher Concordance of Challenging Behaviors in Children with Autism Spectrum Disorder Served in Community Mental Health Settings

Abstract

Children with autism spectrum disorder (ASD) exhibit high rates of challenging behaviors that impair functioning and represent the primary presenting problem in mental health (MH) services. Obtaining symptom reports from multiple informants is critical for treatment planning. This study evaluated caregiver-teacher concordance of ratings of the intensity of challenging behaviors in children with ASD receiving MH services, and identified child clinical factors associated with concordance. This sample included 141 children (M = 9.07 years), their caregivers, and teachers. Caregiver-teacher concordance of challenging behaviors was low and impacted by the degree and type of child psychiatric comorbidity. Findings support need for increased attention to the range of psychiatric problems children with ASD present to tailor treatment recommendations and service delivery.

Introduction

The significant prevalence and associated health care expenditures (estimated to be $268 billion) of caring for children with autism spectrum disorder (ASD) have prompted heightened focus on understanding the needs of these children and improving the care available to them (Centers for Disease Control and Prevention 2014; Leigh and Du 2015). The current healthcare landscape for children with ASD is fragmented and complex; these children are served in multiple service settings (Brookman-Frazee et al. 2009) to address their significant co-occurring medical, educational, and mental health needs. As children with ASD develop, their clinical characteristics and service needs change. In particular, mental health problems often arise as primary concerns in school-age children and adolescents with ASD and these problems often seriously impact functioning and necessitate significant educational and therapeutic intervention (Joshi et al. 2010; Kaat et al. 2013; Mattila et al. 2010; Mazzone et al. 2012). The estimated prevalence of co-occurring mental health problems in children with ASD assessed using structured diagnostic assessment measures in research studies is high and estimated at greater than 70% (Leyfer et al. 2006; Simonoff et al. 2008). In particular, challenging behaviors are exceedingly common and significantly contribute to functional impairment for children with ASD (Horner et al. 2002; Kim et al. 2000; Matson et al. 2009; Wood and Gadow 2010). Challenging behaviors represent a broad range of behavior problems such as aggression, noncompliance, and self-injury (Horner et al. 2002) and that are often manifestations of co-occurring psychiatric symptoms (Wood and Gadow 2010; Kim et al. 2000). Furthermore, within community mental health care, an important service system for children with ASD because of high rates of psychiatric comorbidity, challenging behaviors represent the primary presenting problem for this population (Brookman-Frazee et al. 2009, 2012b, c; Mandell et al. 2005).

Using multiple informants to assess symptoms and monitor response to treatment is considered essential for all children, and particularly for children with ASD who often have complex clinical presentations due to variety of co-occurring conditions and variation in social, communicative, and behavioral skills (Olsson et al. 2016; Brookman-Frazee et al. 2012a; Jepsen et al. 2012; Macintosh and Dissanayake 2006; Matson and Nebel-Schwalm 2007). Obtaining perspectives from key caregivers and providers, such as caregivers and teachers, regarding a child’s functioning can inform accurate diagnoses, appropriately tailored treatment, consistency in care across settings, and caregiver satisfaction with care for this clinical population (Sheridan and Kratochwill 2007; Tucker and Schwartz 2013). However, significant disagreement between informants can lead to difficulty in prioritizing treatment targets and selecting intervention strategies (De Los Reyes et al. 2015), especially for children with ASD who are clinically complex and have multiple service needs (Olsson et al. 2016; Brookman-Frazee et al. 2012a, 2009; Jepsen et al. 2012; Macintosh and Dissanayake 2006; Matson and Nebel-Schwalm 2007). In this paper we focus specifically on caregivers and teachers as informants of child behaviors given their significant role in daily care and ability to report on child functioning within different settings.

Although individual studies have shown variability in the extent to which caregivers and teachers similarly rate symptom severity in children with ASD, a recent meta-analysis (Stratis and Lecavalier 2015) indicates moderate agreement between caregivers and teachers across social skills, behavioral, and emotional problems. This effect size estimated in the meta-analysis for behavior problems specifically, (mean weighted r = .38) is comparable to the effect size (mean weighted r = .32) reported in a meta-analysis using a sample of typically developing youth rated on externalizing problems by parents and teachers (Achenbach et al. 1987). Importantly, Stratis and Lecavalier (2015) found that child characteristics including age, diagnosis (intellectual disability versus ASD), and cognitive functioning moderated patterns of informant agreement across symptom domains. Similarly, in a recent study, Azad et al. (2015), not included in the meta-analysis, found that caregiver and teacher ratings of social impairment were moderated by the degree of ASD symptom severity such that there was significant agreement for more severely affected children with ASD compared to children with less severe ASD symptoms. Taken together, this research highlights the importance of considering key child characteristics in interpretation of cross-informant concordance of behaviors in children with ASD.

Reasons for discrepancy in informant agreement and principles for interpreting and using informant discrepancies have been proposed (De Los Reyes et al. 2013, 2015; Kraemer et al. 2003). Foundational research on multi-informant assessment has highlighted that the following influence the variability in informant reporting: (1) the child trait or characteristic that informants are rating, (2) the situations in which informants have observed the child, (3) the unique perspectives or biases of the informant, and (4) measurement error (Kraemer et al. 2003). De Los Reyes and colleagues (2013) expanded this work by developing a theoretical framework to guide understanding of the clinical utility of divergent reports from multiple informants. A key principle is that discrepancies may represent meaningful differences in observed child behaviors that should not simply be treated as measurement error. In line with this theory, research suggests that caregivers and teachers demonstrate different patterns of rating children with ASD that may convey clinically meaningfully data to inform treatment planning. Compared to teachers, caregivers tend to rate their children as more severely affected when assessing ASD symptoms (Posserud et al. 2006; Ronald et al. 2008), behavior problems (Murray et al. 2009), and comorbid psychiatric problems (Pearson et al. 2012). Detailed analysis of response patterns indicates that parents tend to endorse more polar ratings (i.e. present or absent) while teachers tend to rate skills as developing or use more moderate categories (Ryland et al. 2012; Voelker et al. 2000). These differences in response patterns may be strongly influenced by the contextual variation between informants (De Los Reyes et al. 2015). For example, teachers interact with students in highly structured settings at school whereas caregivers interact with children in a multitude of settings that vary in the level of structure and variety of environmental stimuli.

Based on this literature, there are several gaps in our understanding of multiple informant agreement of behavioral ratings for children with ASD that the current study aims to address. First, there has been limited examination of multiple informant concordance of child functioning for youth with ASD served in community mental health programs, which serve as an important point of care as children with ASD age. As mentioned earlier, the primary presenting problem with which children with ASD served in community mental health settings present is challenging behaviors (Brookman-Frazee et al. 2009, 2012b, c; Mandell et al. 2005), underscoring the need for accurate reporting of these behaviors across informants to promote appropriately tailored care. The education system is a primary referral and funding source of community mental health services for children with ASD (Brookman-Frazee et al. 2012a) further highlighting the importance of obtaining the perspectives of school providers and caregivers based on their context-specific observations of children’s behavior to inform mental health treatment targets (De Los Reyes et al. 2015). Finally, there is limited information on whether and which child clinical factors are associated with caregiver and teacher rating patterns of challenging behaviors for youth with ASD served in community mental health care.

To address these gaps in the literature, the current study aims to answer two research questions: (1) what is the extent to which caregivers and teachers similarly rate the intensity of child challenging behaviors in a sample of school-aged children with ASD receiving community mental health services? and (2) do child clinical characteristics explain differences in caregiver and teacher ratings of challenging behavior intensity? We hypothesized that agreement between caregivers and teachers would be lower than shown in the previously literature (Stratis and Lecavalier 2015) given the characteristics of the children with ASD receiving community mental health care who have complex clinical presentations (see Stadnick et al. 2016). In addition, consistent with the extant literature that has identified child characteristics associated with informant agreement, we hypothesized that child clinical characteristics such as ASD severity and psychiatric comorbidity would help to account for differences in caregiver- and teacher- report of challenging behaviors.

Method

Procedures

Data for this study were extracted from baseline assessment conducted within the context of a randomized community effectiveness trial of “An Individualized Mental Health Intervention for ASD” (AIM HI; Brookman-Frazee and Drahota 2010) that was conducted in publicly-funded community and school-based mental health settings. AIM HI was designed to address challenging behaviors, the most common presenting problems in children with ASD receiving mental health services (Brookman-Frazee et al. 2012b). Per procedures in the randomized community effectiveness trial of AIM HI, therapists were first recruited from participating mental health programs and then child-caregiver dyads were recruited from the caseloads of participating therapists. Children were eligible if they: (1) were aged 5–13 years old at the time of recruitment (2) reported English or Spanish as their primary language, (3) had an existing ASD diagnosis on record, and (4) exhibited clinically significant ASD symptoms on at least one of two standardized ASD diagnostic measures: the Autism Diagnostic Observation Schedule-2 (ADOS-2; Lord et al. 2012) or the Social Responsiveness Scale-2 (Constantino & Gruber 2012). The ADOS-2 was administered by research staff who were clinical psychologists or directly supervised by clinical psychologists. Caregivers were eligible for the community effectiveness trial of AIM HI if they: (1) were the primary caregiver of the eligible child, and (2) spoke English or Spanish as their primary language. Teachers were eligible if they were identified by the participating child’s caregiver as the child’s primary teacher for the child’s current academic year.

The specific data used for the current study were drawn from the baseline assessments of the AIM HI trial. These assessments were conducted in person with separate caregiver and child interview assessments lasting between 2 and 3 h. Families received a $40 gift card for completing the baseline assessment. After an eligible child-caregiver dyad agreed to be part of the study, the caregiver was asked to provide the contact information for the child’s primary school teacher who was sent a web-based survey about the participating child. Teachers received a $20 gift card for completing the online survey. Study procedures were approved by the institution through which this study was conducted. Informed consent was obtained from all individual participants included in the study.

Participants

A subset that included 141 of the 202 children enrolled in the larger effectiveness trial were included in the current study. To be included in the current study, two additional inclusion criteria were required: (1) the child was classified as “autism” or “ASD” on the ADOS-2; and (2) the child had both caregiver and teacher responses on the outcome measure (Eyberg Child Behavior Inventory for caregivers and Sutter-Eyberg Student Behavior Inventory-Revised for teachers) to calculate scores per scoring guidelines (Eyberg and Pincus 1999). The 61 children who were not included in the current study were excluded for the following reasons: 11 children were classified as “non-spectrum” on the ADOS-2, two children had missing classification data on the ADOS-2, 47 children had no teacher data (primarily because they were home-schooled or teachers were unable to be contacted), and one child had insufficient caregiver data on the outcome measure to calculate a score. Differences in demographic and clinical characteristics between children included in the current study (n = 141) and those excluded (n = 61) were examined using one-way analyses of variance (for continuous measures) and Chi square analyses (for categorical measures). Children in the current study did not differ significantly (p-vales >0.05) from excluded children based on age, gender, ethnicity, maternal education, family income, psychiatric comorbidity, or cognitive abilities. Due to caregiver and teacher reporting on multiple children, a total of 138 caregivers and 134 teachers completed measures about these 141 children. Children were an average of 9.07 years (SD = 2.36; Range 4–14 years), 84% (n = 118) male, and 56% were Hispanic (n = 79). See Table 1 for more details about child demographic and clinical information. Caregivers were an average of 40.28 years old (SD = 8.09), 92% were female (n = 127), and 88% were the child’s biological or adoptive mother (n = 121), 7% were the child’s biological father (n = 10), and 5% were relatives or other caregivers (n = 7).

Table 1.

Child characteristics (n = 141)

Child characteristic	M (SD) or n (%)
Gender (male)	118 (84%)
Age (years)	9.07 (SD = 2.36)
Ethnicity: Hispanic/Latino	79 (56%)
Race
White	104 (75%)
African American	12 (9%)
Asian/Pacific Islander	9 (7%)
Multiracial	7 (5%)
American Indian/Alaskan Native	6 (4%)
Cognitive standard score (derived from WASI-II or DAS-II)	88.29 (16.06)
ADOS-2 classification
Autism	120 (85%)
ASD	21 (15%)
ADOS-2 comparison score (1–10)	7.32 (1.68)
MINI-KID-P diagnoses (number for which criteria met)	2.61 (1.71)
Educational placement
General education	64 (46%)
Special day classroom	56 (40%)
Other classroom type	19 (14%)

Teachers were 84% female (n = 112), 84% White (n = 113) and an average age of 45.04 years (SD = 10.83; Range 23–71). In terms of education, 64% had a Master’s degree (n = 85), 31% had a teaching credential (n = 41), 3% (n = 4) had a bachelor’s degree, and 2% (n = 2) reported their education as “Other.” Regarding classroom type, 47% of teachers reported teaching in a general education setting (n = 63), 39% reported teaching in a special day classroom (n = 51), and 14% reported teaching in an “Other” classroom type (n = 18). On average, teachers reported that they had been teaching for 15.51 years (SD = 8.98; Range 1–37 years) and had 19.97 students in their classroom (SD = 8.93; Range 4–38 students). Teachers reported that they knew the child about whom they were reporting at least “somewhat” (48%; n = 63) to “very well” (39%; n = 52). For 67% of children (n = 89), teachers reported that the child spent greater than 75% of their school time in the reporting teacher’s classroom.

Measures

Outcome Measures: Report of Child Challenging Behaviors

Eyberg Child Behavior Inventory (ECBI; Eyberg and Pincus 1999)

The ECBI is a 36-item caregiver-report measure that assesses the frequency and intensity of child disruptive behaviors. The ECBI has strong test-rest reliability (reliability coefficient of 0.86 for the Intensity score) and good construct and concurrent validity (Boggs et al. 1990; Eyberg and Ross 1978; Robinson et al. 1980). Two scores are yielded: an Intensity score that represents the current frequency of disruptive behaviors and it is rated on a 7-point Likert scale, and a Problem score that represents the total number of child behaviors that caregivers endorsed as currently being a problem for them. For this study, the Intensity t-score (M = 50; SD = 10) was used to characterize the severity of the child’s behavior problems. Internal consistency was strong in this study’s sample (α= 93).

Sutter-Eyberg Student Behavior Inventory-Revised (SESBI-R; Eyberg and Pincus 1999)

The SESBI-R is the corresponding teacher-report version of the ECBI. The SESBI-R has strong test–retest reliability (test-rest correlations reported between 0.89 and 0.98) (Funderburk and Eyberg 1989; Ladish et al. 1989) and established convergent and discriminant validity (Dumas 1992; Funderburk and Eyberg 1989; Schaughency et al. 1990). Consistent with the ECBI, the Intensity t-score (M = 50; SD = 10) was used for this study. Internal consistency was similarly strong in this study’s sample (α = 0.97).

Measures of Child Characteristics

Child Demographics

Caregiver-reported child gender, age, race/ethnicity, and educational placement at the time of the baseline assessment were included.

One of two cognitive assessments was administered to characterize children’s global cognitive abilities based primarily on their chronological age. Six trained research staff administered the cognitive assessments to estimate the child’s full scale IQ.

Wechsler Abbreviated Scale of Intelligence-II (WASI-II; Wechsler 2011)

The WASI-II is a brief standardized assessment of cognitive ability that was administered by a trained member of the research team. Four subtests are administered to yield a FSIQ that is represented as a standard score (M = 100; SD = 15). The WASI-II has strong internal consistency, with the average reliability coefficients ranging from 0.87 to 0.91 for children, and good convergent and discriminant validity (Wechsler 2011). The WASI-II was administered to children age 6 years and over at the time of the baseline assessment.

Differential Ability Scale-II (DAS-II; Elliott 2007)

The DAS-II is a comprehensive assessment of cognitive ability that was administered by a trained member of the research team. A General Conceptual Ability (GCA) score and three core composites (Nonverbal Reasoning Ability, Verbal Ability, and Spatial Ability) are produced. The GCA and three core composites are represented as standard scores (M = 100; SD = 15). The DAS-II has established psychometric characteristics with strong internal consistency for both the standardization sample and special clinical populations and strong support for convergent and discriminant validity (Elliott 2007). The DAS-II was administered to children younger than 6 years at the time of the baseline assessment.

Measures of Predictor Variables

Educational Placement

Teachers reported on the child’s educational placement (general education, special day, or another classroom type) at the time of the baseline assessment.

Mini-International Neuropsychiatric Interview, Parent Version (Sheehan et al. 1998)

The Mini-International Neuropsychiatric Interview, parent version (MINI-KID-P; Sheehan et al. 1998) was used to determine the presence of co-occurring psychiatric disorders. The MINI-KID-P is a structured diagnostic interview to assess symptoms of Axis I disorders as listed in the DSM-IV and ICD-10 (World Health Organization 1993). It has strong interrater and test–retest reliability and established construct validity. Sensitivity (0.61–1.00) and specificity (0.81–1.00) are also strong (Sheehan et al. 2010). The MINI-KID-P was administered in person or via phone to the caregiver identified as knowing the child well by a trained member of the research staff. The following twelve MINI-KID-P modules were used based on the most common psychiatric comorbid disorders for children with ASD in MH settings (Brookman-Frazee et al. 2010, 2012b; Joshi et al. 2010): ADHD, ODD, Panic Disorder, Agoraphobia, Separation Anxiety Disorder, Social Phobia, Specific Phobia, OCD, GAD, Tic Disorders, Major Depressive Episode, Manic and Hypomanic Episodes.

The MINI-KID-P was adapted for an ASD sample by adding follow-up probes to aid in differentiation between ASD symptoms and other symptoms of other psychiatric disorders. For example, prior to asking about anxiety, caregivers were provided a brief explanation of the physiological symptoms associated with anxiety to reduce over-endorsement based on similar behaviors that may be observed in ASD. For most modules, caregivers were asked to provide examples following endorsement of screening items, skip patterns were removed, and additional items were added to the OCD and Tic modules to assist in distinguishing between restrictive, repetitive behaviors (characteristic of ASD) and compulsions or tics (required for OCD and Tic diagnoses). Last, questions in the Dysthymia module were embedded within the Major Depressive Episode module to reduce caregiver burden. All interviewers were trained to criterion prior to administering the MINI-KID-P by a study investigator who is a licensed clinical psychologist with clinical expertise in child mental health and ASD diagnostic assessment. Six trained study personnel administered the MINI-KID-P to caregivers as part of the baseline assessments conducted as part of the larger effectiveness trial. Please see Stadnick et al. (2016) for specific details about MINI-KID-P training and quality monitoring procedures. For the current study, the number of non-ASD psychiatric disorders for which the child met criteria on the MINI-KID-P was used as the measure of psychiatric comorbidity.

Autism Diagnostic Observation Schedule-2 (ADOS-2; Lord et al. 2012)

The ADOS-2 is a semi-structured observational assessment administered by a trained provider to assist in the diagnosis of ASD. The quality of the child’s social affect, communication, and restricted, repetitive behaviors are rated. An algorithm is applied to the scores and result in a classification of “Autism,” “Autism Spectrum Disorder,” or “Non-Spectrum” based on standardized cut-off values. The Overall Total score and the child’s chronological age are used to identify the ADOS-2 Comparison Score that ranges from 1 (Minimal-To-No-Evidence of ASD-related symptoms) to 10 (High level of ASD-related symptoms). The ADOS-2 has strong reliability and validity across modules (Lord et al. 2012). Children were administered one of the modules based on their language and developmental level. All ADOS-2 administrators were research reliable. For this study the ADOS-2 Comparison Score was used to characterize ASD severity.

Results

The data analytic approach and results are described by study aim. The outcome of interest (continuous variable) across all analyses was the Intensity t-score of the Eyberg Child Behavior Inventory (caregivers) and the Sutter-Eyberg Student Behavior Inventory-R (teachers).

Characterizing Concordance between Caregiver and Teacher Ratings of the Intensity of Child Challenging Behaviors

The concordance between caregiver and teacher ratings of child problem behavior severity was examined in two ways. First, a one-way ANOVA was performed to determine mean differences between caregiver and teacher ratings. Second, the correlation between caregiver and teacher Intensity t-scores was calculated to examine the extent of agreement between the two informants.

On average, caregiver scores (M = 62.62, SD = 10.30) were significantly higher than teacher scores (M = 56.21, SD = 8.81) with average caregiver t-scores in the clinical range (i.e. greater than 60) and average teacher scores in the normal range, (F (1,280) = 31.62, p < .001). The correlation between caregiver scores and teacher scores was low but statistically significant, r = .27, p < .01. See Fig. 1 for a graphical depiction of caregiver and teacher ratings.

Fig. 1.

Mean differences between caregiver and teacher ratings of child challenging behaviors. *p <.001

Child Clinical Characteristics that Moderate the Association between Caregiver and Teacher Ratings of Child Challenging Behavior Intensity

To examine potential child characteristics that may explain differences between caregiver- and teacher- report, a mixed-model ANOVA was performed with informant (within-subjects) and child clinical factors (between-subjects). The child factors that were included were: ASD severity (as measured by the comparison score from the ADOS-2), educational placement (general education versus special day classroom and versus other classroom type), and diagnostic comorbidity (as measured by the number of non-ASD psychiatric disorders for which the child met criteria on the MINI-KID-P). Child age, gender, and ethnicity (Hispanic versus non-Hispanic) were entered as covariates.

After controlling for child age, gender and ethnicity, there were statistically significant main effects for respondent, F (1, 249) = 37.83, p < .001, and the number of non-ASD diagnoses for which the child met criteria on the MINI-KID-P, F (8, 249) = 4.76, p < .001). There were no significant main effects of educational placement and ADOS-2 severity. See Table 2. To probe the main effects of respondent and number of comorbid diagnoses on differences in teacher and caregiver ratings of child behavior problems, a second model was performed with the interaction term between respondent and the number of MINI-KID-P diagnoses for which a child met criteria. After controlling for child age, gender, and ethnicity, results indicated a significant interaction between respondent type and the number of MINI-KID-P diagnoses for which a child met criteria, F (17,259) = 5.54, p < .001. See Table 3 for model details.

Table 2.

Differences in challenging behavior intensity ratings by respondent and child characteristics

Model terms	F
Intercept	385.73**
Child age	19.79**
Child gender	0.14
Child ethnicity	4.32*
Respondent	46.83**
Educational placement (general education = reference group)
Special day classroom	2.38
Other classroom type	0.88
ADOS-2 comparison	1.57
MINI-KID-P count	4.17**

ADOS-2 Autism Diagnostic Observation Schedule-2, MINI-KID-P mini-international neuropsychiatric interview-parent interview

^*p < .05,

^**p < .001

Table 3.

Interaction between respondent and child psychiatric comorbidity on challenging behavior ratings

Model terms	F
Intercept	522.67***
Child age	18.73***
Child gender	0.69
Child ethnicity	2.49
Respondent * MINI-KID-P Count	5.55***

MINI-KID-P mini-international neuropsychiatric interview-parent interview

^***p < .001

Exploring Number of Co-morbid Psychiatric Conditions

Simple effects analyses of this significant interaction were then conducted. To aid in interpretation, we used the average number of number of diagnoses for which a child met criteria on the MINI-KID-P (M = 2.66) to categorize psychiatric comorbidity into the following: 0 diagnoses, 1–2 diagnoses, and more than 2 diagnoses. These analyses indicated that caregivers rated the intensity of their child’s problem behaviors significantly higher than teachers as the child met criteria for more MINI-KID-P diagnoses, for pairwise comparisons of 1–2 diagnoses (F (1, 271) = 10.71, p < .01) and more than 2 diagnoses (F (1, 271) = 28.78, p < .001). There were no significant differences between caregiver and teacher ratings for children with no diagnoses identified on the MINI-KID-P F (1, 271) = 0.17, p = .68. This signifies that caregivers and teachers rated children equivalently when the child only had an ASD diagnosis but once children demonstrated psychiatric comorbidity (i.e., ASD plus an additional diagnosis), caregivers rated children’s behavior problems significantly higher than teachers. See Fig. 2 for visual display of the simple effects.

Fig. 2.

Differences in informant ratings of child challenging behaviors as a function of child psychiatric comorbidity. *p <.01, **p<.001

Exploring Type of Co-morbid Psychiatric Conditions

Exploratory follow-up analyses were conducted to better understand the impact of psychiatric comorbidity on observed differences in caregiver and teacher ratings. Specifically, we were specifically interested in whether the type of comorbid psychiatric diagnosis might help explain differences in caregiver and teacher ratings of behavior problems. MINI-KID-P diagnoses were dichotomized into two variables: externalizing (included any MINI-KID-P diagnosis of ADHD or ODD) and internalizing (included any MINI-KID-P anxiety or mood diagnosis). A mixed-model ANOVA was performed with respondent (caregiver vs teacher) entered as the within-subjects variable and the externalizing and internalizing variables entered as between-subjects variables. After accounting for child gender, age, and ethnicity, there were no main effects of externalizing and internalizing disorders but there was a significant interaction between respondent and presence of a MINI-KID-P externalizing disorder, F(1, 269) = 28.22, p < .001. There was no significant interaction between the presence of a MINI-KID-P internalizing disorder and respondent. This indicates that caregiver and teacher ratings of child behavior problems differed significantly based on whether the child had a comorbid externalizing disorder identified on the MINI-KID-P. Simple effects analyses of this significant interaction were performed. Results indicated that caregivers rated the intensity of behavior problems significantly higher (EMM = 65.05, SE = 0.82) when a child had an identified externalizing diagnosis on the MINI-KID-P compared to teachers (EMM = 57.60, SE = 0.82), F (1, 269) = 41.46, p < .001. However, caregiver and teacher ratings were not significantly different when the child did not have an externalizing disorder identified on the MINI-KID-P, F (1, 269) = 0.49, p = .49. See Fig. 3 for a graphical presentation of these findings.

Fig. 3.

Mean differences between caregiver and teacher ratings of child challenging behaviors as a function of type of psychiatric comorbidity. *p <.001

Discussion

Consistent with study hypotheses, caregiver-teacher agreement of the intensity of child challenging behaviors was low in this sample of children with ASD receiving community mental health services with a smaller effect size reported in the extant literature (Stratis and Lecavalier 2015). Overall, caregivers rated child challenging behaviors as significantly more intense than teachers. In addition, differences in caregiver and teacher ratings was associated with the degree and type of child psychiatric comorbidity after controlling for child age, gender, and ethnicity. Specifically, caregivers and teachers rated children similarly when the child met criteria for fewer comorbid psychiatric diagnoses whereas caregivers endorsed significantly greater challenging behavior intensity than teachers when children had a greater number of psychiatric diagnoses. No other child clinical characteristics such as severity of ASD symptoms or education placement were associated with caregiver and teacher agreement of child behavior problems.

A potential explanation for these findings is that caregivers and teacher may have distinct “decision thresholds” to rating child behaviors as problematic (De Los Reyes 2013). This is, in part, because each informant is rating the child’s behavior both in different contexts (e.g., home versus school) and compared to a reference group that may differ in size and composition (e.g., the caregiver’s other children versus the teacher’s classroom of students) (De Los Reyes et al. 2015). With an average of 16 years of teaching experience, teachers in our sample had likely worked with a large number of children with varying classroom presentations. Related, Dirks and colleagues (2012) suggested that the extent of cross-informant concordance may be an important indicator of the pervasiveness of behavioral problems across settings. It is not known from our data how pervasive the behavior problems were in our sample of children and whether that impacted the ratings of informants.

Our finding that psychiatric comorbidity was associated with differences in informant ratings is novel but not surprising given the context of the sample. Specifically, there are strong evidence for the high rates of psychiatric comorbidity that children with ASD experience (e.g., Olsson et al. 2016; Simonoff et al. 2008), particularly those served in community mental health settings (Stadnick et al. 2016). Within this sample, children met criteria for an average of nearly three non-ASD psychiatric disorders further demonstrating high rates of psychiatric comorbidity within the ASD child population. Although caregivers and teachers did not differ on their perceptions of the intensity of child behavior problems when a child presented with no comorbid diagnosis (only ASD), when a child met criteria for one or more comorbid disorders, agreement dropped with caregivers reporting greater intensity of behavior problems. Simply stated, there was greater disagreement between caregivers and teachers for children with greater complexity of clinical symptoms.

This finding is somewhat discrepant from the findings from Azad and colleagues (2015) who reported greater caregiver-teacher agreement of social deficits for more severely affected children with ASD. However, this is not surprising as the presence of observable social deficits is required diagnostic criteria for ASD, underscoring the likelihood of agreement between raters when assessing social skills while challenging behaviors may be harder to identify or present differently at home and school. Teachers may be less involved in the child’s mental health treatment and thus less aware of the child’s co-occurring mental health diagnoses to impact their ratings of the child’s behavior. It is also possible that the structure and routine of the school setting may contribute to a reduction (actual or perceived) in child behaviors in this specific setting. Related, it may be the case that schools have greater or more specific behavioral supports in place for children with ASD to mitigate the intensity of challenging behaviors within the school environment. Therefore, for caregivers it appears that the more severely impaired the child is in terms of psychiatric comorbidities, the higher they rate the intensity of their child’s behaviors. This pattern is not the same for teachers. In fact, teacher ratings, on average, never surpassed clinically significant levels (i.e., t-scores greater than 60) for any degree or type of comorbid child mental health diagnoses. However, teacher ratings descriptively increased and approached the clinical range when the child had at least one or more additional diagnoses and if at least one of those diagnoses was an externalizing disorder identified on the MINI-KID-P. These informant-related discrepancies between caregivers and teachers highlight the clinical and pragmatic significance of obtaining reports from multiple informants (De Los Reyes et al. 2013), particularly regarding challenging behaviors to inform treatment planning decisions including the settings and/or additional providers to consider including in mental health treatment delivery for children with ASD.

This study has many notable strengths and some limitations. First, this study represents one of the few to evaluate cross-informant concordance for children with ASD who are receiving community mental health services, an important treatment access point for this clinical population. In addition, our focus on challenging behaviors is unique and clinically essential to appropriately assess and incorporate into treatment planning given that children with ASD exhibit extremely high rates of challenging behaviors that may heighten their risk for greater educational supports and therapeutic intervention (Horner et al. 2002; Joshi et al. 2010; Matson et al. 2009). Therefore, our sample is representative of a significant portion of children with ASD and findings may have strong external validity. Related, few studies have examined the role of psychiatric comorbidity, a common presentation in children with ASD, in cross-informant agreement of challenging behaviors for youth with ASD. Limitations of this study include lack of a comparison group of children without ASD receiving community mental health services to compare our patterns of caregiver and teacher concordance on ratings of challenging behaviors. In addition, findings may be impacted by shared informant variance because caregivers reported on both child psychiatric comorbidity (a predictor variable) and behavior problems (a criterion variable). This methodological limitation highlights the need for future research to include observer-report of child functioning to provide additional confirmation of study findings.

Overall, results of this study provide support for the importance of cross-informant measurement of child behavior problems for children with ASD. It is clear from our findings that caregivers and teachers perceive the intensity of child behavior problems differently, particularly for children with more significant psychiatric comorbidities. This has important implications for mental health therapists who must synthesize data from multiple informants including caregivers and other providers to prioritize the structure, content, and selection of treatment targets. These data also highlight the importance of attending to the child’s non-ASD mental health diagnoses and the impact they may have on the child’s behavior problems, which are likely to be a primary presenting problem for children with ASD served in community mental health care.