Predictors of Informant Discrepancies Between Mother and Middle School Teacher ADHD Ratings

Abstract

A diagnosis of attention-deficit/hyperactivity disorder (ADHD) according to the Diagnostic and Statistical Manual, 5th Eldition (DSM-5) is assessed in youth using ratings from both a parent and a teacher. However, individual and contextual differences between informants may lead to discrepancies in these ratings. The purpose of this study was to examine predictors of discrepancies between mother and middle school teacher reports of ADHD symptoms and related impairment. In an ethnically diverse sample of middle school students with well-diagnosed DSM-IV-TR ADHD (N = 112), we examined a range of mother and school setting characteristics that may contribute to informant discrepancies in this population. Hierarchical multiple regression analyses suggested that mothers with higher levels of education and psychopathology (i.e., ADHD symptom severity, parenting stress) may be most likely to report adolescent ADHD symptom severity that is higher than reported by teachers. Reports from general education teachers (vs. special education) were associated with lower symptom severity compared to mothers. Finally, a documented diagnosis of ADHD in the school was predictive of more severe reports from mothers. We discuss explanations for these findings and implications for assessment of middle school students with ADHD.

Introduction

A DSM-5 diagnosis of attention-deficit/hyperactivity disorder (ADHD) requires the presence of clinically significant symptoms of inattention and/or hyperactivity/impulsivity (H/I) in two or more settings (APA, 2013). To establish this criterion, clinicians consult parameters recommending collection of symptom and impairment reports from both a teacher and a parent (American Academy of Child & Adolescent Psychiatry, 2007; American Academy of Pediatrics, 2011). Informant, rather than self, reports are recommended because young individuals with ADHD underreport symptoms to such a degree that the presence of the disorder may be masked (Fischer, Barkley, Fletcher & Smallish, 1993; Sibley et al., 2012). However, when multiple informant reports are integrated in diagnosis, these reports may yield discrepant characterizations of youth (Achenbach, McConaughey, & Howell, 1987; De Los Reyes & Kazdin, 2005; De Los Reyes et al., 2015). It is well established that discrepancies between informants arise due to systematic factors (De Los Reyes, 2011). However, it can be unclear whether these discrepancies represent true situational differences in symptom expression or are an artifact of informant perceptual differences (De Los Reyes & Kazdin, 2005; De Los Reyes et al., 2015).

Informant discrepancies in the assessment of children and adolescents are commonly observed (De Los Reyes & Kazdin, 2005; De Los Reyes et al., 2015), and there is no gold-standard measure of a youth's behavior. Rather, values measured from behavior rating scales or diagnostic interviews represent an individual reporter's perception of behavior severity given their familiarity with an adolescent, the contexts in which they observe behaviors, and the purpose of assessments (De Los Reyes & Kazdin, 2005; De Los Reyes, 2011). Therefore, both sides of a discrepant report may be valid and can provide additional contextual information that is valuable for diagnostic decision-making (Kraemer et al., 2003; Hunsley & Mash, 2007; Dirks et al., 2012). For example, a classroom teacher may report higher levels of academic impairment than a parent when the student fails to complete in-class work but is thorough during homework time.

Across multiple domains, there is especially poor agreement between parent and teacher symptom reports for adolescents with ADHD (Fischer et al., 1993; Sibley et al., 2012; Sibley, Altszuler, Morrow, & Merrill, 2014). There are several population-specific reasons why informant discrepancies may be particularly pronounced for these youth. Compared with school age children, adolescents spend less time with and disclose less information to parents (Collins & Steinberg, 2006), thereby limiting parent familiarity with daily adolescent behavior. At the same time, middle school teachers teach over 100 adolescents per day and typically spend less than an hour with each student daily (Eccles, 2004), compromising their familiarity with individual students. Given the reduced time spent with students, it is unsurprising that middle school teachers exhibit poor agreement on ADHD ratings when compared with ratings from other teachers as well as observational data (Evans, Allen, Moore, & Strauss 2005).

Additionally, the correlates of an ADHD diagnosis may also increase risk for parent–teacher reporting discrepancies. First, discrepancies in the child psychopathology literature are predicted by parenting stress and depression (De Los Reyes & Kazdin, 2005; Richters, 1992), which occur at an increased risk among parents of youth with ADHD (Chronis et al., 2003; Johnston & Mash, 2001). These findings have been extended to the child ADHD literature (Chi & Hinshaw, 2002; Langeberg et al., 2010) but have not been yet been examined in adolescence. Second, there is some evidence that parent ADHD symptoms, which occur with greater frequency when a child has ADHD, may increase a parent's sensitivity to their child's behavior, possibly leading to inflated ratings (Chronis-Tuscano et al., 2008), though findings on this point are mixed (Faraone et al., 2003). In addition, parent social adversity (i.e., single parenthood, poor education level) is associated both with ADHD (Sauver et al., 2004; Wymbs, Pelham, Gnagy, & Molina, 2008) and informant discrepancies (De Los Reyes & Kazdin, 2005).

Within the school setting, teachers may rate youth with overt behavior problems as possessing inflated levels of unrelated symptoms and impairments (Abikoff, Courtney, Pelham & Koplewicz, 1993). This problem may be compounded when the student possesses a documented diagnosis at school (Ohan, Visser, Strain, & Allen, 2011), which is true for nearly half of students with ADHD (Barkley, Fischer, Smallish, & Fletcher, 2006). Finally, increased knowledge about ADHD (e.g., special education training) and related impairments has been associated with increased recommendations for treatment and elevated endorsement in the belief that students with ADHD will be more disruptive and thus more problematic to teach (Ohan et al., 2008). These findings suggest that students with ADHD may be susceptible to teacher biases by virtue of being identified as having a diagnosis.

The findings above suggest that informants tasked with rating adolescents with ADHD are at particular risk for displaying biased or inaccurate reports of functioning—a critical concern given that reports from parents and secondary school teachers are recommended for thorough diagnosis (American Academy of Child & Adolescent Psychiatry, 2007; American Academy of Pediatrics, 2011). Surprisingly, no work examines factors that predict parent–teacher disagreement on adolescent ADHD symptom severity and related impairments. In the current study, mothers and teachers of 112 middle school students with ADHD completed identical ADHD symptom and impairment measures at baseline intake into a randomized clinical trial (Sibley et al., 2016). We examined mother characteristics and school setting variables that predicted informant discrepancies. Our hypotheses were that: (1) maternal demographic and psychological variables (i.e., single parent status, maternal education level, depression, ADHD, parenting stress) would predict more severe reports from mothers and (2) school setting variables (i.e., a documented ADHD diagnosis by the school district, being rated by a special education teacher) would predict more severe reports from teachers, leading to more pronounced discrepancies.

Methods

Participants

Participants were 112 middle school students with ADHD between the ages of 11 and 15 (M = 12.78, SD = .86) who had a female primary caregiver. Participants were initially recruited into a federally funded randomized controlled trial of psychosocial treatment for adolescents with ADHD (N = 128) at a large urban university research clinic between 2011 and 2013. Sixteen adolescents were excluded from the current study because their primary caregiver was male. In order to participate, adolescents were required to: (a) meet DSM-IV-TR diagnostic criteria for ADHD (APA, 2000), (b) be enrolled in the sixth, seventh, or eighth grade, (c) have an estimated IQ of 80 or higher, and (d) have no history of an autism spectrum disorder. Table 1 provides demographic and clinical characteristics of the sample.

Table 1.

Demographic and diagnostic characteristics of the sample

Demographic
Age M (SD)	12.78 (.86)
Sex (%)
Male	66.1
Female	33.9
Race/ethnicity (%)
Non-Hispanic White	9.2
Hispanic any race	77.1
Black	8.3
Asian	0.9
Mixed race	4.6
Highest maternal education level
High school or less	19.8
Some college or technical training	21.6
Bachelor's degree	37.8
Master's degree or higher	20.7
Single parent household (%)	34.8
Diagnostic
Estimated Full Scale IQ M (SD)	101.14 (12.53)
Reading achievement standard score M (SD)	104.88 (9.62)
Math achievement standard score M (SD)	99.90 (16.15)
DSM-IV-TR ADHD diagnosis (%)
ADHD-PI	38.4
ADHD-C	61.6
LD (%)	9.2
ODD (%)	44.6
CD (%)	12.5
Current ADHD medication (%)	37.5

Overall, participants attended 114 different schools (94.5 % public) in the fourth largest school district in the USA, which comprises 392 schools, over 350,000 students, and over 40,000 employees. The school district covers over 2000 square miles including rural, suburban, and urban neighborhoods. The school district is the second most ethnically diverse in the USA, with students speaking 56 different languages at home and representing 160 countries of origin. The district reports that 70.2 % of students receive free or reduced priced lunch (Miami-Dade County Public Schools, 2015). Annual per pupil expenditure in the local school district ($12,298; Miami-Dade County Public Schools, 2015 for peer review) is very similar to the national average for the same year ($12,608; U.S. Department of Education, 2014).

Procedures

Participants were recruited through direct school mailings and parent inquiries at the university clinic. For all potential participants, the primary caretaker was administered a brief phone screen containing the DSM-IV-TR ADHD symptoms and questions about functional impairment. At intake assessment, informed parental consent and youth assent were obtained and study eligibility was assessed. During the assessment, ADHD diagnosis was assessed through a combination of parent structured interview (Computerized Diagnostic Interview Schedule for Children) and parent and teacher rating scales (Shaffer, Fischer, Lucas, Dulcan, & Schwab-Stone, 2000; Pelham, Gnagy, Greenslade, & Milich, 1992) based on the standard and recommended practice in the field (Pelham, Fabiano, & Massetti, 2005). Parents and teachers were instructed to rate students’ behavior while unmedicated. Additionally, a clinician administered a brief intelligence test (Wechsler Abbreviated Scale of Intelligence, first and second editions; Wechsler, 1999, 2011a), achievement testing (Wechsler Individual Achievement Test, second and third editions; Wechsler, 2002, 2011b), and a standard rating scale battery that included the measures utilized in the current study. Cross-situational impairment was assessed for the purpose of ADHD diagnosis by examining parent and teacher impairment ratings and school grades obtained from official report cards. Dual clinician review was conducted by doctoral level psychologists to determine diagnosis and study eligibility. Sibley et al. (2016), provides more detailed information about screening and assessment procedures.

Measures

Student Symptoms and Impairment

Mother and teacher reports of the students’ DSM-IV-TR ADHD symptoms were obtained using the Disruptive Behavior Disorders Rating Scale (DBD; Pelham et al., 1992). The DBD lists DSM-IV-TR symptoms of ADHD. Mothers and teachers provided ratings for each symptom using a 4-point Likert scale. The psychometric properties of the DBD rating scale are very good in both child and adolescent samples (Evans et al., 2013; Pelham et al., 1992). Overall Inattention and H/I severity scores were calculated by averaging subscale responses. Internal consistency estimates for mother and teacher ratings were 91–.92 and 89–.93, respectively, for Inattention and H/I. To measure functional impairment, the 24-item Adolescent Academic Problems Checklist (AAPC) was administered to mothers and teachers (Sibley et al., 2014). The AAPC measures adolescent use of secondary-school-specific organization, time management, and planning skills and is validated for samples of adolescents with ADHD (Sibley et al., 2014). The AAPC reliably detects effects in evaluations of psychosocial treatment (Sibley et al., 2013) for adolescents with ADHD. Alphas for the total score in this study ranged from .90 for parents to .94 for teachers.

Maternal Characteristics

Mothers completed the Symptom Checklist-90-Revised (SCL-90-R; Derogatis, 1996), a 90-item broadband scale of adult psychopathology that measures nine symptom domains using a 5-point Likert scale. The SCL-90-R has good internal consistency for each subscale and possesses convergent, discriminant, and predictive validity (Derogatis & Cleary, 1977). T-scores from the depression subscale (DEP) served as a maternal depression index. Internal consistency for this scale in our study was .89. The Adult ADHD Self-Report Scale (ASRS; Adler et al., 2006) was used to measure maternal ADHD severity. Eighteen adult-specific ADHD symptoms are rated on a five-point scale (0 = never to 4 = very often). The ASRS self-report rating scale correlates highly with clinician ratings of ADHD and displays strong internal consistency (Adler et al., 2006). ADHD severity was calculated for each mother by calculating the mean score of the ASRS items. Internal consistency for this scale was .93 in our study. The 11-item objective strain subscale on the Caregiver Strain Questionnaire was used as a measure of parenting stress (CSQ; Brannan, Heflinger, & Bickman, 1997). This scale measures observable negative events or consequences that have affected mothers throughout the past month (e.g., financial hardship, loss of personal time). Internal consistency for this scale in this study was .90. The CSQ uses a 5-point scale ranging from 1 = not at all to 5 = very much a problem, shows strong internal consistency, and correlates well with other measures of family functioning (Branna et al., 1997). Mothers reported their marital status and education level on standard forms as a part of intake procedures. Dual parent household was coded if the parent indicated that they were married or living with a partner. Education level open-ended responses were coded on an ordinal scale that ranged from (1 = high school or less to 4 = graduate degree).

School Setting Characteristics

IEP and Sect. 504 documents were directly obtained from mothers and schools at baseline. Teachers reported the classroom setting in which they instructed the adolescent student (special education or mainstream) as a part of ratings completion.

Analytic Plan

To calculate discrepancies in Inattention, H/I, and academic impairment severity, we computed a standardized difference score between mother and teacher ratings (z score of mother rating minus z score of teacher rating) based on the recommendation of De Los Reyes and Kazdin (2004). Positive difference scores reflected maternal report that was more severe than teacher report. To examine rater agreement, we calculated Pearson's bivariate correlations for each outcome. Three multiple regression analyses separately modeled discrepancies in Inattention, H/I, and academic impairment ratings. Mother level predictors were dual parent household (0 = no, 1 = yes), education level, depression, maternal ADHD, and parenting stress. School setting predictors were class placement (0 = general or advanced, 1 = special education) and documented ADHD diagnosis at school (0 = no, 1 = yes). Predictors were entered hierarchically with mother characteristics in block 1 and school setting characteristics in block 2.

Prior to these analyses, Pearson's bivariate correlations were calculated for each variable.

Results

Parent–Teacher Agreement

On average, mothers rated Inattention, H/I, and academic impairment more severely than teachers. Results indicated modest yet significant correlations between mother and teacher reports of H/I and academic impairment, but no significant association between mother and teacher ratings of Inattention (see Table 2).

Table 2.

Mother–teacher agreement on middle school student ADHD symptoms and academic impairment

Symptoms	Mother M (SD)	Teacher M (SD)	Pearson r	p	ICC	p
Inattention	2.16 (.63)	1.70 (.77)	.15	.11	.26	.06
Hyperactivity/impulsivity	1.20 (.77)	.77 (.85)	.40	<.01	.57	<.01
Academic impairment	1.80 (.46)	1.48 (.64)	.46	<.01	.61	<.01

Predictors of Mother–Teacher Symptom and Impairment Reports

Bivariate correlations for each variable are presented in Table 3. For Inattention, the mother characteristic block was significant [R² = .16, F(5,102) = 3.99, p < .01]. The incremental change from the contribution of school setting was nonsignificant [R²Δ = .01, F(2,100) = .43, p = .65]. Interpretation of step 1 results indicated that higher maternal education (b = .22, SE = .09, p = .01) predicted maternal reports that were more severe than teacher reports. Higher parenting stress (b = .19, SE = .10, p = .06) also appeared to be marginally significant in predicting more severe maternal report. For H/I, the mother characteristic block was significant [R² = .15, F(5, 102) = 3.59, p < .01], as was the incremental change from the school setting block [R²Δ = .08, F(2,100) = 5.48, p < .01]. Higher maternal ADHD (b = .34, SE = .14, p = .02), higher parenting stress (b = .27, SE = .11, p = .02), and a documented ADHD diagnosis (b = .41, SE = .20, p = .04) predicted mother reports more severe than teacher reports. Teaching in a special education classroom (b = −.86, SE = .28, p < .01) predicted teacher reports that were more severe than maternal reports. Regarding academic impairment, the mother characteristic block was significant [R² = .12, F(5,98) = 2.58, p = .03]. However, the incremental change from the contribution of school setting was not [R²Δ = .02, F(2,96) = .83, p = .44]. In this model, higher parenting stress predicted mother reports that were more severe than teacher reports (b = .37, SE = .12, p < .01). See Table 4 for more details regarding results from linear regression models.

Table 3.

Intercorrelations among predictor variables

Variable	1	2	3	4	5	6	7
1. Maternal marital status	–	.16	–.04	.24*	.09	.24**	.10
2. Maternal education		–	–.01	.21*	.18	.07	<.01
3. Maternal depression			–	.08	–.07	.01	–.07
4. Maternal ADHD				–	.19	.02	.08
5. Parenting stress					–	.01	.08
6. Classroom documented ADHD (Y/N)						–	.32**
7. Classroom type (special or general)							–

^*p < .05

^**p < .01

Table 4.

Hierarchical regression analyses for predictors of mother–teacher discrepancies of adolescent ADHD symptoms

	R 2 Δ	F Δ	p Δ	b	SE	β	p
Inattentiona
Step 1: Mother characteristics	.16	3.99	<.01
Step 2: School setting characteristics	.01	.43	.65
Maternal marital status				–.07	.19	–.04	.71
Maternal education				.22	.09	.24	.01
Maternal depression				<.01	<.01	.08	.40
Maternal ADHD				.21	.12	.17	.09
Parenting stress				.19	.10	.18	.06
Hyperactivity/impulsivityb
Step 1: Mother characteristics	.15	3.59	<.01
Step 2: School setting characteristics	.08	5.48	<.01
Maternal marital status				–.15	.21	–.07	.46
Maternal education				.15	.10	.15	.11
Maternal depression				<.01	<.01	–.06	.51
Maternal ADHD				.34	.14	.23	.02
Parenting stress				.27	.11	.22	.02
Classroom documented ADHD (Y/N)				.41	.20	.19	.04
Classroom type (special or general)				–.86	.28	–.29	<.01
Academic impairmentc
Step 1: Mother characteristics	.12	2.58	.03
Step 2: School setting characteristics	.02	.83	.44
Maternal marital status				–.09	.22	–.04	.69
Maternal education				–.03	.10	–.03	.75
Maternal depression				<.01	<.01	–.04	.70
Maternal ADHD				.18	.14	.13	.21
Parenting stress				.37	.12	.30	<.01

b unstandardized beta, SE standard error, β standardized beta, p significance

^aN = 102

^bN = 100

^cN = 98

Discussion

The present study examined key predictors of mother and middle school teacher rating discrepancies in a sample of middle school students with ADHD. Maternal education level, parenting stress, and maternal ADHD predicted elevated mother ratings relative to teacher report. Additionally, teachers who instructed in special education settings reported higher levels of symptoms relative to maternal report, while having a documented ADHD diagnosis at school predicted more severe reports by mothers compared to teachers. We discuss these findings below.

Our findings are consistent with previous work suggesting little to no agreement for ratings of Inattention and H/I symptoms (Fischer et al., 1993; Sibley et al., 2012). In our study, there was modest agreement for H/I (r = .40), but no significant agreement for Inattention (r = .15). This is consistent with studies establishing that overt and observable behaviors (e.g., symptoms of H/I) have a higher degree of agreement when compared with subjective or subtle behaviors (e.g., Inattention symptoms; Achenbach et al., 1987; De Los Reyes & Kazdin, 2005; Narad et al., 2015). Mother–teacher agreement was highest on ratings of academic impairment (r = .46), which is consistent with prior work, suggesting that inter-teacher agreement is stronger for academic impairment than for ADHD symptoms (Evans et al., 2005). Academic problems are the most frequently cited presenting problem for adolescents with ADHD (Barkley, 2006; Wolraich et al., 2005) and may be observed more easily than symptoms of the disorder.

With respect to predictors of informant discrepancies, findings varied by domain. Higher maternal education predicted mother ratings of Inattention severity that exceeded teacher reports. Highly educated mothers may spend more time with youth in academic activities (e.g., supervising homework; Guryan, Hurst, & Kearney, 2008) and may have higher expectations for adolescent functioning in academic settings (Davis-Kean, 2005). Thus, educated mothers may be more attuned to the adolescent's symptoms of Inattention and may display lower tolerance for these difficulties due to high expectations. Maternal psychological dysfunction was predictive of more severe ratings by mothers compared with teachers. On both symptom dimensions and on ratings of academic impairment, maternal parenting stress was associated with the more severe mother-rated symptom reports. This finding may reflect true situational variability in behavior expression (i.e., parenting stress stemming from disproportionately severe home behavior) or a tendency for mothers with high stress to inflate ratings (Youngstrom, Loeber, & Stouthamer-Loeber, 2000). Maternal ADHD significantly predicted maternal ratings that were higher than teacher ratings, but only for the H/I dimension. Perhaps, mothers with ADHD symptoms exacerbate adolescent symptoms at home due to a lack of structure, poor discipline, or elevated parent–teen conflict (Chronis-Tuscano et al., 2008). Furthermore, mothers with ADHD may have less patience with behavior problems, leading to inflated severity ratings (Chronis-Tuscano et al., 2008). Notably, maternal depression, which is a widely reported and reliable predictor of multiple informant discrepancies (De Los Reyes & Kazdin, 2005; Richters, 1992), did not significantly predict discrepancies in this population of students after controlling for parenting stress and maternal ADHD symptoms.

Teachers in special education classroom settings tended to report more H/I symptoms than mothers. This may occur because special education teachers frequently teach fewer students and may work more closely with each pupil (U.S. Department of Labor, 2014) allowing increased opportunities to observe symptoms. Additionally, students in special education classrooms may display more severe behavior problems at school than at home, warranting their restricted placement. Furthermore, these teachers may have more knowledge and experiences with students with ADHD, which has been shown to lead to more severe reports of problematic behaviors (Ohan et al., 2008). General education teachers showed much lower symptom ratings—as noted by Evans et al. (2005), these teachers may be poorly acquainted with students, not noticing certain ADHD symptoms. Thus, the school setting finding may partially reflect symptom omissions by mainstream middle school teachers, compared with their better acquainted special education counterparts. Finally, a documented ADHD diagnosis at school led to more severe ratings from mothers rather than teachers. We hypothesize that mothers who perceived their teen as highly symptomatic were more likely to pursue documentation of and school-based accommodations for ADHD (Johnson & Duffett, 2002).

These results should be interpreted within the context of their limitations. First, there is no “gold standard” by which to gauge the validity of informant reports of subjective symptoms. Therefore, we cannot fully determine whether discrepancies between mother and teacher reports represent true contextual and situational differences in symptom expression or are an artifact of informant perceptual differences. Additionally, recent statistical advances suggest that polynomial regression models (Laird & LaFleur, 2016) are more sensitive approaches to examining informant discrepancies than the traditional method described herein (De Los Reyes & Kazdin, 2004); however, utilizing these models to assess seven simultaneous predictors of informant discrepancies is not practical due to sample size requirements of this approach. In addition, our sample is predominantly Hispanic and findings may not generalize to all middle school students with ADHD. All parents were asked to fill out rating scales in English or Spanish depending on the language with which they felt most comfortable. In our sample, 17 mothers (16 %) filled out translated rating scales in Spanish, which may have led to subtle changes in rating scale interpretation. Finally, although teachers and parents were instructed to consider students off medication, it is possible that some teachers had never observed students without medication, which may have influenced their ratings. Although a minority of students in the sample received medication, parents of those who did may have had more opportunities to observe students off medication.

Nonetheless, our findings advance prior work by identifying key parent and school characteristics that may signal the presence of informant biases when interpreting ratings of middle school students with ADHD. Certain perceptual differences may emerge when mothers are highly educated, have pursued documentation of the diagnosis at school, or possess elevated levels of stress or ADHD symptoms. Accordingly, clinicians who evaluate adolescents with ADHD should assess the influence of mothers’ psychological and educational profiles on their expectations for and perceptions of the adolescent. Though symptom underreporting is established among middle school teachers who may not be well acquainted with students (Evans et al., 2005), this study suggests that clinicians should also be vigilant of overreporting by parents who are sensitized to the adolescent symptoms due to stress, their own ADHD symptoms, or high expectations for educational success. When parental overreporting is suspected, a semi-structured interview, rather than structured interview or rating scale, may protect against informant biases. Semi-structured interviews prompt parents to provide objective examples and descriptions of ADHD symptoms, deferring severity ratings to the judgment of the assessor (Pelham et al. 2005). In addition, teachers in special education settings may have more opportunities to accurately assess ADHD symptoms, and thus collection of reports from teachers who instruct the adolescent student in intimate settings may be more informative than reports from general education middle school teachers.