Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jan 1.
Published in final edited form as: J Youth Adolesc. 2019 Aug 24;49(1):311–322. doi: 10.1007/s10964-019-01107-x

Informant Discrepancies in Internalizing and Externalizing Symptoms in an At-Risk Sample: The Role of Parenting and School Engagement

Alexa L Curhan 1, Jill A Rabinowitz 1, Elise T Pas 1, Catherine P Bradshaw 2
PMCID: PMC6992516  NIHMSID: NIHMS1050095  PMID: 31446584

Abstract

A number of studies have used variable-centered approaches to examine informant discrepancies on children’s behavior problems; however, few such studies have used person-centered approaches to explore patterns of informant discrepancies or correlates of discrepancies in informant symptom ratings. The present study addressed these gaps by examining profiles of informant agreement on internalizing and externalizing symptoms and examining whether two important contextual factors, parenting and school engagement, are associated with profile membership. Data from an at-risk, urban sample of youth participants (N = 346, M age = 12.47 +/− 0.60 years, 56% male, and 75% Black), their caregivers, and one of their teachers were analyzed in the current study. Youth from 20 schools in a Mid-Atlantic state were screened for elevated levels of aggression and were selected to participate in the Early Adolescent Coping Power study. At baseline, youth, caregivers, and teachers reported on youth’s internalizing symptoms and caregivers and teachers reported on youth’s externalizing symptoms. Caregivers reported on their parenting; youth reported on their school engagement. Two internalizing symptoms profiles were identified: Low Symptoms Agreement and Youth-Reported High Somatization and Anxiety. Three externalizing symptoms profiles were identified: Low Symptoms Agreement, Teacher-Reported High Externalizing, and Caregiver-Reported High Externalizing. These profiles differed significantly on parenting behaviors and school engagement, shedding light on factors that may underlie informant discrepancies.

Keywords: internalizing symptoms, externalizing symptoms, multiple informants, person-centered approach, parenting, school engagement

Introduction

Internalizing and externalizing symptoms may occur in adolescence and place youth on a trajectory of socioemotional problems, poor academic performance, substance use, and peer and parental conflict (Bornstein, Hahn, & Haynes, 2010; Fanti & Henrich, 2010; King, Iacono, & McGue, 2004). Internalizing symptoms include anxiety, depression, and withdrawal, while externalizing symptoms include aggression, conduct problems, and hyperactivity (Bornstein, Hahn, & Haynes, 2010; Fanti & Henrich, 2010). Identifying these symptoms during adolescence can facilitate early intervention efforts to prevent the onset of adult psychopathology and improve youth socioemotional outcomes and academic success (Bornstein et al., 2010; Fanti & Henrich, 2010).

However, the identification of these symptoms can be complicated. Often, youth’s symptoms manifest differently depending on the context, resulting in discrepant informant reporting of symptoms (e.g., Dirks et al., 2012). For example, meta-analyses show low to moderate correlations between multiple informants (De Los Reyes et al., 2015). The varying reporter perspectives both demonstrates the need to include multiple informants (De Los Reyes & Kazdin, 2005) and presents challenges. For example, a multi-informant approach is more costly and burdensome and few assessments can be adequately or meaningfully compared across informants (Alexander et al., 2017; Olino & Klein, 2015). However, ignoring discrepancies creates a missed opportunity to identify symptoms, and subsequently intervene or increase awareness among reporters who may not notice symptoms. Although research has established that discrepancies are common (De Los Reyes et al., 2015), less is known about factors associated with informant consensus or discrepancies regarding youth’s symptoms. The current study aimed to identify profiles of informant agreement on internalizing and externalizing symptoms, and then explored whether parenting behaviors and school engagement were associated with profile membership.

The Utility of a Multi-Informant Approach

Previous research has shown limitations in using single-informant reports when making clinical and treatment decisions, estimating prevalence, and determining impact of internalizing and externalizing symptoms on a child (De Los Reyes et al., 2015). Because of the context dependency of behavior, one informant may only report on functioning that they observe in that context, missing symptoms that occur in other contexts. Thus, one informant may not accurately and completely describe symptomology, which may potentially result in misdiagnosis or an inappropriate treatment plan (Goolsby et al., 2018).

There are two primary statistical approaches to examine multi-informant data in research, which include variable-centered and person-centered methods. A variable-centered approach is focused on how two or more variables relate to one another (e.g., predictors and outcomes; how groups differ) and assumes that the sample’s response on a given variable is primarily homogenous (Laursen & Hoff, 2006). This is the most commonly-used approach and is made possible by combining rater reports (e.g., summing or averaging) to create overall scores (De Los Reyes & Kazdin, 2004) and examining correlations in informant reports across the sample (Sinclair et al., 2019). In contrast, a person-centered approach assumes that the sample’s response on a given variable is more likely to be heterogeneous, and it is used to examine whether subgroups exist within a sample and differ on variables of interest (Laursen & Hoff, 2006). One person-centered approach is latent profile analysis (LPA), which can allow for the identification of subgroups that exhibit different patterns and degrees of rater agreement or discrepancy (Masyn, 2013). LPA also allows one to investigate whether certain variables, such as context- or rater-specific factors, are associated with profile membership, which may shed light on factors that contribute to discrepancies.

Factors Related to Reporter Discrepancies

Consistent with Bronfenbrenner’s (1979) social ecological theoretical model, youth development is influenced by many environments (most notably, home and school) and people with whom they come into contact within these settings. Because behavior is often context specific, rater discrepancies may be related to behavioral differences in the home or school that relate to contextual factors. Of specific interest, are malleable environmental factors that have been shown to be associated with youth behavior and are responsive to intervention, including relationships at school and parenting strategies.

School Context.

Youth’s connection to their teachers may be associated with teacher report of symptoms, although there is a dearth of research in this area. Extant research indicates that higher levels of teacher-reported familiarity with students is associated with fewer parent-teacher rater discrepancies of internalizing symptoms (Ines & Sacco, 1992). By extension, a student’s relationships with other students and general school connectedness may also relate to informant discrepancies. Teachers’ perceptions of youth’s psychosocial functioning, such as acting-out, moodiness, and depression, are associated with their perception of the quality of the child’s peer relationships (Totura et al., 2009). Thus, if a student experiences poor peer relationships and school connectedness, teachers may report higher levels of symptoms.

Children with learning problems may demonstrate a range of behaviors (e.g., withdrawal, inattention, disruptiveness) in the classroom to avoid the demands of the school environment, including classwork or in response to frustration. This may lead teachers to report higher levels of externalizing or internalizing symptoms for students with learning problems (Totura et al., 2009). Thus, students with behavioral and learning challenges often have lower school engagement, which may be associated with rater discrepancies (Totura et al., 2009).

Home Context.

Parenting behaviors may influence the elicitation of symptoms (Van heel et al., 2019), as well as the parent’s perception of those symptoms. Although research has examined indices of family structure (e.g., family dysfunction) in relation to informant discrepancies, few studies have examined parenting behaviors. Available work indicates that greater parent-reported stress and lower family cohesion predicted greater parent-child discrepancies for internalizing symptoms (Kolko & Kazdin, 1993). Further, parents who report greater acceptance of their child demonstrate higher parent-child agreement on externalizing symptoms (Al Ghriwati et al., 2018).

Specific parent behaviors of interest that may relate to informant discrepancies are parental monitoring and positive parenting, given their known associations with youth’s internalizing and externalizing symptoms (Eisenberg et al., 2005; Racz & McMahon, 2011). For example, higher parental monitoring is associated with lower parent- and teacher-reported externalizing symptoms (Racz & McMahon, 2011), suggesting a cross-context influence. Thus, one might expect lower externalizing symptoms across contexts and potentially greater parent-teacher agreement on these symptoms among youth with more parental monitoring. Similarly, positive parenting (e.g., praising a youth for positive behaviors) may increase youth engagement in emotion regulation strategies, which in turn may mitigate risk for internalizing and externalizing symptoms (Eisenberg et al., 2005). This should also have a cross-context impact and increase rater agreement. Examining other facets of parenting, such as involvement, may shed light on informant discrepancies given that both over-involvement and under-involvement has been associated with higher symptom levels (Scaramella & Leve, 2004).

Parental stress, such as lower social support, is also associated with informant discrepancies (Grills & Ollendick, 2002; Kolko & Kazdin, 1993) and is likely interrelated with positive parenting. For example, parents with greater social support often exhibit greater warmth and lower hostility towards their children (Lippold et al., 2018), which may be related to reduced symptoms across contexts and greater informant agreement. Additionally, increased parenting stress has been found to be associated with parent-teacher discrepancies in ratings of youth’s functioning (Yeguez & Sibley, 2016). Thus, an examination of whether social support is associated with rater consensus on youth’s symptoms is warranted.

The Current Study

The first aim of the current study was to examine agreement and discrepancy profiles between informants for youth’s internalizing symptoms and externalizing symptoms using a person-centered method, building on studies that have examined informant discrepancies using a variable-centered approach (e.g., Sinclair et al., 2019). This approach allowed for the identification of distinct subgroups of individuals that varied in the severity of caregiver, teacher, or youth symptom reports and may be differentially associated with course, correlates, outcomes, and intervention responses. There is a dearth of person-centered research in this area, and thus no clear prior research indicating specific patterns of concordance and discordance. However, given prior research indicating moderate correlations between multiple raters (e.g., De Los Reyes et al., 2015) a pattern of concordance and at least one pattern of discordance was expected. Further, given prior work indicating lower levels of agreement on reports of internalizing versus externalizing symptoms (De Los Reyes & Kazdin, 2005), discordance between adults and youth on internalizing symptoms was expected. Consistent with a recent call to study factors associated with informant discrepancies (e.g., De Los Reyes et al., 2019a), this study sought to examine malleable context- and rater-specific factors that may be associated with both agreement and discrepancies. In line with this recent research, the second aim of the study was to extend prior research by investigating whether school engagement and/or parenting behaviors and social supports were associated with profile membership. Based on the literature reviewed above, it was expected that the fewest rater discrepancies would emerge for youth who reported higher levels of school engagement and connectedness, as well as for youth whose caregivers had the most social support and engaged in higher levels of parental monitoring, involvement, and positive parenting, as opposed to those who engaged in inconsistent parenting and discipline.

Method

Participants

The study sample included a teacher screening process to identify 346 seventh-grade students (M age = 12.47 +/− 0.60 years) manifesting elevated levels of aggressive behavior; for each student, the sample also included a caregiver and one core teacher. The data came from 20 middle schools across two urban school districts in a Mid-Atlantic state. The student sample was 56% male and mostly Black (i.e., 75%). The demographic composition of this sample was aligned to the 2016–17 school-level demographics, which on average, included a student body of which 68% of students were Black, 18% White, and 9% Hispanic/Latino. The schools in the study had an average enrollment of 560 students (range = 332–800).

Procedures

The main study was conducted collaboratively by three universities, all of which received Institutional Review Board approval. Sixth grade teachers completed a universal, six-item screening measure assessing student proactive and reactive aggression in the spring (Lochman, & Conduct Problems Prevention Research Group, 1995). Students who received a score of 11 (i.e., possible range of 6–30), tending to be ranked in the top 25% of the student body for aggressive behaviors, were recruited for participation in the study during the following year (i.e., the fall of seventh grade) and comprise the study sample. Participation was voluntary. Caregivers and students provided consent and assent, respectively. All study data were collected at baseline (i.e., during the fall of seventh grade), prior to participation in the Early Adolescent Coping Power preventive intervention (Bradshaw, Lochman, Powell, & Ialongo, 2017).

Measures

Youth and Family Demographics.

Student demographic data were collected in a survey adjoined to the Behavior Assessment System for Children 2 (BASC-2: Adolescent [ages 12–21]; Reynolds & Kamphaus, 2004).

Internalizing and Externalizing Symptoms.

Teachers, caregivers, and youth reported on youth’s symptoms using the BASC-2: Adolescent, ages 12–21 (Reynolds & Kamphaus, 2004). Teachers and caregivers reported on three overlapping externalizing behavior scales (aggression, conduct problems, and hyperactivity), whereas youth only completed one scale (i.e., hyperactivity); thus, youth-reported externalizing symptoms were not included in the analyses. The externalizing scales included aggression (10 items for teacher report, α = .71; 10 items for caregiver, α = .88; all alphas reported are from the larger intervention study), conduct problems (12 items for teacher report, α = .91; 14 items for caregiver, α = .87), and hyperactivity (11 items for teacher report, α = .78; 8 items for caregiver, α = .81). Teachers, youth, and caregivers reported on three scales of internalizing behaviors, including: anxiety (7 items for teacher report, α = .81; 11 items for caregiver, α = .86; 13 items for youth, α = .83); depression (11 items for teacher report, α = .83; 12 items for caregiver, α = .86; 12 items for youth, α = .85); and somatization (8 items for teacher report, α = .83; 11 items for caregiver, α = .79; 7 items for youth, α = .69). Each informant rated the frequency of the behaviors over the past 6 months on a three-point Likert scale ranging from 0 (“never”) to 3 (“always”). A summed score was created for each scale, and the scores were converted into t-scores, which were used in this study. The BASC has demonstrated adequate reliability and validity (Reynolds & Kamphaus, 2002).

Parenting.

The Alabama Parenting Questionnaire-Parent Version (APQ; Shelton, Frick & Wootton, 1996), a 42-item questionnaire, was used to assess caregiver report of their own involvement, positive parenting, monitoring, and inconsistent discipline. The involvement scale included 10 items (α = .88; e.g. item, “You play games or do other fun things with your child”). The positive parenting scale was a 6-item scale (α = .87; “You let your child know when he/she is doing a good job with something”). The monitoring scale included 10 items (α = .78; e.g., “Your child goes out without a set time to be home”). The inconsistent discipline scale included 6 items (α = .76; e.g., “You threaten to punish your child and then do not actually punish him/her”). Caregivers reported on the frequency of their parenting behaviors on a four-point Likert scale from 1 (“never”) to 4 (“always”) and items were summed for each scale. Higher scores reflect higher involvement, positive parenting, monitoring, and inconsistent discipline. The APQ has demonstrated reliability and validity in an adolescent sample (Hurley et al., 2014; Zlomke et al., 2014). These scales were included in auxiliary analyses.

Caregiver Social Support.

The Exosystem Social Support Scale is a 21-item self-report index of social support that was an adapted form of the Social Support Questionnaire (Sarason et al., 1983) and administered to caregivers. Each caregiver indicated whether a series of individuals (e.g., grandparent, cousin) provided social supports in his/her life and then indicated whether they were satisfied with each individual’s support of their care for the child. Responses were rated on a six-point Likert scale from 0 (“very dissatisfied”) to 5 (“very satisfied”). The number of supports were summed to reflect the total social support (i.e., total number of individuals who provided the caregiver with social support). The summed satisfaction score reflected the caregivers’ level of social support satisfaction with the child-specific social support they received. Both measures were included in auxiliary analyses.

School Engagement.

Youth reported on four facets of school engagement by completing the Youth Self-Report of School Climate, a validated measure of school climate demonstrating theoretical model fit and measurement invariance across student gender, race/ethnicity, and school grade (Bradshaw et al., 2014). The academic engagement scale (4 items; e.g., “My teachers believe that I can do well in school”; α = .75) measures perceptions of attaining academic success. The connection to teachers scale (6 items; e.g., “My teachers listen to me when I have something to say”, α = .77) reflects connectedness to all of the student’s teachers. The whole-school connectedness scale (4 items; e.g., “I enjoy learning at this school”, α = .83) measures student’s feelings about their school experience. The student connectedness scale (5 items; e.g., “Students respect one another”, α = .73) measures how the student perceives student relationships and bonding to one another. All items were rated on a four-point Likert scale from 1 (“strongly disagree”) to 4 (“strongly agree”) and were summed to create each subscale (see Bradshaw et al., 2014). These scales were included in auxiliary analyses.

Statistical Analyses

Bivariate correlations and descriptive statistics were conducted using SPSS Version 25 (IBM, 2016). Two latent profile analyses (LPA) were conducted in Mplus Version 8 (Muthén & Muthén, 1998–2017) to identify profiles of caregiver-, youth-, and teacher-reported agreement regarding youth internalizing symptoms (first LPA), and caregiver- and teacher-reported agreement regarding externalizing symptoms (second LPA). Given that the sample was comprised of youth attending 20 schools, the data were nested. Nesting was controlled for in the analyses by utilizing the TYPE = complex command. Full Information Maximum Likelihood (FIML) estimation was used to estimate missing data (Enders, 2001).

In the latent profile analysis, models were built in an iterative fashion (i.e., beginning with a single class and adding one additional class until fit was determined inadequate). Several model fit indices were evaluated to determine the model that best fit the data. These included the Akaike Information Criterion (AIC; Akaike, 1987), Bayesian Information Criterion (BIC; Schwartz, 1978), sample-size adjusted BIC (ABIC; Sclove, 1987), the Vuong-Lo-Mendell-Rubin (VLMR) Likelihood Ratio Test, and entropy (Masyn, 2013; Nylund, et al., 2007). A decreasing AIC, BIC, and adjusted BIC, a higher entropy, as well as a statistically significant VLMR Likelihood Ratio Test generally indicate better model fit (Ramaswamy, DeSarbo, Reibstein, & Robinson, 1993). The VLMR Likelihood Ratio Test compares the fit of the model with n profiles to the fit of the model with n-1 profiles (Masyn, 2013). Entropy reflects the degree of classification certainty; values closest to 1.00 and posterior probabilities greater than .70 are favorable (Nagin, 2005; Ramaswamy et al., 1993). Class sample size was also considered, as profile sizes less than 5% of the sample may indicate an over-fitted model. Last, each class was examined to determine whether they were distinct and conceptually meaningful. Once the internalizing and externalizing LPA models were selected, auxiliary analyses were conducted using omnibus chi-square tests. These tests were conducted to test the equality of means of the targeted variables (i.e., caregiver reports of involvement, positive parenting, monitoring, inconsistent discipline, total social support, and child-specific social support and student reports of academic engagement, connection to teachers, whole-school connectedness, and student connectedness) across all profiles and provided chi-square values and an effect size (phi; ϕ) for all omnibus chi-square tests (Masyn, 2013). Pairwise comparisons were examined only if the omnibus tests were significant (p < .05).

Results

Descriptive statistics and bivariate correlations for informant-reported internalizing and externalizing scales, as well as informant-reported symptoms and the auxiliary variables (e.g., caregiver-reported involvement) are shown in the appendices. Results from the primary analyses are below.

Latent Profile Analysis for Internalizing Symptoms

An LPA was fit using the t-scores for the caregiver-, youth-, and teacher-reported subscales of internalizing symptoms (i.e., anxiety, somatization, and depression). For the internalizing model, the BIC, AIC, and adjusted BIC decreased from the one-profile model to the two-profile model, the sample sizes of both classes were adequate, and the VLMR Likelihood Ratio Test showed a trend for significance (p = .09). Although the BIC decreased in the 3-profile model, the VLMR Likelihood Ratio Test was not significant and it included a class that was very small. Thus, the 2-profile model was selected (see Table 2). Profile 1 (n = 240, 69%) was labeled “Low Symptoms Agreement,” as caregivers, youth, and teachers all agreed upon subclinical symptom levels (see Figure 1A). Profile 2 (n = 106, 31%) was labeled “Youth-Reported High Somatization and Anxiety” (referred to hereafter as Youth High Internalizing), as youth reported symptoms above the at-risk cutoff (t-scores > 60) and discordant with the below at-risk caregiver and teacher reports. All informants reported at-risk levels of depression.

Table 2.

Fit Indices for Each Latent Profile Analyses

Internalizing Model
Number of profiles Number of free parameters Log likelihood AIC BIC ABIC VLMR LRT Entropy
1 18 −9975.40 19986.80 20056.04 19998.94 -- --
2 28 −9820.85 19697.71 19805.41 19716.58 .09 .79
3 38 −9731.39 19538.78 19684.95 19564.40 .29 .86
Externalizing Model
Number of profiles Number of free parameters Log likelihood AIC BIC ABIC VLMR LRT Entropy
1 12 −6614.27 13252.53 13298.62 13260.55 -- --
2 19 −6321.86 12681.71 12754.69 12694.41 .002 .85
3 26 −6230.07 12512.14 12612.00 12529.52 .03 .84
4 33 −6145.46 12356.92 12483.67 12378.98 .68 .80
Open in a new tab

Note. AIC = Akaike Information Criterion, BIC = Bayesian Information Criterion, ABIC = Sample-size Adjusted BIC, VLMR LRT = Vuong Lo-Mendell Rubin Likelihood Ratio Test. VLMR LRT and entropy are not calculated for the 1-profile model. Profiles bolded are the best-fitting and selected models.

Figure 1.

Figure 1.

Open in a new tab

A. Caregiver-, youth-, and teacher-reported internalizing symptoms for each profile in the two profile-model; and B. Caregiver- and teacher-reported externalizing symptoms for each profile in the three-profile model. CR = caregiver report, YR = youth report, TR = teacher report, Som = somatization, Anx = anxiety, Dep = depression. Agg = aggression, CP = conduct problems, Hyper = hyperactivity. At-Risk T-Score: 60–69; Clinical T-Score: > 70.

Auxiliary Analyses for Internalizing Symptoms Model

Parenting.

Significant between-profile differences were found regarding parental monitoring (χ2 = 10.32, p < .01, ϕ = .17) and inconsistent discipline (χ2 = 4.33, p < .04, ϕ = .11). Caregivers of youth in the Low Symptoms Agreement profile reported higher monitoring and lower inconsistent discipline than caregivers of youth in the Youth High Internalizing profile (see Table 3). There were no significant differences for involvement or positive parenting.

Table 3.

Results of Auxiliary Analyses for the 2-Profle Model of Internalizing Symptoms

Low Symptoms Agreement (Profile 1; n = 240) Youth-Reported High Somatization and Anxiety (Profile 2; n = 106) Omnibus χ2 test Pairwise Comparisonsa
M SD M SD χ2 P ϕ
Parenting (CR)
Involvement 25.87 8.70 24.35 7.28 2.76 .10 .09 --
Positive Parenting 19.80 4.01 19.39 4.52 0.61 .43 .04 --
Monitoring 33.67 8.21 31.61 8.03 10.32 <0.01 .17 1 > 2
Inconsistent Discipline 7.32 4.45 8.69 6.31 4.33 .04 .11 2 > 1
Caregiver Total Social Support 6.07 3.70 6.74 7.39 0.61 .43 .04 --
Caregiver Social Support-Satisfaction 22.52 13.35 23.62 36.97 0.07 .79 .01 --
School Engagement (YR)
Connection to Teachers 12.94 3.86 10.67 3.39 20.88 <0.01 .25 1 > 2
Academic Engagement 11.08 2.60 9.60 2.48 27.93 <0.01 .28 1 > 2
Whole-School Connectedness 9.25 2.99 7.37 2.99 21.59 <0.01 .25 1 > 2
Student Connectedness 10.50 3.27 8.22 4.49 16.35 <0.01 .22 1 > 2
Open in a new tab

Note. CR = caregiver report, YR = youth report.

a

For significant omnibus tests, all pairwise comparisons are significant at the p < .05 level.

Caregiver Social Support.

There were no between-profile differences.

School Engagement.

There were significant between-profile differences for academic engagement and connectedness. Youth in the Low Symptoms Agreement profile reported greater academic engagement (χ2 = 27.93, p < .01, ϕ = .28), connection to teachers (χ2 = 20.88, p < .01, ϕ = .25), school connectedness (χ2 = 21.59, p < .01, ϕ = .25) and student connectedness (χ2 = 16.35, p < .01, ϕ = .22) than youth in the Youth High Internalizing profile.

Latent Profile Analysis for Externalizing Symptoms

An LPA was conducted including caregiver- and teacher-reported externalizing symptoms (i.e., conduct problems, hyperactivity, and aggression). Moving from a one- to a four-profile model resulted in a decrease in the AIC, BIC, and adjusted BIC and distinct classes. However, the VLMR Likelihood Ratio Test was not significant for the 4-class model and had a class with a very small sample size (~8.0%). Given that the 3-profile model showed a significant VLMR Likelihood Ratio Test, had good entropy, and the classes identified were distinct and had adequate sample sizes, the 3-profile solution was selected (see Table 2). Profile 1 (n = 216, 63%), was named “Low Symptoms Agreement,” as caregivers and teachers in this profile reported agreement on low symptom levels below the at-risk cutoff (t-scores < 60; see Figure 1B). Profile 2 (n = 59, 17%) was labeled “Teacher-Reported High Externalizing” (referred to hereafter as Teacher High Externalizing), as teachers in this profile consistently reported higher levels of externalizing symptoms (t-scores > 70), which was discordant with caregiver reports. Profile 3 (n = 69, 20%) was labeled “Caregiver-Reported High Externalizing” (referred to hereafter as Caregiver High Externalizing), as caregivers in this profile consistently reported higher and clinically significant conduct disorder symptoms and hyperactivity (t-scores > 70), which were discordant with teacher ratings. Both caregivers and teachers reported clinical levels of aggression.

Auxiliary Analyses for Externalizing Model

Parenting.

There were significant between-profile differences regarding monitoring (χ2 = 9.34, p < .01, ϕ = .16) such that caregivers of youth in the Low Symptoms Agreement profile reported higher monitoring than caregivers of youth in the Caregiver High Externalizing profile (χ2 = 9.33, p < .01, ϕ = .16; see Table 4). There were significant differences in caregiver-reported inconsistent discipline (χ2 = 12.38, p < .01, ϕ = .19) such that the caregivers of youth in the Caregiver High Externalizing profile reported higher inconsistent discipline compared to caregivers of youth in the Low Symptoms Agreement profile (χ2 = 11.61, p < .01, ϕ =.18) and the Teacher High Externalizing profile (χ2 = 5.46, p < .02, ϕ = .13). There were also significant differences for caregiver-reported positive parenting (χ2 = 8.17, p < .02, ϕ = .15) such that caregivers in the Teacher High Externalizing profile reported higher positive parenting than caregivers in the Low Symptoms Agreement profile (χ2 = 7.49, p < .01, ϕ = .15).

Table 4.

Results of Auxiliary Analyses for the 3-Profle Model of Externalizing Symptoms

Low Symptoms Agreement (Profile 1; n = 216) Teacher-Reported High Externalizing (Profile 2; n = 59) Caregiver-Reported High Externalizing (Profile 3; n = 69) Omnibus χ2 test Pairwise Comparisonsa
M SD M SD M SD χ2 P ϕ
Parenting (CR)
Involvement 25.50 6.44 25.94 8.66 24.06 9.10 1.83 .40 .07 --
Positive Parenting 19.40 3.06 21.03 5.15 19.26 4.82 8.17 .02 .15 2 > 1
Monitoring 33.67 6.76 32.80 11.60 29.75 11.88 9.34 .01 .16 1 > 3
Inconsistent Discipline 7.16 4.84 7.51 6.30 11.33 9.33 12.38 < 0.01 .19 3 > 2,1
Caregiver Total Social Support 6.61 3.09 5.56 6.30 5.64 10.05 2.41 .30 .08 --
Caregiver Social Support-Satisfaction 23.37 15.28 22.02 15.82 21.49 39.46 0.47 .79 .04 --
School Engagement (YR)
Connection to Teachers 12.48 3.34 11.34 2.91 12.23 3.36 5.97 .05 .13 --
Academic Engagement 10.88 2.42 9.38 2.80 10.83 1.84 15.42 <0.01 .21 1,3 > 2
Whole-School Connectedness 8.69 3.35 8.54 3.08 8.73 2.64 0.14 .94 .02 --
Student Connectedness 9.82 3.23 9.73 4.80 10.04 2.84 0.37 .83 .03 --
Open in a new tab

Note. CR = caregiver report, YR = youth report.

a

For significant omnibus tests, all pairwise comparisons are significant at the p < .05 level.

Caregiver Social Support.

There were no significant between-profile differences.

School Engagement.

There were significant differences for academic engagement, (χ2 = 15.42, p < .01, ϕ = .21), such that youth in the Teacher High Externalizing profile reported lower academic engagement than youth in the Low Symptoms Agreement profile (χ2 = 13.48, p < .01, ϕ = .20) and the Caregiver High Externalizing profile (χ2 = 13.89, p < .01, ϕ = .20). No significant differences were found for whole-school, teacher, or student connectedness.

Discussion

Previous research has predominantly examined the degree of correspondence in informant reports of youth’s behavior problems using variable-centered approaches (e.g., Roskam et al. 2018). This study sought to extend prior research using multi-informant data by employing a person-centered approach to identify profiles of rater agreement and discrepancy of youth’s internalizing and externalizing symptoms. Further, whether and how membership in these profiles was associated with several factors was examined, representing the intersection of context and raters (e.g., parenting behaviors and youth-reported school engagement). Two profiles were identified for internalizing symptoms and three profiles were identified for externalizing symptoms, both of which differed in the frequency and quality of informant reports of children’s symptoms. Profiles also differed in reported parenting behaviors and school engagement, highlighting the importance of considering multiple informants and contexts in which youth are embedded.

Regarding the internalizing symptom profiles, the majority of the sample (i.e., 69%) were in the Low Symptoms Agreement profile, which was characterized by agreement among all three raters, whereby each informant reported generally low symptom levels that were below the at-risk cutoff (t-scores < 60). The second profile, Youth High Internalizing, included 31% of the sample, whereby youth were rated by all raters as exhibiting signs of depressions, and (only) youth reported at-risk levels of somatization and anxiety. This class is consistent with prior literature that has shown that discrepancies for internalizing symptoms often arise between youth self-report and caregiver and teacher reports (De Los Reyes et al., 2015) and extends previous work by showing that informant disagreement may be more likely to occur with somatization and anxiety rather than depression. Given this finding, future assessments should measure specific facets of internalizing behaviors, rather than broadband internalizing symptoms. Previous work has also demonstrated the presence of informant discrepancies across different symptoms of depression in youth (Cole et al., 2017). Moreover, future work is needed to identify factors that contribute to the greater observed teacher-parent agreement in depressive symptoms as compared to anxiety and somatization. Youth in this class may have been experiencing anxiety symptoms and somatization that parents and teachers were unable to detect. The presence of this profile highlights the importance of considering youth report and suggests that greater education of parents and teachers regarding what anxiety and somatization may look like and h manifest may improve parent and teacher detection of internalizing symptoms among youth.

Youth in the Low Symptoms Agreement profile had caregivers who reported higher monitoring and consistent discipline, compared to caregivers of youth displaying internalizing symptoms. This finding is consistent with research that higher parental monitoring and consistent discipline reduce risk for internalizing symptoms (Balan et al., 2017; Lionetti et al., 2018). Caregiver monitoring may be associated with increased communication between a caregiver and their child, which may play a role in increased caregiver-youth agreement. Caregivers of students displaying internalizing symptoms reported more inconsistent discipline, which aligns with work linking inconsistent parenting to higher levels of internalizing symptoms (Balan et al., 2017; Lionetti et al., 2018). The unpredictability of parental disciplinary responses to youth may cause them to feel insecure in their relationships or may be experienced as harshness or hostility by youth, which could contribute both to feelings of anxiety and depression (Balan et al., 2017; McLeod, Weisz, & Wood, 2007).

Youth in the Low Symptoms Agreement profile reported significantly greater academic engagement, connections to teachers, school connectedness, and student connectedness, with small to moderate effect sizes observed. Students who are more committed to school work and have better relationships with their teachers may be less likely to exhibit internalizing symptoms, potentially because they are performing well academically and view teachers as supportive of their learning and development (Rolland, 2012). On the other hand, students with fewer internalizing symptoms may find it easier to excel in school and develop relationships with peers and teachers (Hurd, Hussain, & Bradshaw, 2018). Thus, higher levels of school engagement may be associated with lower cross-context internalizing symptoms, resulting in agreement across raters on these symptom levels.

Three profiles were identified for externalizing symptoms: (1) Low Symptoms Agreement, (2) Teacher High Externalizing, and (3) Caregiver High Externalizing. The Low Symptoms Agreement profile demonstrated consistent agreement across raters of low symptom levels (average t-scores < 57) for 63% of students, which was unexpected given that these youth were screened into this study for externalizing problems. The presence of this profile provides support for looking at subgroups within a sample. Within this sample of youth with relatively higher levels of aggression on a universal screening, there was still a subgroup of youth with less severe symptoms. There were two groups displaying externalizing symptoms, in which rater discrepancies emerged. Teachers reported clinical levels of externalizing symptoms across all scales for 17% of the sample, whereas caregivers reported subclinical levels of symptoms on all scales. Teachers may have different behavioral expectations (e.g., sitting still, requiring permission to speak) compared to caregivers; some children may have trouble adhering to these expectations, resulting in higher teacher reports of externalizing symptoms than caregivers. Alternatively, youth may exhibit high externalizing symptoms across both contexts, but caregivers and teachers have different thresholds for problematic behavior.

Caregivers reported higher levels of positive parenting and lower levels of monitoring among youth in the Teacher High Externalizing profile, as compared to caregivers in the Low Symptoms Agreement profile. A parent who reports positive behaviors in conjunction with lower levels of monitoring may be engaging in permissive parenting, which has been associated with externalizing behaviors (Pinquart, 2017). Caregivers also reported higher levels of inconsistent discipline for youth in the Caregiver High Externalizing profile compared to caregivers in the other profiles. It may be more difficult for a caregiver to consistently discipline and monitor a child who is exhibiting externalizing symptoms. Children may also receive inconsistent messages regarding what behavior is appropriate and/or may perceive parental reprimands as unpredictable, which may increase the likelihood of externalizing behaviors.

Youth in the Teacher High Externalizing profile reported the lowest levels of academic engagement, which is consistent with previous research showing that teachers report higher levels of symptoms among youth with school-specific difficulties (De Los Reyes & Kazdin, 2005; Totura et al., 2009). The discrepancies that were found between caregiver- and teacher-reported externalizing problems may be better understood if youth report was included, as youth could provide further insight as to why they may be exhibiting different symptoms across contexts or why their symptoms are being perceived differently. For example, youth may be exhibiting higher levels of externalizing symptoms if they are also experiencing factors associated with decreased school engagement (e.g., bullying or low school connectedness) that has gone unrecognized by teachers. Additionally, incorporating more objective measures of school engagement, such as attendance or change in academic performance, could help elucidate other aspects of the school environment that are associated with informant discrepancies.

There are some limitations of the present study to acknowledge. Youth self-report of externalizing symptoms was not included in the analyses given that youth do not complete the same BASC-2 scales as adult reporters. Similarly, teacher ratings of the classroom climate, expectations, or connectedness to students were not available. Future studies should incorporate such teacher-reported variables as well as caregiver-reported behavioral expectations at home to determine whether the symptom discrepancies by rater encompass a contextual or expectations discrepancy. An additional avenue for future work is to assess youth perceptions of their caregiver’s parenting since discrepant parent-adolescent perspectives of parenting have also been found to be associated with internalizing and externalizing symptoms (De Los Reyes et al., 2019b). The sample of youth were identified through a pre-screening process as having higher levels of aggression; thus, replication of these findings is needed in other population-based samples (e.g., community-based settings) or samples identified for other behavioral or mental health problems (e.g., internalizing symptoms). Notably, there was still a sizeable sample of youth with internalizing symptoms and those without clinical levels of aggression (or other externalizing behaviors), as well as rater variability in the report of aggression. This study was cross-sectional and causality cannot be inferred. Additional longitudinal research examining how youth symptoms influence parenting and school engagement is needed. Within the broader study from which these data were collected, the main research focus was on assessing the efficacy of the Early Adolescent Coping Power intervention (Bradshaw et al., 2017), which takes an ecological approach to address youth aggression by targeting the skills of students, parents, and teachers. An examination of whether the intervention was more or less efficacious with students in these varying subgroups is an additional area for further research.

Conclusion

Informant discrepancies in children’s behavior problems have long been considered a nuisance by researchers and clinicians alike and have often been interpreted as a problem when administering assessments and developing treatments plans. While some researchers have attempted to understand informant discrepancies by examining the associations between rater reports within a sample, few studies have attempted to identify (a) distinct profiles of individuals that vary in their frequency and level of symptoms across informant reports, or (b) what factors are associated with profile membership for informant agreement or disagreement on youth’s symptoms. Building on previous work, this study utilized a person-centered approach, allowing for the examination of patterns of informant agreement or disagreement in reports of internalizing and externalizing behaviors. Further, this study identified factors (i.e., parenting and school engagement) that were associated with the identified profiles. Study findings highlight several potential points of intervention. For example, teaching parenting skills, such as monitoring, may be especially important during adolescence when youth begin to communicate less with their caregivers or when adolescents are misbehaving in the classrooms as reflected in higher levels of teacher-reported externalizing symptoms. In addition, given that adolescence is characterized by numerous maturational changes and a general rebellion towards authority figures, the importance of maintaining youth’s engagement to school is essential to ensure the development of positive behaviors, rather than those that are disruptive and interfere in their own and other students’ learning and potential. Greater communication between school personnel, parents, and youth may also aid in the detecting of symptoms that are occurring in specific environments or across environments, thereby allowing for the development of appropriate support systems and interventions that may help attenuate youth’s symptoms not only during adolescence, but across the developmental course.

Supplementary Material

1

Table 1.

Student and Caregiver Demographic Information for the Sample

Student race n (%)
Black 258 (75%)
White 37 (11%)
Latino/Hispanic 25 (7%)
Other 24 (7%)
Student gender
Female 152 (44%)
Male 193 (56%)
Parental marital status
Married 22 (22%)
Single 55 (56%)
Divorced 15 (15%)
Separated 4 (4%)
Other 3 (3%)
Annual income
< $10,000 17 (18%)
$10,000 – $29,000 32 (34%)
$30,000 – $49,000 28 (29%)
$50,000 – $99,000 10 (11%)
> $100,000 8 (8%)
Open in a new tab

Acknowledgement

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A140070 (PI: C. Bradshaw) to the University of Virginia. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education. The authors would like to acknowledge Dr. John Lochman who is a Co-PI on the grant and supported the data collection for this study.

Biography

Alexa Curhan is a PhD Student in the Department of Mental Health at the Johns Hopkins Bloomberg School of Public Health. Her major research interests include child and adolescent mental health, mood disorders, Autism Spectrum Disorder, neuroimaging, and school-based interventions.

Dr. Elise Pas is an Associate Scientist in the Department of Mental Health at the Johns Hopkins Bloomberg School of Public Health. Her major research interests include school-based implementation and dissemination of evidence-based interventions to promote behavioral, social, emotional, and mental health development.

Dr. Jill Rabinowitz is an Assistant Scientist in the Department of Mental Health at the Johns Hopkins Bloomberg School of Public Health. Her major research interests include the interplay between genetic predispositions and contextual factors on the development of substance use problems among adolescents and young adults.

Dr. Catherine Bradshaw is a Professor and the Senior Associate Dean for Research and Faculty Development at the Curry School of Education and Human Development at the University of Virginia. Her major research interests include aggressive behavior and school-based prevention, bullying and school climate, the development of aggressive and problem behaviors, effect of exposure to violence, peer victimization and environment stress on children, children with emotional and behavioral disorders and autism, and the design, evaluation, and implementation of evidence-based prevention programs in schools.

Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Data Sharing Declaration

This manuscript’s data will not be deposited.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional review board at the University of Virginia (IRB#2014-0247-00) and the Johns Hopkins School of Public Health (IRB #5807) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

References

  1. Akaike H (1987). Factor analysis and AIC. Psychometrika, 52, 317–332. [Google Scholar]
  2. Al Ghriwati N, Winter MA, Greenlee JL, & Thompson EL (2018). Discrepancies between parent and self-reports of adolescent psychosocial symptoms: Associations with family conflict and asthma outcomes. Journal of Family Psychology, 32(7), 992–997. [DOI] [PubMed] [Google Scholar]
  3. Alexander LA, McKnight PE, Disabato DJ, & Kashdan TB (2017). When and How to use Multiple Informants to Improve Clinical Assessments. Journal of Psychopathology and Behavioral Assessment, 39(4), 669–679. [Google Scholar]
  4. Balan R, Dobrean A, Roman GD, & Balazsi R (2017). Indirect effects of parenting practices on internalizing problems among adolescents: The role of expressive suppression. Journal of Child and Family Studies, 26(1), 40–47. [Google Scholar]
  5. Bornstein MH, Hahn CS, & Haynes OM (2010). Social competence, externalizing, and internalizing behavioral adjustment from early childhood through early adolescence: Developmental cascades. Development and Psychopathology, 22(4), 717–735. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bradshaw CP, Lochman J, Powell N, & Ialongo N (2017). Preventing bullying in middle schoolers by using the Coping Power Program: A targeted group intervention In Bradshaw CP (Ed.). Handbook on Bullying Prevention: A Lifecourse Perspective. (pp. 215–220). New York: National Association of Social Workers Press. [Google Scholar]
  7. Bradshaw CP, Waasdorp TE, Debnam KJ, & Johnson SL (2014). Measuring school climate in high schools: A focus on safety, engagement, and the environment. Journal of School Health, 84(9), 593–604. [DOI] [PubMed] [Google Scholar]
  8. Bronfenbrenner U (1979). The ecology of human development: Experiments by nature and design. Cambridge, Massachusetts: Harvard University Press. [Google Scholar]
  9. Cole DA, Martin JM, Jacquez FM, Tram JM, Zelkowitz R, Nick EA, & Rights JD (2017). Time-varying and time-invariant dimensions of depression in children and adolescents: Implications for cross-informant agreement. Journal of Abnormal Psychology, 126(5), 635–651. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. De Los Reyes A, & Kazdin AE (2004). Measuring informant discrepancies in clinical child research. Psychological Assessment, 16(3), 330–334. [DOI] [PubMed] [Google Scholar]
  11. De Los Reyes A, & Kazdin AE (2005). Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychological Bulletin, 131(4), 483–509. [DOI] [PubMed] [Google Scholar]
  12. De Los Reyes A, Augenstein TM, Wang M, Thomas SA, Drabick DAG, Burgers DE, & Rabinowitz J (2015). The validity of the multi-informant approach to assessing child and adolescent mental health. Psychological Bulletin, 141(4), 858–900. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. De Los Reyes A, Cook CR, Gresham FM, Makol BA, & Wang M (2019a). Informant discrepancies in assessments of psychosocial functioning in school-based services and research: Review and directions for future research. Journal of School Psychology, 74, 74–89. [DOI] [PubMed] [Google Scholar]
  14. De Los Reyes A, Ohannessian CM, & Racz SJ (2019b). Discrepancies between adolescent and parent reports about family relationships. Child Development Perspectives, 13, 52–58. [Google Scholar]
  15. Dirks MA, De Los Reyes A, Briggs–Gowan M, Cella D, & Wakschlag LS (2012). Annual research review: Embracing not erasing contextual variability in children’s behavior--theory and utility in the selection and use of methods and informants in developmental psychopathology. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 53(5), 558–574. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Eisenberg N, Zhou Q, Spinrad TL, Valiente C, Fabes RA, & Liew J (2005). Relations among positive parenting, children’s effortful control, and externalizing problems: A three-wave longitudinal study. Child Development, 76(5), 1055–1071. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Enders CK (2001). A primer on maximum likelihood algorithms available for use with missing data. Structural Equation Modeling, 8(1), 128–141. [Google Scholar]
  18. Fanti KA, & Henrich CC (2010). Trajectories of pure and co-occurring internalizing and externalizing problems from age 2 to age 12: Findings from the National institute of Child Health and Human Development study of early child care. Developmental Psychology, 46(5), 1159–1175. [DOI] [PubMed] [Google Scholar]
  19. Goolsby J, Rich BA, Hinnant B, Habayeb S, Berghorst L, De Los Reyes A, & Alvord MK (2018). Parent-child informant discrepancy is associated with poorer treatment outcome. Journal of Child and Family Studies, 27(4), 1228–1241. [Google Scholar]
  20. Grills AE, & Ollendick TH (2002). Issues in parent-child agreement: The case of structured diagnostic interviews. Clinical Child and Family Psychology Review, 5(1), 57–83. [DOI] [PubMed] [Google Scholar]
  21. Hurd NM, Hussain S, & Bradshaw CP (2018). School disorder, school connectedness, and psychosocial outcomes: Moderation by a supportive figure in the school. Youth & Society, 50(3), 328–350. [Google Scholar]
  22. Hurley KD, Huscroft–D’Angelo J, Trout A, Griffith A, & Epstein M (2014). Assessing parenting skills and attitudes: A review of the psychometrics of parenting measures. Journal of Child and Family Studies, 23(5), 812–823. [Google Scholar]
  23. Ines TM, & Sacco WP (1992). Factors related to correspondence between teacher ratings of elementary student depression and student self-ratings. Journal of Consulting and Clinical Psychology, 60(1), 140–142. [DOI] [PubMed] [Google Scholar]
  24. King SM, Iacono WG, & McGue M (2004). Childhood externalizing and internalizing psychopathology in the prediction of early substance use. Addiction, 99(12), 1548–1559. [DOI] [PubMed] [Google Scholar]
  25. Kolko DJ, & Kazdin AE (1993). Emotional/behavioral problems in clinic and nonclinic children: Correspondence among child, parent and teacher reports. Journal of Child Psychology and Psychiatry, 34(6), 991–1006. [DOI] [PubMed] [Google Scholar]
  26. Laursen BP, & Hoff E (2006). Person-centered and variable-centered approaches to longitudinal data. Merrill-Palmer Quarterly, 52(3), 377–389. [Google Scholar]
  27. Lionetti F, Palladino BE, Moses Passini C, Casonato M, Hamzallari O, Ranta M, Dellagiulia A, & Keijsers L (2018). The development of parental monitoring during adolescence: A meta-analysis. European Journal of Developmental Psychology. Advanced online publication. [Google Scholar]
  28. Lippold MA, Glatz T, Fosco GM, & Feinberg ME (2018). Parental perceived control and social support: Linkages to change in parenting behaviors during early adolescence. Family Process, 57(2), 432–447. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Lochman JE, & Conduct Problems Prevention Research Group. (1995). Screening of child behavior problems for prevention programs at school entry. Journal of Consulting and Clinical Psychology, 63(4), 549–559. [DOI] [PubMed] [Google Scholar]
  30. Masyn K (2013). Latent class analysis and finite mixture model In Little TD (Ed.), The Oxford handbook of quantitative methods: Statistical Analysis (pp. 551–611). New York, NY, US: Oxford University Press. [Google Scholar]
  31. McLeod BD, Weisz JR, & Wood JJ (2007). Examining the association between parenting and childhood depression: A meta-analysis. Clinical Psychology Review, 27(8), 986–1003. [DOI] [PubMed] [Google Scholar]
  32. Muthén LK and Muthén BO (1998–2017). Mplus user’s guide. Eighth Edition. Los Angeles, CA: Muthén & Muthén. [Google Scholar]
  33. Nagin DS (2005). Group-based modeling of development over the life course. Cambridge, MA: Harvard University Press. [Google Scholar]
  34. Nylund KL, Asparouhov T, & Muthén BO (2007). Deciding on the number of latent classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling: A Multidisciplinary Journal, 14(4), 535–569. [Google Scholar]
  35. Olino TM, & Klein DN (2015). Psychometric comparison of self-and informant-reports of personality. Assessment, 22(6), 655–664. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Pinquart M (2017). Associations of parenting dimensions and styles with externalizing problems of children and adolescents: An updated meta-analysis. Developmental Psychology, 53(5), 873–932. [DOI] [PubMed] [Google Scholar]
  37. Racz SJ, & McMahon RJ (2011). The relationship between parental knowledge and monitoring and child and adolescent conduct problems: A 10-year update. Clinical Child and Family Psychology Review, 14(4), 377–398 [DOI] [PubMed] [Google Scholar]
  38. Ramaswamy V, Desarbo WS, Reibstein DJ, & Robinson WT (1993). An empirical pooling approach for estimating marketing mix elasticities with PIMS data. Marketing Science, 12, 103–124. [Google Scholar]
  39. Reynolds CR, & Kamphaus RW (2002). The clinician’s guide to the Behavior Assessment System for Children (BASC). Guilford Press. [Google Scholar]
  40. Reynolds CR, & Kamphaus RW (2004). BASC-2: Behavior assessment system for children, second edition manual. Circle Pines, MN: American Guidance Service. [Google Scholar]
  41. Rolland RG (2012). Synthesizing the evidence on classroom goal structures in middle and secondary schools: A meta-analysis and narrative review. Review of Educational Research, 82(4), 396–435. [Google Scholar]
  42. Roskam I (2018). The Clinical Significance of Informant Agreement in Externalizing Behavior from Age 3 to 14. Child Psychiatry & Human Development, 49(4), 563–571. [DOI] [PubMed] [Google Scholar]
  43. Sarason IG, Levine HM, Basham RB, & Sarason BR (1983). Assessing social support: The social support questionnaire. Journal of Personality and Social Psychology, 44(1), 127–139. [Google Scholar]
  44. Scaramella LV, & Leve LD (2004). Clarifying parent-child reciprocities during early childhood: The early childhood coercion model. Clinical Child and Family Psychology Review, 7(2), 89–107. [DOI] [PubMed] [Google Scholar]
  45. Schwartz G (1978). Estimating the dimension of a model. Annals of Statistics, 6(2), 461–464. [Google Scholar]
  46. Sclove SL (1987). Application of model-selection criteria to some problems in multivariate analysis. Psychometrika, 52(3), 333–343. [Google Scholar]
  47. Shelton KK, Frick PJ, & Wootten J (1996). Assessment of parenting practices in families of elementary school-age children. Journal of Clinical Child Psychology, 25(3), 317–329. [Google Scholar]
  48. Sinclair SM, Schmidt F, Kowatch KR, & Mushquash CJ (2019). Mental health cross-informant agreement for indigenous and non-indigenous adolescents. Journal of Child and Family Studies, 28, 906–916. [Google Scholar]
  49. Totura CMW, Green AE, Karver MS, & Gesten EL (2009). Multiple informants in the assessment of psychological, behavioral, and academic correlates of bullying and victimization in middle school. Journal of Adolescence, 32(2), 193–211. [DOI] [PubMed] [Google Scholar]
  50. Van Heel M, Van Den Noortgate W, Bijttebier P, Colpin H, Goossens L, Verschueren K, & Van Leeuwen K (2019). Parenting and externalizing problem behavior in adolescence: Combining the strengths of variable-centered and person-centered approaches. Developmental Psychology, 55(3), 653–673. [DOI] [PubMed] [Google Scholar]
  51. Yeguez CE, & Sibley MH (2016). Predictors of informant discrepancies between mother and middle school teacher ADHD ratings. School mental health, 8(4), 452–460. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Zlomke KR, Lamport D, Bauman S, Garland B, & Talbot B (2014). Parenting adolescents: Examining the factor structure of the Alabama Parenting Questionnaire for adolescents. Journal of Child and Family Studies, 23(8), 1484–1490. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1050095-supplement-1.docx (44.2KB, docx)

RESOURCES