Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Apr 1.
Published in final edited form as: Child Psychiatry Hum Dev. 2015 Apr;46(2):308–319. doi: 10.1007/s10578-014-0472-4

Multi-domain predictors of Oppositional Defiant Disorder symptoms in preschool children: Cross-informant differences

John V Lavigne 1,2, Katharine P Dahl 1, Karen R Gouze 1,2, Susan A LeBailly 4, Joyce Hopkins 3
PMCID: PMC4284149  NIHMSID: NIHMS611418  PMID: 24997089

Abstract

Existing research suggests that parent and teacher reports of children's behavior problems are often discrepant. The current study examined whether contextual (stress and family conflict), parent (depression), parenting (hostility, support, and scaffolding), and child factors (receptive vocabulary; negative affect, NA; effortful control, EC; inhibitory control, IC; attachment; and sensory regulation, SR) are related to parent-teacher reporting discrepancies. Participants included a community sample of 344 4-year-old children. A multi-informant approach was used to assess contextual, parent, parenting, and child factors. Parents and teachers completed the Oppositional Defiant Disorder (ODD) scale of the Child Symptom Inventory. Consistent with previous data, there was poor agreement between parents and teachers (r =.17). After correcting for multiple comparisons, child effortful control, parent hostility, and family conflict were significant predictors of parent-rated symptoms of ODD symptoms but not teacher-rated ODD symptoms. Only family conflict was a significant predictor of discrepancies in parent and teacher ratings.

Keywords: young children, Oppositional Defiant Disorder, externalizing behavior, informant discrepancies

Oppositional Defiant Disorder (ODD), characterized by disobedient, hostile, and defiant behavior [1], is a common childhood disorder [2-4]. A large number of studies have examined the factors associated with the occurrence of the symptoms of ODD, ODD as a categorical disorder, or broad-band externalizing behavior problems that include ODD symptoms. The multi-level domains of factors associated with these symptoms include contextual, parental, parenting, and child characteristics [5]. Contextual factors associated with higher levels of ODD symptoms or externalizing symptoms include lower socioeconomic status (SES) [6, 7], and higher levels of parental stress [8] and family conflict [9, 10]. Parent and parenting characteristics associated with child externalizing and ODD symptoms include maternal depression [11, 12] and parental hostility [13], warmth [14] and scaffolding skills [15]. Child characteristics associated with either externalizing symptoms or ODD symptoms include higher levels of temperamental negative affect (NA), lower levels of effortful control (EC) [16] and inhibitory control (IC) [17], sensory regulation (SR) difficulties [18], and insecure attachment [19].

Based on these empirical findings, Campbell's model of externalizing symptoms [20], and Smeekens et al.'s [5] model of multiple domains of factors associated with externalizing symptoms, we (reference withheld) examined these factors in an integrated, multi-level, cross-sectional model of factors thought to be associated with parent-reported symptoms of ODD. Direct or indirect effects associated with higher levels of parent-rated ODD were found for each of the contextual factors, including: (a) lower SES, higher stress, and higher levels of family conflict; (b) higher parental depression; (c) parent factors, including support, hostility, and scaffolding skills; (d) child factors, including temperamental characteristics of NA, EC, SR, and insecure attachment.

The models and empirical studies described above were all concerned with factors associated with parent-rated ODD symptoms. Recent studies suggest that ratings of children's symptoms of ODD gathered from different informants are frequently discrepant. Drabick Gadow, and Loney [21], Munkvold, Lundervold, Lie, and Manger [22], and Korsch and Petermann [23] found poor agreement between mother and teacher ratings of ODD symptoms among school-age children. Drabick, Bubier, Chen, Price, and Lanza [24] found that this source-specificity is stable from one school grade to the next. Both Drabick et al. and Monkvold et al. argue that their results support an informant- or source-specific conceptualization of ODD. Such findings have led the Fifth Edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) [1] to note that ODD symptoms may be source-specific. While most rater-discrepancy studies have been conducted with school age children, the few studies of informant discrepancies with preschool children also find low parent-teacher agreement [25, 26].

While informant discrepancies have often attributed to measurement error, it is likely that some portion of the differences capture meaningful variation [27]. Children's behavior is known to differ across situations [28], and there is compelling evidence that these differences result from factors other than measurement error [21, 24]. In addition to measurement error, other factors may contribute to discrepancies in informants’ reports of child behavior across situations [24, 29], including informant characteristics and possibly other contextual, parent, and parenting, and child factors as noted above.

Informant characteristics

Considerable attention has been paid to the effects of parent characteristics on reports of child behavior. The depression-distortion hypothesis suggests that maternal depression biases mother's reports and inflates ratings of child behavior problems [30]. While the belief that maternal depression inflates reports of child problem behaviors has been based on comparisons of ratings of depressed mothers and teachers or other informants [31, 32], there is little evidence that rater differences result from distortion rather than genuine source- or situation- differences in behavior [30].

Contextual, family and parenting characteristics related to source-specific child ODD symptoms

Rather than being attributable to measurement error or informant bias, a wide range of contextual, parent, and parenting factors associated with ODD symptoms or disruptive behaviors may contribute to source-specific discrepancies in observed ODD symptoms. Several studies have examined whether some of the multi-domain factors associated with parent-reported ODD or externalizing symptoms noted by Smeekens and [withheld for blind review] are also associated with teacher-parent report discrepancies [21]. Most studies have focused on contextual factors, with some studies, but not others [33-35], finding that socioeconomic factors (SES) were related to parent-teacher discrepancies. In addition, higher levels of parent-reported stress [32, 36-38], poor family functioning, [39] and more family conflict [38] were associated with higher parent than teacher reports of behavior problems.

Results for parent factors are inconsistent and possibly related to the child's age. Age 4 ratings of maternal mental health were not associated with source-specificity of disruptive behavior at age 11 [40], while parental depression was unrelated to parent-teacher discrepancies in young children [38]. In contrast, Chi and Hinshaw [31] found maternal depression to be more strongly associated with parent- than teacher-reported externalizing problems in school-age children.

Among parenting practices, parental warmth and disciplinary practices were not associated with parent-teacher discrepancies in behavior problems reports for preschoolers [26], while parental use of punishment was associated with mother-reported but not teacher-reported ODD symptoms in 10-12 year olds [41]. Mothers of 6- to 10-year-old boys with high levels of mother-rated but not teacher-rated ODD symptoms reported greater maternal detachment, while maternal use of hostile control was associated with high levels of both mother- and teacher-rated ODD symptoms [21].

Child factors

Child characteristics interacting with situational demand characteristics could lead to actual differences in child behavior observed by parents and teachers because they tend to see the child in different situations [21, 24]. Many existing studies on informant discrepancies have focused on demographic characteristics, with inconsistent results for child age [42-44], gender [25, 29, 37], and race [32, 34, 45].

Studies of other child characteristics associated with source discrepancies have been concerned with cognitive processes and academic performance. Executive functioning (EF) deficits were associated with parent report of ODD symptom stability over 1 year [24], but were not associated with higher teacher-reported symptoms [4]. Two studies [46, 47] found that poorer academic functioning was associated with higher teacher-reported academic problems, but did not describe the relationship between academic functioning and rater discrepancies. In another report [48], poorer pre-academic skills were associated with higher levels of teacher-rated ADHD or ADHD/ODD but the relationship of pre-academic skills to ODD per se is unclear. Presently, the contributions to parent-teacher rating discrepancies of other child factors associated with parent-rated ODD symptoms that were described earlier, such as child temperamental EC, NA, and SR, as well as poorer child inhibitory control (IC) [49], insecure attachment [50]; and verbal skills intellectual or cognitive ability [40, 51] have not been examined.

The present study

The present report uses data from a large community study of risk factors and correlates associated with ODD symptoms (reference withheld) to examine whether factors from the four risk domains [5], described above, that were included in a cross-sectional, multi-domain model of factors associated with parent-rated symptoms of child ODD (reference withheld), were associated with source-specific manifestations of ODD symptoms in a sample of 4-year-old children. There was empirical support from prior work noted above that each of the variables was associated with parent-rated ODD or externalizing problems, The specific factors examined included the contextual factors of SES, life stress, and family conflict; parental depression; parenting factors, including parental hostility, support, and scaffolding skills; and child factors, including receptive vocabulary, negative affect, effortful control, inhibitory control, attachment, and sensory regulation symptoms. The inclusion of a wide range of predictors has the added advantage of reducing the “omitted variable” problem [52] and providing a better estimate of the strength of the association with ODD symptoms specific to each factor.

In the present study, we initially examined which factors were associated with behavior problems reported by parents or teachers. Subsequently, we examined whether the magnitudes of the factors that were associated with either parent or teacher ratings differed from one another. We hypothesized that each of the variables would be associated with both parent and teacher rated ODD, but would differ in magnitude. With few exceptions, existing studies have been conducted with school-age samples. An important characteristic of this study is the examination of variables contributing to source-specificity of ODD symptoms in preschoolers.

Methods

Participants

Study participants (N = 796) were recruited from 13 city, including 3 inner-city, schools and 23 pediatric practices, including 3 inner city clinics, in a Midwestern metropolitan area. Inclusion criteria were: (a) child age 4 years at the time of assessment; (b) child lived with his or her English-or Spanish-speaking parents or guardian for at least 6 months, (c) child scored ≥ 70 on a language screen, was not enrolled in a classroom for children with intellectual disabilities, did not score below 70 on a school IQ test, and (d) child did not meet criteria for an autism spectrum disorder.

Data for the present report comes from a subset of the children for whom both parent and teacher ratings of externalizing symptoms were available at age 4 years. Efforts to collect teacher-reports were challenging because children in the sample attended over 350 different programs. Participants in this study were 344 children (43.2% of the total study sample) with both parent and teacher report data available at age 4 (M = 4.42, SD = 0.33), of whom178 (51.7%) were boys. Racial/ethnic identity according to parent report was 217 (63.1%) White non-Hispanic, 46 (13.4%) African American, 54 (15.7%) Hispanic, 8 (2.3%) Asian, 14 (4.1%) two or more race/ethnic groups or other, and 5 (1.4%) not reporting. All five Hollingshead social classes [54] were represented (Class I and II (high), 279, 81.1%; Class III-V, 65, 18.9%). Children whose teachers completed the questionnaire were more likely to be White, 2(4) = 19.70, p <.001, and from the higher two social classes, 2(1) = 13.92, p <.001. There were no differences between groups in child's age, t = 1.69, p = .09, or gender, 2(1) = 1.67, p = .20.

Measures

A multi-informant approach was used, including parent report, teacher-report, direct observation and standardized psychological child assessment. There were multiple measures of some of the constructs (e.g., caregiver depression) that were converted to z scores and summed to create composite measures.

Contextual measures

SES

Parents provided information about their education and employment which was coded for SES as a continuous measure using the Hollingshead Four-Factor Index of Social Status [53]. Parents also provided other demographic information.

Life stress

A composite measure of life stress (composite measure reliability [54, 55] = .80; alphas and composite reliabilities reported below are for this study) was created from the (a) Parenting Stress Index- Short Form (PSI-SF) [56]; (b) Perceived Stress Scale [57]; and (c) McCubbin Family Changes and Strains Scale [58]. The PSI-SF is a 36-item self-report measure of parental distress; higher scores reflect greater stress (alpha = .45). The Perceived Stress Scale is a 14-item measure of the degree to which adults appraised life as stressful, unpredictable, and uncontrollable (alpha = .87). The McCubbin Family Changes and Strains Scale measures life stress (alpha = .44).

Family conflict

Parents completed the conflict scale of the Family Environment Scale (FES) [59], the McCubbin Family Distress Index [60], and the McCubbin Family Problem-Solving/Communication Scales [61]. Scores were combined to create a “family conflict” composite score. The FES conflict scale is a 9-item inventory measuring the family social climate (alpha = .28). This study included the family conflict subscale. The McCubbin Family Distress Index, an 8-item scale assessing family conflict ( alpha = .75) and has been found to be related to family pressures and communication problems [60]. The McCubbin Family Problem-Solving/Communication Scales, a 10-item scale (alpha = .30) assesses incendiary family communication patterns (e.g., “we yell and scream at each other”). While the alphas for two individual scales were low, the composite reliability for the conflict measure was .71.

Parent and parenting measures

Parental depression

Parents completed the Beck Depression Inventory (BDI) [62] and the Center for Epidemiological Studies- Depression Scale (CES-D) [63]. The BDI is a self-report measure of depression (alpha = .86) with high concurrent validity with other depression scales [64]. The CES-D (alpha = .55), correlates highly with other depression scales and differentiates depressed patients from non-depressed adults [63]. Alpha was .70 for the parental depression composite score.

Parent support and hostility

Parents completed the Parent Behavior Inventory (PBI) [65]. The PBI has two factor-analytically derived subscales, Support/Engagement (alpha = .90; test-retest = .74) and Hostility/Coercion (alpha = .87; test-retest = .69). In the present study, alpha was .85 for support/ engagement and .73; for hostility/coercion.

Parental sensivitiviy/scaffolding

The National Institute of Child and Human Development (NICHD) Three Boxes task [66], a semi-structured videotaped parent-child interaction paradigm, was used to assess sensitivity/scaffolding. This paradigm consists of three tasks, two of which are designed to be somewhat challenging for the child to complete without help, and a third that involves free-play. Parental behaviors were rated by trained research assistants on the following dimensions, supportive presence, respect for autonomy, quality of assistance, cognitive stimulation, confidence, and hostility (reverse scored). Each dimension was rated on a 7-point Likert scale (1 = “very low”, 2 = very high). Factor analysis of these dimensions revealed a one-factor solution that was used as a composite measure of sensitivity/scaffolding (alpha = .81). Coders were trained to a criterion of 80% reliability with two master coders by Margaret Tresh-Owen, Ph.D. of the NICHD study. Inter –rater reliabilities ranged from .80 for quality of assistance to .69 (likely deflated by a low base rate) for maternal hostility (mean reliability = .74).

Child Measures

Child temperament

Rothbart and her colleague's conceptualization of temperament [67] includes components of negative affect (NA) and effortful control (EC). While Rothbart et al. considered sensory regulation (SR) to be a component of both EC and NA, a recent study (withheld for blind review) found a better fit for a model in which SR, NA, and EC were viewed as separate components of temperament. Because EC is a “hot” measure of emotion regulation, a measure of inhibitory control (IC), a task in which the child must inhibit a prepotent cognitive or motor response to provide the less prepotent but correct response on a problem solving task arousing little emotion, was included as a “cool” measure of EF. Separate measures of NA, EC, SR, and IC were thus included in the present analyses. While statistically significant, the low correlation between EC and IC (r = −.25, p <.001) suggest they are best construed as independent measures.

Concern has been raised about the possibility that item contamination could inflate the relationships between measures of temperament (i.e., NA and EC) and behavior problems [16, 68] as well as measures of SR and behavior problems [69]. To address this, we followed procedures developed by Lengua et al. [70] using expert opinion and confirmatory factor analysis (CFA) to refine the temperament and sensory regulation measures described below. CFA shows that the factor structure is the same for the full set of items and those retained after reducing item contamination [reference withheld].

Child NA

Parents completed the Children's Behavior Questionnaire (CBQ) [67] which yields a measure of NA; the NA scale in this study was a composite measure calculated by summing the 7 items from the CBQ NA sub-scales that the experts felt were measures of NA rather than psychopathology. Alpha for the expert-rated NA scale composite is .62. While a cutoff of .70 is often used for assessing the adequacy of alpha, at times lower scores may be acceptable when the measure has other desirable properties [70] as in this study, in which the expert-rated NA items were distinct from SR and EC in the CFA noted above, and showed a good fit in the overall measurement model of risk factors for psychopathology (withheld for blind review).

Child EC

After reducing item contamination as noted above, there were three EC measures on the CBQ, attentional focusing and inhibitory control which were used to create a composite EC measure with an alpha of .72.

Child IC

Children were administered the Statue subtest from the Attention/Executive Function Domain of the Developmental Neuropsychological Assessment (NEPSY) [71]. The Statue subtest assesses motor inhibition by asking the child to maintain a body position for 75-seconds while inhibiting responses to sound distracters. Alpha for this measure in this study was .91.

Child SR

The Short Sensory Profile (SSP) [72] is a 38-item measure of SR (which refers to threshold and reactivity across all sensory domains) in children ages 3-10. SSP scores are significantly correlated with physiological measures of sensory processing and differ for groups of children with and without sensory processing difficulties [72]. The expert-rated items on the index of SR in the present study measured taste/smell, tactile, movement, visual/auditory sensitivity, and low energy/weakness (alpha = .82).

Attachment

The Attachment Q-Sort (AQS) [73], a continuous measure of attachment security, shows good convergent validity with the Strange Situation Paradigm [74]. In a 20% random sample, inter-rater reliability in the present study was .77.

Child receptive vocabulary

The Peabody Picture Vocabulary Test (PPVT-L) [75] is an individually-administered measure of single word receptive language skills. Internal consistency is .94 and test-retest reliability is .89 [76]. The PPVT correlates .88 with the Wechsler Intelligence Scale for Children verbal index [77].

ODD

Parents rated symptoms of ODD on the Child Symptom Inventory (CSI): Parent Checklist for young children [78] and teachers completed the CSI-4: Teacher Checklist. Items on the checklist are derived from DSM-IV diagnostic criteria. Parents and teachers rate symptom occurrence on a 4-point scale (from “never” to “very often”). In this sample, Cronbach's alpha for the ODD scale was .85 for the parent scale and .91 for the teacher scale.

Procedure

Families were recruited at 23 primary care pediatric practices, including 3 inner city clinics, and 13 Chicago Public School preschool programs. The school sample included 90% minority children. In pediatric practices, parents were approached individually in pediatric waiting rooms by study staff. Public school children were recruited by study staff members who approached parents at morning drop-off and school events that occurred during the enrollment period. Subsequently, research assistants scheduled a home visit and mailed questionnaires to interested families. At the home visit, research assistants administered study tasks (e.g., PPVT, Statue subtest, Q-Sort, Three Boxes Task) and additional questionnaires (including demographic questionnaire and CSI). After the visit, the CSI-Teacher form was mailed to the child's teacher. Institutional Review Boards at the author's institutions approved the procedures; informed consent was obtained.

Data Analysis

Because we had more than one measure for each of the variables, we considered two alternate approaches to measurement of the predictor and outcome factors. One approach would involve structural equation modeling with 1-3 manifest indicators for each latent factor. This approach was not chosen, however, because the sample size was not large enough. In a structural equation model using all the available latent factors, there would have been 188 model parameters (14 variances of exogenous variables, 13 measurement errors associated with manifest indicators not set to zero, 1 disturbance associated with the endogenous variable, 116 unanalyzed correlations between exogenous factors, 19 free factor loadings, and 14 direct effects on the endogenous variable). That is less than 2 subjects per parameter, when 10-20 is ideal and Kline [79] indicates that, at a minimum, there should be 5 subjects per parameter.

As a result, we used a regression-based approach that included the use of composite variables. There is clear precedent for creating composite variables by converting individual variables into standard scores and combining them. This approach was used by us in a previous report [80], as well as in papers in the fields of medicine [81], developmental psychology [82], epidemiology [83], and management [84]. This approach is warranted in constructing composite scales when the constituent items have different measurement response-scales and one wishes to weight the multiple items equally [84]. The use of composite score has been discussed in various books and websites [85, 86]. In discussing a study using composite scores, Nunnally and Bernstein [85] note that, in creating composite scores, variables are standardized before being summed; psychassessment.com.au [86] agrees, indicating that it is reasonable to create composites by converting the score on each test into a z-score, and then adding the individual z-scores to create the composite score.

In the preliminary analyses, Pearson correlations between predictors and source-specific outcomes were conducted. These correlations allow for examination of the relationships between each of the predictors and outcomes, but the magnitude of the correlations will be increased by any variance that is shared between predictors. With hierarchical linear regression, the contributions of specific predictors to ODD symptoms could be determined.

As noted above, studies of correlates of parent and teacher-reported symptoms of ODD have generally not examined a wide variety of contextual factors simultaneously, making it difficult to assess the specific associations of various factors with ODD symptoms at school or at home. For this reason, separate hierarchical linear regression analyses of teacher and parent-reported ODD symptoms were examined before examining factors associated with source specificity. Because it has been suggested that child characteristics in particular could be associated with ODD symptoms at school [21], child variables (EC, IC, SR, PPVT, NA, and attachment security) were entered first, followed by parenting (SE, hostile parenting, scaffolding skill), parent (caretaker depression), and contextual and demographic variables (stress, conflict, SES, race). Race as well as SES was included because the subsample including teacher reports differed from the total sample on those measures. To determine if possible suppression effects could be operating, we examined the Pearson correlations to determine if any of the predictor variables were highly correlated. If any pair of variables was highly correlated, separate analyses were conducted that included only one variable from each pair. With 14 variables and a sample size of 344, power to detect a significant R2 of .40 was .90. Because of the large number of comparisons, corrections for multiple comparisons were made using a sequential Sidak procedure [87]. When studying factors associated with parent-teacher discrepancies, De Los Reyes and Kazdin [88] recommend the use of the standardized difference score, i.e., converting parent and teacher ratings to z-scores and calculating the difference, and that approach was adopted for use in this report.

Results

Correlations

The correlations between predictor and outcomes variables are presented in Table 1. The correlation between parent and teacher ratings of ODD symptoms was very low (r = .17) but statistically significant because of the large sample size. Each predictor variable was significantly correlated with either parent or teacher ratings of ODD, or with both. Correlations were low to moderate in magnitude; for the largest correlation, the amount of variance in parent-rated ODD symptoms accounted for by EC was 18.5%. Similarly, correlations between predictor variable were low to moderate. One correlation, stress with caretaker depression was moderate to high, raising the possibility of suppressor effects if both variables were included in regression analyses.

Table 1.

Correlations between predictor variables and with parent- and teacher-rated ODD symptoms

Outcome 1. PODD 2. TODD 3. IC 4. EC 5. NA 6. AQS 7. SR 8. PPVT 9. Scaff 10. SE 11. HP 12. Dep 13. Str 14. Con
1. Parent ODD (PODD) 1
2. Teacher ODD (TODD) 17** 1
Child
3. Inhibitory control (IC) −0.1 −.14* 1
4. Effortful control (EC) −.43** −.18* .16** 1
5. Negative affect (NA) 37** .08 −.06 −.48** 1
6. Attachment (AQS) −.30** −.12* 27** .31** −.20** 1
7. Sensory regulation (SR) −.35** −.01 .11* .28** −.25** 19** 1
8. PPVT −.10 −.09 .13* .18** .12* 29** 17** 1
Parenting
9. Scaffolding (Scaff) −.14** −.19** .23** .22** −.08 .25** .10 .38** 1
10. Supportive-engagement (SE) −.15** −.09 .01 .32** −.22** .18** .28** 23** 23** 1
11. Hostile parent (HP) .36** .10 −.07 19** .12* −.19** −.20** −.13* −.12* −.09 1
Parent
12. Caretaker depression (Dep) .35* .06 −.13* −.20** .21** −.27** −.34** −.25** −.26** −.30** .29** 1
Contextual
13. Stress (Str) .32** .03 .04 −.22** .21** −.23** −.30** −.24** −.26** −.27** .23** .66** 1
14. Conflict (Con) .49** .03 .05 −.32** .24** −.29** −.30** −.12* −.05 −.04 .23** .33** 32** 1
15. Socioeconomic status (SES) −.02 −.11* .05 .13* −.03 .16** .13* .50** 40** .26** −.04 −.32** −.26** −.11*
Open in a new tab

Note

*

p <.05

**

p <.01

***p <.001. The amount of variance accounted for is the square of the Pearson correlation coefficient

Predictors of parent- and teacher-reported ODD symptoms

Parent-rated ODD symptoms

Table 2 summarizes the regression analyses for parent and teacher ratings. For parent-rated ODD symptoms, the overall R2 of .43 was significant, F = 17.82, p < .001 when both stress and caretaker depression were included as predictors. The R2 change score was significant for each block of variables. Before correcting for multiple comparisons, poorer child EC, more difficulties with SR, higher levels of NA, and greater caretaker hostility and family conflict were significant predictors of parent-rated ODD symptoms. Because of the possibility of suppression effects involving stress and caretaker depression, separate analyses were conducted that included stress but not depression, and vice versa. In these analyses, both caretaker depression and stress were significant predictors of parent-rated ODD. After correction for multiple comparisons, only child EC, parent hostility, and family conflict emerged as significant predictors of parent-rated ODD symptoms.

Table 2.

Regression analyses and comparisons of magnitudes of path coefficients

Block Variable Regression analyses Regression analyses Path coefficient (both free)
Parent ODD Teacher ODD Parent-teacher discrepancies
R2/R2 change Beta R2/R2 change Beta R2/R2 change Beta
Child .27*** .05** .10***
EC −0.18** − .13 * −.01
IC 0.01 −.08 .02
SR − 0.12 ** .07 −.06
NA 0.14 ** .01 .05
PPVT −0.02 .01 −.01
Parenting .08*** .02 .04*
SE 0.01 −.03 .02
Hostile 0.18*** .07 .04
Scaffold −0.08 − .13 * .02
Attachment −0.06 −.03 −.05
Parent .01** .00 .01
Caretaker depression (with stress) 0.07 .06 .08
Caretaker depression (without stress) .10 * .02 .08
Contextual .08*** .01 .06***
Stress (with depression) 0.06 −.08 .05
Stress (without depression) .10 * −.05 .07
Conflict 0.28*** −.03 .18***
SES 0.08 −.06 .01
Race (White v minority) −.10 −.02 −.29
Total .4.3*** .08* 46***
Open in a new tab

Note

*

p <.05

**

p <.01

***

p <.001. Italicized betas are not significant after correcting for multiple comparisons.

Teacher-rated ODD symptoms

For teacher-rated ODD, the overall R2 of .08 was small but significant, F = 1.972, p <.02. While the block of child variables was significantly associated with teacher-rated ODD, the inclusion of parenting, parent, and contextual factors did not improve the prediction of teacher-rated ODD. Prior to correcting for multiple comparisons, significant predictors were child EC and parent scaffolding skills, but neither one was significant after those corrections were made. In the analysis in which stress was not included, neither caretaker depression nor stress, respectively, were significant predictors of parent-rated ODD.

Predictors of discrepancies in parent- and teacher-rated ODD symptoms

For scores of discrepancies in parent-teacher ratings, the overall R2 of .46 was significant, F = 5.78, p < .001 when both stress and caretaker depression were included as predictors. The R2 change score was significant for parent and child factors but not for parenting. Before correcting for multiple comparisons, more difficulties with SR and family conflict were significantly associated with more parent-rated ODD symptoms. Neither stress nor caretaker depression were significant in separate analyses. After correction for multiple comparisons, only family conflict was a significant predictor of parent-teacher discrepancies in ratings of ODD symptoms.

Discussion

Results of the present study show that there is poor agreement between parent and teacher ratings of child ODD symptoms in preschoolers, a finding consistent with prior reports of studies of preschoolers [26, 89] and school-age children [21, 22, 24]. These consistent findings have implications for future studies: (a) parent- and teacher-ratings of ODD symptoms should not be considered to be equivalent; (b) the factors associated with the occurrence of ODD differ across situations observed by different raters.

Prior studies have found a wide variety of factors are associated with parent-reported ODD symptoms. Consistent with those studies, several of these factors in the present sample, including child EC, SR, and NA, parental hostility, family conflict, stress and caretaker depression were all significantly associated with parent-reported ODD symptoms prior to correcting for multiple comparisons. In addition, the associations of EC, family conflict, and caretaker hostility were significantly associated with parent-reported ODD symptoms after experiment-wise corrections. While these factors have been associated with parent-reported ODD symptoms, relatively few studies have examined their associations with teacher-reported ODD symptoms.

The results of the present study provide only limited support for the associations of several family and parent factors with teacher-rated ODD symptoms. In this study of young children, poorer child rated EC and poorer parenting scaffolding skills were associated with teacher-rated ODD, but neither variable was significant after correcting for multiple comparisons. The results of prior studies of school-age children are similar, with these studies also failing to find an association between teacher-rated externalizing problems or ODD symptoms. In those studies, maternal depression and family stress [32] as well as maternal mental health and home environment [40] were unrelated to teacher-rated externalizing problems, and findings for maternal hostility and hostile control [21] were inconsistent. Findings from this study suggest that factors associated with EC and parental scaffolding skills may be promising areas for future research on factors associated with ODD symptoms observed by teachers, but their association with teacher-rated ODD symptoms cannot be confirmed in this report because they were not significant after correcting for multiple comparisons.

With regard to studies examining factors associated with discrepancies in parent and teacher reports, prior studies found inconsistent results for SES, while family stress was associated with higher parent than teacher ratings of externalizing symptoms. In the present study, neither SES nor parental stress was associated with parent-teacher discrepancies. Because this study was conducted with young children and other studies were conducted primarily with older children, it may be that the effects of SES and parental stress on parent-teacher discrepancies do not emerge until children are older. Family conflict was the only variable associated with higher ratings of ODD symptoms at home than at school, a finding that is consistent with prior research.

Although Drabick et al. suggested that child factors may be associated with parent-teacher discrepancies in ratings of ODD symptoms, it is noteworthy that child factors were unrelated to reporting discrepancies in the young children in this sample. Prior studies found that executive functioning was unrelated to teacher reports of ODD symptoms in school-age children, while the present study found that inhibitory control was correlated with, but did not have a specific association with, teacher ratings of ODD.

While pre-academic skills were associated with ADHD or ADHD/ODD in one study of young children, receptive language was unrelated to teacher reports of ODD symptoms or parent-teacher discrepancies in the present study. Furthermore, the present study extended prior work on source discrepancies for ODD symptoms by examining the relationship of child temperament with parent-teacher discrepancies and found no relationship. While academic skills may be related to teacher reported ODD symptoms or parent-teacher rating discrepancies in older children, there is little evidence that child factors are associated with source-specificity for ODD symptoms in preschoolers.

In any study in which the same raters provide information concerning predictor and outcome variables, there is the possibility that common method variance (CMV) may inflate the associations between predictors and outcomes. CMV may have affected the results of this study as well as all prior studies of parent-teacher discrepancies. Generally, one approach to addressing this problem in studies of developmental psychopathology has involved the use of multiple raters, with different raters for predictor than outcome variables. This is difficult to do when studying ODD symptoms in young children because: (a) evidence from multiple studies suggest that ODD symptoms are source-specific and ratings by teachers and parents are not interchangeable; (b) preschooler's self-reports are not a viable option for measuring ODD symptoms; (c) observer ratings of the predictor variables such as family conflict, stress, caretaker depression, etc. are not available for use as predictors of parent-rated ODD symptoms.

Because of these limitations, statistical corrections provide the next best alternative. As noted earlier, there has been concern that maternal depression biases ratings of child behavior problems, so the inclusion of caretaker depression as a predictor in these regression analyses allows for an assessment of the contributions of other predictors independent of caretaker depression, thereby reducing the effects of that possible source of bias. It should also be noted that the relationship between maternal depression and parent-rated externalizing problems was not significant for ODD symptoms. Of course, CMV was not present when examining the association of parent- and observer-rated predictors of teacher-reported ODD symptoms.

There are other limitations to the present study. First, while all races and SES groups were represented, teachers from more prosperous school systems were more likely to return forms. As a result, the children in this sample were primarily White and middle- to upper-middle class, and results may differ with other race/ethnic and SES groups.

Second, this study examined contextual variables related to the parent and home. Several studies have examined teacher and/or classroom specific factors that are more closely related to teacher than to parent ratings of externalizing behaviors. Verhulst and Akkerhuis [46] found that, compared to parents’, teachers’ ratings are more closely related to child peer relationships and classroom academic functioning. Similarly, Harvey and colleagues [48] found that teachers were more likely than mothers to rate preschoolers low on attention problems if they had higher pre-academic skills. Teacher, but not parent, reports of preschooler's externalizing behaviors were associated with observer ratings of children's sustained classroom activity and, inappropriate classroom behavior, and negative affect displayed in the classroom [90]. Finally, Berg-Nielsen and colleagues [38] found that higher teacher than parent ratings of preschooler's externalizing behaviors were directly related to teacher-reported conflict with the child. Future studies should incorporate informant and situational characteristics related to the teacher and school, such as number of years of experience of the teacher, how well the teacher knows the child, and other personal information about the teacher (e.g., stress level, classroom disciplinary style).

Despite these shortcomings, this study contributes to the literature on informant discrepancies by examining the relationship between specific contextual, parent, parenting and child characteristics and reports of children's behavior at school and home. The results support the likelihood that ODD symptoms are source specific, and that few family factors are associated with teacher reported ODD symptoms. Clinical implications of this are: (a) that teacher, as well as parent ratings are essential to fully understand a child's behavior and, more specifically, that child's ability to be successful in a given setting; and (b) that prevention and intervention efforts should be tailored to the specific setting in which the externalizing behavior is manifest.

Summary

Existing research suggests that parent and teacher reports of children's externalizing behavior problems, such as symptoms of ODD, are often discrepant. Because clinicians rely on parent and teacher reports to assess ODD symptoms in young children, informant discrepancies may significantly impact the classification and treatment of these problems, and more information is needed about the factors contributing to discrepancies in parent and teacher ratings. The current study examined whether contextual (stress and family conflict), parent (depression), parenting (hostility, support, and scaffolding), and child factors (receptive vocabulary; negative affect, NA; effortful control, EC; inhibitory control, IC; attachment; and sensory regulation, SR) are related to parent-teacher reporting discrepancies. Participants included a community sample of 344 4-year-old children. Consistent with prior studies, there was poor agreement between parents and teachers on ratings of ODD symptoms. Prior to correcting for multiple comparisons, a number of the multi-domain factors were associated with both parent and teacher ratings of ODD symptoms. After correcting for multiple comparisons, however, child effortful control, parent hostility, and family conflict were significant predictors of parent-rated symptoms of ODD symptoms but not teacher-rated ODD symptoms. Only family conflict was a significant predictor of the discrepancies in ratings between parents and children. Further research is needed to identify school-specific factors that might lead to discrepancies in ratings of ODD symptoms in children.

Acknowledgments

This research was supported by National Institute of Mental Health grant MH 063665.

References

  • 1.American Psychiatric Association . Diagnostic and statistical manual of mental disorders. 5th ed. Washington: 2013. Author. [Google Scholar]
  • 2.Boylan K, Vaillancourt T, Boyle M, Szatmari P. Comorbidity of internalizing disorders in children with oppositional defiant disorder. Eur Child Adolesc Psychiatry. 2007;16:484–494. doi: 10.1007/s00787-007-0624-1. [DOI] [PubMed] [Google Scholar]
  • 3.Lavigne JV, LeBailly SA, Hopkins J, Gouze KR, Binns HJ. The prevalence of ADHD, ODD, depression and anxiety in a community sample of 4-year-olds. J Clin Child Adolesc Psychol. 2009;38:315–328. doi: 10.1080/15374410902851382. [DOI] [PubMed] [Google Scholar]
  • 4.Nock MK, Kazdin AE, Hiripi E, Kessler RC. Lifetime prevalence, correlates, and persistence of oppositional defiant disorder: Results from the National Comorbidity Survey Replication. J Child Psychol Psychiatry. 2007;48:703–713. doi: 10.1111/j.1469-7610.2007.01733.x. [DOI] [PubMed] [Google Scholar]
  • 5.Smeekens S, Riksen-Walraven JM, van Bakel HJA. Multiple determinants of externalizing behavior in 5-year-olds: a longitudinal model. J Abnorm Child Psychol. 2007:347–361. doi: 10.1007/s10802-006-9095-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Greene RW, Biederman J, Zerwas S, Monuteaux MC, Goring JC, Faraone SV. Psychiatric comorbidity, family dysfunction, and social impairment in referred youth with Oppositional Defiant Disorder. Am J Psychiatry. 2002;159:1214–1224. doi: 10.1176/appi.ajp.159.7.1214. [DOI] [PubMed] [Google Scholar]
  • 7.McLeod JD, Shanahan MJ. Poverty, parenting, and children's mental health. Am Sociol Rev. 1993;58:351–366. [Google Scholar]
  • 8.McMahon SD, Grant KE, Compas BE, Thurm AE, Ey S. Stress and psychopathology in children and adolescents: is there evidence of specificity? J Child Psychol Psychiatry. 2003;44:107–133. doi: 10.1111/1469-7610.00105. [DOI] [PubMed] [Google Scholar]
  • 9.Zimet DM, Jacob T. Influences of marital conflict on child adjustment: Review of theory and research. Clin Child Fam Psychol Rev. 2001;4:319–335. doi: 10.1023/a:1013595304718. [DOI] [PubMed] [Google Scholar]
  • 10.Grant KE, Compas BE, Thurm AE, McMahon SD, Gipson PY, Campbell AJ, Krochock K, Weserholm RI. Stressors and child and adolescent psychopathology: Evidence of moderating and mediating effects. Clin Psychol Rev. 2006;26:257–283. doi: 10.1016/j.cpr.2005.06.011. [DOI] [PubMed] [Google Scholar]
  • 11.Kim-Cohen J, Moffitt TE, Taylor A, Pawlby SJ, Caspi A. Maternal depression and children's antisocial behavior. Arch Gen Psychiatry. 2005;62:173–181. doi: 10.1001/archpsyc.62.2.173. [DOI] [PubMed] [Google Scholar]
  • 12.Goodman SH, Rouse MH, Connell AM, Broth MR, Hall CM, Heyward D. Maternal depression and child psychopathology: a meta-analytic review. Clin Child Fam Psychol Rev. 2011;14:1–27. doi: 10.1007/s10567-010-0080-1. [DOI] [PubMed] [Google Scholar]
  • 13.Heller TL, Baker BL. Maternal negativity in children's externalizing behavior. Early Educ Dev. 2000;11:483–498. [Google Scholar]
  • 14.Hipwell A, Keenan K, Kasza K, R L, Stouthamer-Loeber M, Bean T. Reciprocal influences between girls’ conduct problems and depression, and parental punishment and warmth: a six year prospective analysis. J Abnorm Child Psychol. 2008;36:663–677. doi: 10.1007/s10802-007-9206-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Hoffman C, Crnic KA, Baker JK. Maternal depression and parenting: Implications for children's emergent emotion regulation and behavioral functioning. Parent Sci Prac. 2006;6:271–295. [Google Scholar]
  • 16.Eisenberg N, Valiente C, Spinrad TL, Cumberland A, Liew J, Reiser M, Zhou Q, Losoya SH. Longitudinal relations of children's effortful control, impulsivity, and negative emotionality to their externalizing, internalizing, and co-occurring behavior problems. Dev Psychol. 2009;45:988–1008. doi: 10.1037/a0016213. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Zeman J, Cassano M, Perry-Parrish C, Stegall S. Emotion regulation in children and adolescents. J Dev Behav Pediatr. 2006;27:155–168. doi: 10.1097/00004703-200604000-00014. [DOI] [PubMed] [Google Scholar]
  • 18.Gouze KR, Hopkins J, LeBailly SA, Lavigne JV. Re-examining the epidemiology of sensory regulation dysfunction and comorbid psychopathology. J Abnorm Child Psychol. 2009;37:1077–1087. doi: 10.1007/s10802-009-9333-1. [DOI] [PubMed] [Google Scholar]
  • 19.DeVito C, Hopkins J. Attachment, parenting, and marital dissatisfaction as predictors of disruptive behavior in preschoolers. Dev Psychopathol. 2001;13:215–231. doi: 10.1017/s0954579401002024. [DOI] [PubMed] [Google Scholar]
  • 20.Campbell SB. Behavior problems in preschool children: Clinical and developmental issues. Guilford; New York: 1990. [Google Scholar]
  • 21.Drabick DAG, Gadow KD, Loney J. Source-specific oppositional defiant disorder: Comorbidity and risk factors in referred elementary schoolboys. J Am Acad Child Adolesc Psychiatry. 2007;46:92–101. doi: 10.1097/01.chi.0000242245.00174.90. [DOI] [PubMed] [Google Scholar]
  • 22.Munkvold L, Lundervold A, Lie SA, Manger T. Should there be separate parent and teacher-based categories of ODD? Evidence from a general population. J Child Psychol Psychiatry. 2009;50:1264–1272. doi: 10.1111/j.1469-7610.2009.02091.x. [DOI] [PubMed] [Google Scholar]
  • 23.Korsch F, Petermann F. Agreement between parents and teachers on preschool children's behavior in a clinical sample with externalizing behavior problems. Child Psychiatry Hum Dev. 2013 doi: 10.1007/s10578-013-0430-6. doi 10.1007/s10578-013-0430-6. [DOI] [PubMed] [Google Scholar]
  • 24.Drabick DAG, Bubier JL, Chen D, Price J, Lanza H. Source-specific oppositional defiant disorder among inner-city children: Prospective prediction and moderation. J Clin Child Adolesc Psychol. 2011;40:23–35. doi: 10.1080/15374416.2011.533401. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Strickland J, Hopkins J, Keenan K. Mother-teacher agreement on preschoolers’ symptoms of ODD and CD: Does context matter? J Abnorm Child Psychol. 2012;40:933–943. doi: 10.1007/s10802-012-9622-y. [DOI] [PubMed] [Google Scholar]
  • 26.Gross D, Fogg L, Garvey C, Julion W. Behavior problems in young children: An analysis of cross-informant agreements and disagreements. Res Nurs Health. 2004;27:413–425. doi: 10.1002/nur.20040. [DOI] [PubMed] [Google Scholar]
  • 27.De Los Reyes A. Introduction to the special section: More than measurement error: Discovering meaning behind informant discrepancies in clinical assessments of children and adolescents. J Clin Child Adolesc Psychol. 2011;40:1–9. doi: 10.1080/15374416.2011.533405. [DOI] [PubMed] [Google Scholar]
  • 28.Mischel W. Personality and Assessment. Wiley; New York: 1968. [Google Scholar]
  • 29.De Los Reyes A, Kazdin AE. Informant discrepancies in the assessment of childhood psychopathology: a critical review, theoretical framework, and recommendations for further study. Psychol Bull. 2005;131:483–509. doi: 10.1037/0033-2909.131.4.483. [DOI] [PubMed] [Google Scholar]
  • 30.Richters JE. Depressed mothers as informants about their children: A critical review of the evidence for distortion. Psychol Bull. 1992;112:485–499. doi: 10.1037/0033-2909.112.3.485. [DOI] [PubMed] [Google Scholar]
  • 31.Chi TC, Hinshaw SP. Mother-child relationships of children with ADHD: The role of maternal depressive symptoms and depression-related distortions. J Abnorm Child Psychol. 2002;30:387–400. doi: 10.1023/a:1015770025043. [DOI] [PubMed] [Google Scholar]
  • 32.Youngstrom EA, Loeber R, Stouthamer-Loeber M. Patterns and correlates of agreement between parent, teacher, and male adolescent ratings of externalizing and internalizing problems. J Consult Clin Psychol. 2000;68:1038–1050. doi: 10.1037//0022-006x.68.6.1038. [DOI] [PubMed] [Google Scholar]
  • 33.Stone SL, Speltz ML, Collett B, Werler MM. Socioeconomic factors in relation to discrepancy in parent versus teacher ratings of child behavior. J Psychopathol Behav Assess. 2013;35:314–320. doi: 10.1007/s10862-013-9348-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Duhig AM, Renk K, Epstein MK, Phares V. Interparental agreement on internalizing, externalizing, and total behavior problems: A meta-analysis. Clin Psychol Sci Prac. 2000;7:435–453. [Google Scholar]
  • 35.Treutler CM, Epkins CC. Are discrepancies among child, mother, and father reports on children's behavior related to parents’ psychological symptoms and aspects of parent-child relationships? J Abnorm Child Psychol. 2003;31:13–27. doi: 10.1023/a:1021765114434. [DOI] [PubMed] [Google Scholar]
  • 36.Jensen PS, Xenakis SN, Davis H, Degroot J. Child psychopathology rating scales and interrater agreement: II. Child and family characteristics. J Am Acad Child Adolesc Psychiatry. 1988;27:451–461. doi: 10.1097/00004583-198807000-00013. [DOI] [PubMed] [Google Scholar]
  • 37.Kolko DJ, Kazdin AE. Emotional/behavioral problems in clinic and nonclinic children: correspondence among child, parent and teacher reports. J Child Psychol Psychiatry. 1993;34:991–1006. doi: 10.1111/j.1469-7610.1993.tb01103.x. [DOI] [PubMed] [Google Scholar]
  • 38.Berg-Nielsen TS, Solheim E, Belsky J, Wichstrom L. Preschoolers’ psychosocial problems: In the eyes of the beholder? Adding teacher characteristics as determinants of discrepant parent-teacher reports. Child Psychiatry Hum Dev. 2013;43:393–413. doi: 10.1007/s10578-011-0271-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Collishaw S, Goodman R, Ford T, Rabe-Hesketh S, Pickles A. How far are associations between child, family and community factors and child psychopathology informant-specific and informant-general? J Child Psychol Psychiatry. 2009;50:571–580. doi: 10.1111/j.1469-7610.2008.02026.x. [DOI] [PubMed] [Google Scholar]
  • 40.Burt KB, Hay DF, Pawlby SJ, Harold G, Sharp D. The prediction of disruptive behaviour disorders in an urban community sample: the contribution of person-centred analyses. J Child Psychol Psychiatry. 2004;45:1159–1170. doi: 10.1111/j.1469-7610.2004.00308.x. [DOI] [PubMed] [Google Scholar]
  • 41.Drabick DAG, Gadow KD, Carlson GA, Bromet EJ. ODD and ADHD symptoms in Ukranian children: external validators and comorbidity. J Am Acad Child Adolesc Psychiatry. 2004;43:735–743. doi: 10.1097/01.chi.0000120019.48166.1e. [DOI] [PubMed] [Google Scholar]
  • 42.Grills AE, Ollendick TH. Issues in parent-child agreement: The case of structured diagnostic interviews. Clin Child Fam Psychol Rev. 2002;5:57–83. doi: 10.1023/a:1014573708569. [DOI] [PubMed] [Google Scholar]
  • 43.Achenbach TM, McConaughy SH, Howell CT. Child/adolescent behavioral and emotional problems: Implications of cross-informant correlation for situational specificity. Psychol Bull. 1987;101:213–232. [PubMed] [Google Scholar]
  • 44.Choudhury MS, Pimentel SS, Kendall PC. Childhood anxiety disorders: Parent-child (dis) agreement using a structured interview for the DSM-IV. J Am Acad Child Adolesc Psychiatry. 2003;42:957–964. doi: 10.1097/01.CHI.0000046898.27264.A2. [DOI] [PubMed] [Google Scholar]
  • 45.Walton JW, Johnson SB, Algina J. Mother and child perceptions of child anxiety: Effects of race, health status, and stress. J Pediatr Psychol. 1999;24:29–39. [Google Scholar]
  • 46.Verhulst FC, Akkerhuis GW. Agreement between parents’ and teachers’ ratings of behavioral/emotional problems of children aged 4-12. J Child Psychol Psychiatry. 1989;30:123–136. doi: 10.1111/j.1469-7610.1989.tb00772.x. [DOI] [PubMed] [Google Scholar]
  • 47.Zahner GEP, Daskalaks C. Modelling sources of informant variance in parent and teacher ratings of child psychopathology. Int J Methods Psychiatric Res. 1998;7:3–16. [Google Scholar]
  • 48.Harvey EA, Fischer C, Weieneth JL, Hurwitz SD, Sayer AG. Predictors of discrepancies between informants’ ratings of preschool-aged children's behavior: An examination of ethnicity, child characteristics, and family functioning. Early Child Res Q. 2013;38:668–682. doi: 10.1016/j.ecresq.2013.05.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Zeman J, Cassano M, Perry-Parish C, Stegall S. Emotion regulation in children and adolescents. J Dev Behav Pediatr. 2006;27:155–168. doi: 10.1097/00004703-200604000-00014. [DOI] [PubMed] [Google Scholar]
  • 50.Moss E, St-Laurent D. Attachment at school age and academic performance. Dev Psychol. 2001;37:836–874. [PubMed] [Google Scholar]
  • 51.Dietz KR, Lavigne JV, Arend R, Rosenbaum D. Relation between intelligence and psychopathology among preschoolers. J Clin Child Psychol. 1997;26:99–107. doi: 10.1207/s15374424jccp2601_10. [DOI] [PubMed] [Google Scholar]
  • 52.Tomarken AJ, Waller NG. Potential problems with “well fitting” models. J Abnorm Psychol. 2003;112:578–598. doi: 10.1037/0021-843X.112.4.578. [DOI] [PubMed] [Google Scholar]
  • 53.Hollingshead AB. Four-factor Index of Social Position. Yale University Department of Sociology; New Haven, CT: 1975. [Google Scholar]
  • 54.Evans LD. A two-score composite program for combining standard scores. Behav Res Methods. 1996;28:209–213. [Google Scholar]
  • 55.Mosier CI. On the reliability of a weighted composite. Psychometrika. 1943;8:161–168. [Google Scholar]
  • 56.Abidin RR. Manual for the parenting stress index. Psychological Assessment Resources; Odessa, FL: 1995. [Google Scholar]
  • 57.Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24:385–396. [PubMed] [Google Scholar]
  • 58.McCubbin HI, Patterson J. In: Family member well-being. In Family assessment: Resiliency, coping and adaptation -Inventories for research and practice. McCubbin HI, Thompson A, McCubbin MA, editors. University of Wisconsin; Madison: 1996. pp. 735–782. [Google Scholar]
  • 59.Moos RH, Moos BS. Family environment scale manual. Consulting Psychologists Press; Palo Alto, CA: 1981. [Google Scholar]
  • 60.McCubbin HI, Thompson A, Elver KM. In: Family distress index. in family assessment: resiliency, coping, and adaptation-inventories for research and practice. McCubbin HI, Thompson A, McCubbin MA, editors. University of Wisconsin; Madison: 1996. pp. 783–788. [Google Scholar]
  • 61.McCubbin MA, McCubbin HI, Thompson A. In: Family problem solving and communication. in family assessment: resiliency, coping,and adaptation-inventories for research and practice. McCubbin HI, Thompson A, McCubbin MA, editors. University of Wisconsin; Madison: 1996. pp. 639–686. [Google Scholar]
  • 62.Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4:561–571. doi: 10.1001/archpsyc.1961.01710120031004. [DOI] [PubMed] [Google Scholar]
  • 63.Radloff LS. the CES-D scale. Applied Psycholl Meas. 1977;1:385–401. [Google Scholar]
  • 64.Beck AT, Steer RA, Garbin MG. Psychometric properties of the Beck Depression Inventory: Twenty-five years of evaluation. Clin Psychol Rev. 1988;8:77–100. [Google Scholar]
  • 65.Lovejoy MC, Weis R, O'Hare E, Rubin EC. Development and initial validation of the Parent Behavior Inventory. Psychol Assess. 1999;4:1–12. [Google Scholar]
  • 66.NICHD Early Childhood Research Network Child care and mother-child interaction in the first three years of life. Dev Psychol. 1999;35:1399–1413. [PubMed] [Google Scholar]
  • 67.Rothbart MK, Ahadi SA, Hershey KL, Fisher P. Investigations of temperament at three to seven years: the children's behavior questionnaire. Child Dev. 2001;72:1394–1408. doi: 10.1111/1467-8624.00355. [DOI] [PubMed] [Google Scholar]
  • 68.Lemery KS, Essex MJ, Smider NA. Revealing the relation between temperament and behavior problem symptoms by eliminating measurement confounding: expert ratings and factor analyses. Child Dev. 2002;73:867–882. doi: 10.1111/1467-8624.00444. [DOI] [PubMed] [Google Scholar]
  • 69.Ben-Sasson A, Cermak SA, Orsmond GI, Tager-Flusberg H, Carter AS, Kadlec MB, Dunn W. Extreme sensory modulation behaviors in toddlers with austism spectrum disorders. Am J Occup Ther. 2007;61:584–592. doi: 10.5014/ajot.61.5.584. [DOI] [PubMed] [Google Scholar]
  • 70.Lengua LJ, West SG, Sandler IN. Temperament as a predictor of symptomatology in children: Addressing contamination of measures. Child Dev. 1998;69:164–181. [PubMed] [Google Scholar]
  • 71.Korkman M, Kirk U, Kemp S. NEPSY-II: clinical and interpretive manual. The Psychological Corporation; San Antonio, TX: 2007. [Google Scholar]
  • 72.McIntosh D, Miller L, Shyu V, Dunn W. The sensory profile: examiner manual. Psychological Corporation; San Antonio: 1999. Overview of the short sensory profile (SSP) [Google Scholar]
  • 73.Waters E. Attachment Q-set (version 3.0) State University of New York at Stony Brook; Stony Brook, NY: 1987. [Google Scholar]
  • 74.van IJzendoorn M, Vereijken C, Bakermans-Kranenburg M, Riksen-Walraven J. Assessing attachment security with the attachment q-sort: meta-analytic evidence for the validity of the observer AQS. Child Dev. 2004;75:1188–1213. doi: 10.1111/j.1467-8624.2004.00733.x. [DOI] [PubMed] [Google Scholar]
  • 75.Dunn LM, Service AG. Examiner's Manual for the PPVT-III, Peabody Picture Vocabulary Test, AGS. 1997 [Google Scholar]
  • 76.Dunn L, Dunn L. The peabody picture vocabulary test. 3rd Edition American Guidance Service; Circle Pines, MN: 1997. [Google Scholar]
  • 77.Hodapp A, Gerken K. Correlations between scores for Peabody picture vocabulary test-III and the Wechsler intelligence scale for children-III. Psychol Rep. 1999;84:1139–1142. [Google Scholar]
  • 78.Gadow KD, Sprafkin J. Screening Manual. Checkmate Plus; Stonybrook, NY: 2000. Early Childhood Symptom Inventory-4. [Google Scholar]
  • 79.Kline RB. Principles and Practice of Structural Equation Modeling. Guilford; New York: 1998. [Google Scholar]
  • 80.Lavigne JV, Hopkins J, Gouze KR, Bryant FB, LeBailly SA, Binns HJ, Lavigne PM. Is smoking during pregnancy a risk factor for psychopathology in young children? A methodological caveat and report on preschoolers. J Ped Psychol. 2011;36(1):10–24. doi: 10.1093/jpepsy/jsq044. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 81.Andres PL, Finison LJ, Conlon T, Thibodeau LM, Mursat TL. Use of composite scores (megascores) to measure deficit in amyotrophic lateral sclerosis. Neurology. 1988;38:405–408. doi: 10.1212/wnl.38.3.405. [DOI] [PubMed] [Google Scholar]
  • 82.Conley C, Travers L, Bryant FB. Promoting psychosocial adjustment and stress management in first-year college students: The benefits of engagement in a psychosocial wellness seminar. J Am Coll Health. 2013;61:75–86. doi: 10.1080/07448481.2012.754757. [DOI] [PubMed] [Google Scholar]
  • 83.Save the Children . State of the World's Mothers 2004: Children Having Children. Save the Children Foundation; Westport CT: 2004. [Google Scholar]
  • 84.Deshpande SP, Fiorito J. Specific and general beliefs in union voting models. Acad Manage J. 1989;32:883–897. [Google Scholar]
  • 85.Nunnally JL, Bernstein IH. Psychometric Theory. 3rd ed. McGaw-Hill; New York: 1994. [Google Scholar]
  • 86. Psychassessment.com.au.
  • 87.Yarnold PR, Soltysik RC. Optimal Data Analysis: A guidebook with software for Windows. American Psychological Association; Washington, DC: 2005. [Google Scholar]
  • 88.De Los Reyes A, Kazdin AE. Measuring informant discrepancies in clinical child research. Psychol Assess. 2004;16:330–334. doi: 10.1037/1040-3590.16.3.330. [DOI] [PubMed] [Google Scholar]
  • 89.Strickland J, Keller J, Lavigne JV, Gouze KR, Hopkins J, LeBailly SA. The structure of psychopathology in a community sample of preschoolers. J Abnorm Child Psychol. 2011;39:601–610. doi: 10.1007/s10802-011-9487-5. [DOI] [PubMed] [Google Scholar]
  • 90.Winsler A, Wallace GL. Behavior problems and social skills in preschool children: Parent-teacher agreement and relations with classroom observations. Early Educ Dev. 2002;13:41–58. [Google Scholar]

RESOURCES