Context specificity in stability of hyperactivity-impulsivity

Abstract

This paper exemplifies a secondary data analysis of context-specific differences in children’s hyperactivity-impulsivity while controlling for informant-specific effects. Participants were boys and girls from the NICHD Study of Early Child Care and Youth Development who were measured in 1^st, 3^rd, and 5^th grades. Latent factor models were structured using multi-informant reports including mothers, fathers, teachers, and observers. Temporal stability within a context was stronger than cross-context consistency, and the magnitude of longitudinal stability was higher in the home context compared to the school context. Controlling for informant-specific effects resulted in a significantly improved model fit and increased within-context stability. Our findings highlight the importance of considering both context and informant effects when studying longitudinal stability and change in personality development.

Many developmental theorists agree that temperament represents underlying dispositions, and possibly traits. Individual differences in temperament are evident as behaviors that vary widely across individuals, are readily observed from early in life, are somewhat stable over time and across settings, and include a biological foundation (Buss & Plomin, 1984; Goldsmith, Buss, & Lemery, 1997; Rothbart & Bates, 1998; Sanson, Hemphill, & Smart, 2004; Strelau, Zawadzki, & Piotrowska, 2001). Understanding how individual differences in temperament—and related personality factors—unfold over time is greatly enhanced by its accurate assessment aided by using multiple measures that span at least two contexts and that involve observations by multiple informants. In the current quantitative longitudinal study, we used factor analytic models with such data to examine the potential effects of contexts and informants on longitudinal stability and change in two indicators of child behavior that are related to individual differences in temperament, personality and psychopathology: hyperactivity and impulsivity.

Temperament is one of the bases of personality, the latter being a much broader array of attributes including perceptions and interests (Keogh, 2004; Rothbart, Ahadi, & Evans, 2000). Typically, developmental theorists state that temperament represents underlying dispositions or traits. However, some theorists have focused on individual differences in temperament reflect mainly characteristic styles of behavioral response (Buss & Plomin, 1975; Thomas & Chess, 1977) whereas other theorists view temperament as including individuals’ affective qualities (Rothbart, Ahadi, Hershey, & Fisher, 2001; Rothbart & Bates, 1998). In the further development of the operationalization of child temperament, Rothbart and her colleagues suggested the importance of individual differences in attentional self-regulation as a basic dimension of temperament (Posner & Rothbart, 1998; Rothbart et al., 2000) and demonstrated evidence for three distinct dimensions of child temperament – surgency/extraversion, negative affectivity, and effortful control (Rothbart et al., 2001). Surgency/extraversion captures “approach” tendencies and includes activity level, impulsivity, high intensity pleasure, and low shyness. Negative affectivity includes sadness, anger, discomfort, and low soothability. Effortful control includes enjoyment of low-intensity stimulation, perceptual sensitivity, and self-control of impulses and attention.

There is evidence of context and method/informant effects in child temperament (Sanson et al., 2004). Agreement between different informants on measures of temperament is far less than unity, with correlations ranging from .15 to .51 between maternal and observer ratings, and between mothers’ and fathers’ ratings (Bornstein, Gaughran, & Segui, 1991; Carnicero, Perez-Lopez, Salinas, & Martinez-Fuentes, 2000; Goldsmith, 1996; Goldsmith & Campos, 1990; Hayden, Klein, & Durbin, 2005; Rothbart et al., 2001; Seifer, Sameroff, Barrett, & Krafchuk, 1994). This modest to moderate agreement between informants could arise from informants using different bases of knowledge when evaluating child behavior (De Los Reyes & Kadzin, 2005), but it also could indicate that they are reliably reporting children’s context-specific behaviors (Mangelsdorf, Schoppe, & Buur, 2000; Beenstra et al., 2008). It might also arise from informant-specific “rater bias”. For example, depression is associated with systematic bias in maternal ratings of their children’s hyperactivity-impulsivity (Boyle & Pickles, 1997).

In early childhood through middle childhood, exploratory and confirmatory factor analyses indicate that impulsivity and activity level together provide the highest loadings on the surgency/extraversion factor, forming the foundation for the surgency/extraversion temperament dimension (e.g., Mullineaux, Deater-Deckard, Petrill, Thompson, & DeThorne, 2009; Rothbart et al., 2001).Hyperactivity-impulsivity seems to be one of the most phenotypically stable temperament dimensions although longitudinal stability of childhood hyperactivity-impulsivity varies widely depending on the assessment method utilized (see Pelham, Fabiano, & Massetti, 2005 for a review). Olson, Schilling and Bates (1999) reported stability in impulsivity between 1^st and 3^rd grade of r = .44. A study by Rothbart and her colleagues (2001) further demonstrated that the two components of surgency/extraversion – impulsivity and activity level – were two of the most stable facets over time (i.e., two-year stability from 5 to 7 years of age was in .7 range for mothers and fathers), compared to all other facets of temperament measured by the Children’s Behavior Questionnaire (CBQ). More recently, Majdandzic and van den Boom (2007) used latent factor models to examine consistency across situations and time in early childhood and reported higher temporal stability for the parent perceptions of activity level based on mothers’ and fathers’ reports on the CBQ (.83), compared to the observed activity level measured by ratings on the Laboratory Temperament Assessment Battery (Lab-TAB) (.70). In regard to inter-rater agreement, impulsivity and activity level both have shown moderate to substantial (r = .4 to .7 range) mother-father agreement, as shown in multiple samples (e.g., Majdandzic & van den Boom, 2007; Mullineaux et al., 2009; Rothbart et al., 2001).

There is a substantial literature supporting the heritability of temperament (e.g., Auerbach, Faroy, & Ebstein, 2001; Goldsmith, Lemery, & Essex, 2004). In particular, prior behavioral genetic studies have demonstrated that hyperactivity-impulsivity is highly heritable (see Levy, McStephen, & Hay, 2001). However, the high end of such heritability estimates (i.e., .8 and above) have been found in investigations in which parents were the informants who provided information on the twins’ zygosity and rated their children’s behaviors (Nigg, Hinshaw, & Huang-Pollock, 2006). It is plausible that those estimates were inflated by rater contrast effects (Eaves et al., 2000).

According to a trait/state view of temperament behaviors like hyperactivity-impulsivity, genetic and environmental factors work together to cause variation in neuronal and biochemical factors that in turn cause variation in an abstract trait, which itself is observed directly as transient (state-like) and stable (trait-like) behaviors (Strelau, 2001). These behaviors also are influenced by immediately contingent situational, physiological, and psychological setting conditions. In the current study, we considered variance that is stable across time as well as variance that is time/context/informant-specific, in an effort to test directly the veracity of the disposition concept. By developing a multi-informant measurement model and testing its sensitivity to detecting time and context variants, we attempted to derive accurate estimates of the most likely ‘trait’ and ‘state’ components of variance in some behavioral indicators of dispositional hyperactivity-impulsivity—as a conceptual and statistical model that could be applied to all dimensions of temperament and personality.

There remains a longstanding debate about which methods and informants are most valid and reliable to operationalize temperament. One approach that has the potential to resolve this debate is the use of multi-informant and method constructs with good internal and external validity (Karp, Serbin, Stack, & Schwartzman, 2004). The use of multi-informant, multi-context temperament constructs not only can elucidate systematic method/rater biases if they exist, but also can promote the development of new measurement models and assessment tools with improved psychometric properties, including enhanced predictive validity and replicability (Nunnally & Bernstein, 1994; Rushton, Brainerd, & Pressley, 1983).

One common approach that has been used involves computation of multi-informant composites that are calculated as a simple average score across different informants. Such composites constrain the data by obscuring the examination of the variance in behavior that may or may not overlap across different contexts and informants. Another approach utilizes latent variables to combine information from multiple informants and to isolate informant-specific effects as measurement error that is not examined. In the current study, we offer an alternative that integrates these common approaches without removing or obscuring context and informant effects. We examined longitudinal factor models of temperament that utilizes multi-informant scores, in an effort to describe and predict longitudinal stability and change from 4.5 to 11 years of age.

Many studies of child temperament share the same data structure: parents (and sometimes observers) report on child behavior in the home context, and teachers (and sometimes different observers) report on child behavior in the childcare or school context. The development of models that can address potential context and informant effects is important, especially since context and informant effects are always confounded in the longitudinal data in this common design. Our goal was to identify systematic context effects— for home and school settings—on stability and change in behavioral indicators of hyperactivity-impulsivity in the transition to and through middle childhood, while also considering mother, father, teacher, and observer informant variance. We chose to examine hyperactivity-impulsivity because it is a temperament trait that is highly heritable, highly stable, and easily observable by others. Furthermore, hyperactivity-impulsivity plays a central role in the conceptualization of child psychopathology (Quay, 1993).

Although secondary analysis offers opportunities for increasing the informational value as well as a relatively low cost way to ask original research questions (Bullock, 2007), we were limited as to the available variables in constructing the measurement models of hyperactivity-impulsivity. One consequence was that the items used in the current analysis were narrowly defined behavioral indicators of the temperamental dimension of surgency/extraversion. Although it is beyond the scope of this study, it should be noted that researchers of child temperament and psychopathology generally agree that temperament incorporates the normative range of responding, whereas behavior problems are symptom clusters that are dysfunctional, excessive, maladaptive, or debilitating (Achenbach, 1995). Measurement confounding (i.e., item-content overlap) has been an issue in the studies of child temperament and behavior problems, because most temperament and symptom measures tap behaviors, and similar behaviors may stem from either temperament or behavior problems (Lemery, Essex, & Smider, 2002). Empirical evidence, however, has been accumulated suggesting that measurement confounding does not account for the association between temperament and behavior problems (Lemery et al., 2002; Lengua, West, & Sander, 1998). The items used in this study were adopted from the CBCL and the TRF that rated hyperactive behaviors (rather than activity) and impulsive behaviors. Thus, these item scan be viewed as reflecting the distribution in the range of behavior problems rather than normal range behaviors. Nevertheless, we believe that these items could capture individual differences in the temperamentally based behaviors that were part of a surgency/extraversion dimension.

The existing research on temperament described above (though almost all of it based on mono-method/informant assessments of constructs) suggests moderate to substantial short-term (1–2 years) stability of child hyperactivity-impulsivity, with the estimates of stability declining over longer time periods(e.g., Green, Loeber, & Lahey, 1991). In light of this literature, we hypothesized that individual differences in the multi-informant construct of hyperactivity-impulsivity would show moderate to high stability within the same context across time (2 years).Based on previous findings of stronger stability with particular contexts compared to consistency across contexts (e.g., Majdandzic & van den Boom, 2007; Ruff, Capozzoli, & Weissberg, 1998), we hypothesized that within-context longitudinal stability would be comparable or exceed the magnitude of correlations between contexts within a concurrent time point. We also expected that longitudinal stability for the measurement construct would be stronger for the home context than for the school context, in line with previous research that suggest higher consistency for parent reports compared to observational ratings. In addition, it was important for us to consider informant effects in the longitudinal factor model in order to identify and distinguish stable informant effects (i.e., systematic effects of informants due to the unique aspects of the relationship with the child and/or due to informant-specific rater bias) from reliable cross-informant variance when estimating context-specific hyperactivity-impulsivity. Overall, current theory in temperament emphasizes the role of social influences (e.g., demands of the situation such as home or school, or cultural expectations) in the development of individual differences (e.g., Sanson et al., 2004). The examination of reliable estimates of the ‘trait’ and ‘state’ components of temperamentally-based hyperactive-impulsive behaviors among children has significant implications for understanding of the developmental continuity in child temperament.

Method

Participants

In the current study, we examined the public datasets of the NICHD (National Institute of Child Health and Development) Study of Early Child Care and Youth Development (http://www.nichd.nih.gov/research/supported/seccyd/datasets.cfm). Data collection began in 1991 in 9 states (Arkansas, California, Kansas, New Hampshire, North Carolina, Pennsylvania, Virginia, Washington, and Wisconsin) and included 1364 children (52% male) and their families when the children were one month of age. The current sample included 978 children (482 boys and 496 girls) who had mothers’ ratings of hyperactivity-impulsivity measures for at least two time points over the three assessments in 1^st, 3^rd, and 5^th grades. Participating children were from five racial categories: White (83%); Black (11%); Asian or pacific islander (1%); American Indian, Inuit, or Aleutian (1%); and Other (4%). At the first time point (1^st grade) these 978 children did not differ from the 102 children who were not included in the current sample because they had data only in 1^st grade but not in 3^rd or 5^th grade (i.e., ‘non-participants’) with respect to gender (49% boys among participants vs. 57 % boys among non-participants), race (83% whites among participants vs. 77 % whites among non-participants), maternal education (an average of 14.5 years among participants vs. 14.2 years among non-participants), and the presence of a husband or partner for the mothers (82% among participants vs. 83 % among non-participants).Mothers of those 978 children, however, had significantly higher family incomes compared to the 102 non-participants (income-to-need ratio; M = 4.06 vs. 3.01, t (980) = 4.29, p< .05). The analysis sample did not differ from the non-participants with respect to mothers’, fathers’, and teachers’ reports on hyperactivity/impulsivity in 1^st grade, but showed slightly lower levels of hyperactivity/impulsivity according to observers’ ratings (M = 4.35 vs. 4.14, t (949) = 2.03, p = .04).

Measures

We used mothers’, fathers’, teachers’, and observers’ (trained paid research staff) ratings on items pertaining to several key indicators of hyperactivity and impulsivity. The items were selected based on 1) face validity from a variety of instruments that are described here (see Table 1 for an overview), and 2) availability across the three measurement points (i.e., measured consistently in 1^st, 3^rd, and 5^th grade.

Table 1.

Measures used to construct hyperactivity-impulsivity, by time point, and N with valid data for mothers’ (M), fathers’ (F), teachers’ (T), and observers’ (O) reports

	1st Grade				3rd Grade				5th Grade
	M F TO				M F TO				M F TO
(Number)
Child Behavior Checklist	906	603	--	--	953	698	--	--	939	696	--	--
Teacher Report Form	--	--	921	--	--	--	925	--	--	--	886	--
Classroom Observation System	--	--	--	881	--	--	--	910	--	--	--	900

Parent and teacher ratings of children’s hyperactive-impulsive behavior were gathered at each time point. The Child Behavior Checklist or CBCL and the closely related Teacher Report Form or TRF (Achenbach, 1991) are scored using a three-point Likert-type scale: 0 = not true, 1 = somewhat or sometimes true, 2 = very true or often true. From the CBCL and the TRF, we included the same two items from each (can’t sit still, restless, hyperactive; impulsive or acts without thinking).

The Classroom Observation System or COS was developed by the SECC Steering Committee for the National Institute of Child Health and Development Study of Early Child Care and Youth Development (2006). The COS captured discrete child behaviors and interactions with others in the classroom over the course of two 44-minute observation cycles. Each cycles consisted of three ten-minute time sampled periods (30-second observe and 30-second record intervals). In addition to the discrete behavioral recordings completed, a qualitative rating was completed for each these cycles using a global 7-point scale (1 = uncharacteristic to 7 = extremely characteristic). These three global ratings were then averaged to yield an overall child rating across the observational cycles. For hyperactivity, we used the activity level scale which reflected an overall rating of how physically active, restless, and fidgety the child was during the observation period. Inter-rater reliability estimates for the activity level based upon repeated measures ANOVA (i.e., the unbiased estimate of the reliability of the mean of k = 2 measurements after taking into account differences in the raters, k= number of raters, described by Winer, 1971) were .70 for 1^st grade ratings, .67 for 3^rd grade ratings, and .61 for 5^th grade ratings.

Statistical Analysis

We used confirmatory factor analyses (CFA) models via structural equation modeling (SEM; Bollen, 1989) to estimate measurement models of the behavioral dimensions of hyperactivity-impulsivity. We used two criteria to determine empirical indices of a certain latent psychological construct (Nunnally & Bernstein, 1994). First we considered content validity by examining the adequacy with which a specified domain of content was sampled. Each item/or scale that is said to comprise the latent construct must stand on its own as an adequate representation of that construct. Next we conducted a series of factor analyses to test internal validity—demonstrated empirically as showing that correlations among the manifest variables can be patterned according to the mathematical expectations of a single latent construct (McArdle, 1996). Only the observed variables that have significant factor loadings on the latent factors were maintained using a cutoff of .3 (accounting for 9% of the variance; Pedhazur & Schmelkin, 1991) for what we consider minimally acceptable factor loadings (with one exception of .22 for the COS rating on the school context latent factor in 5^th grade).

We focused on testing measurement models for the latent context factors (i.e., home and school) based on the scores reported by multiple informants to study the factorial composition of hyperactivity-impulsivity while differentiating context effects. The latent factors of the home context consisted of the scores reported by mothers and fathers, and the latent factors of the school context consisted of the scores reported by teachers and observers. Items were reverse scored if necessary so that higher scores indicated higher levels of hyperactivity-impulsivity.

In order to examine longitudinal stability and changes in these behavioral indicators of temperament, we estimated longitudinal factor analysis models that specified within-occasion correlations between home and school context factors (i.e., cross-context correlations) as well as between-occasion auto-regressive effects for the same context factor (i.e., stability of contexts). We used the Amos 7.0 program (Arbuckle, 2006) that estimated parameters incorporating full information maximum likelihood (FIML) methods. The FIML methods allowed data from all individuals to be included regardless of their pattern of missing data and are more appropriate than other commonly used methods such as mean substitution. In evaluating the overall goodness of fit of each model, we report the chi-square goodness-of-fit statistic (χ²), degrees of freedom (df), and corresponding p value; the Comparative Fit Index (CFI), and the Root Mean Square Error of Approximation (RMSEA). CFI values greater than .90 and RMSEA values of .08 or lower are indicative of good-fitting models (Browne & Cudeck, 1993). In order to the significance of cross-context correlations and informant effects, equality constraints were hierarchically imposed to test the adequacy of the constraints using nested χ² difference tests (Bollen, 1989). We also examined changes in CFI, because the chi-square difference test for nested model comparisons may be too sensitive with relatively large samples (Cheung & Rensvold, 2002). We tested if a value of CFI is greater than the recommended .01 (Cheung & Rensvold, 2002)to reflect a meaningful difference in model fit.

Results

Table 2 presents descriptive statistics (Means and SDs) of hyperactive-impulsivity scores based on mother, father, teacher, and observer reports in 1^st, 3^rd, and 5^th grades – for the whole sample and separately for boys and girls. Two indicators for mother, father, and teacher ratings were averaged within informant and time. We performed a series of t-tests to examine differences between boys and girls, using a Bonferroni-adjusted alpha level (i.e., dividing the per comparison alpha level by the number of outcome variables, α = .004) to control for inflated Type I error rates (Jaccard & Guilamo-Ramos, 2002). Boys were rated as showing higher levels of hyperactive-impulsive behaviors regardless of informants and measurement time points (t = 3.51 to 9.51, p < .001).

Table 2.

Means and Standard Deviations for Mothers’, Fathers’, Teachers’, and Observers’ Ratings of Hyperactivity-Impulsivity

	Full Sample Mean (SD)	Boys Mean (SD)	Girls Means (SD)
Grade 1
Mothers’ Ratings	.46 (.50)	.55 (.53)	.39 (.45)
Fathers’ Ratings	.47 (.46)	.57 (.48)	.38 (.43)
Teachers’ Ratings	.36 (.56)	.51 (.62)	.23 (.40)
Observers’ Ratings	4.15 (.85)	4.24 (.83)	4.04 (.85)
Grade 3
Mothers’ Ratings	.43 (.50)	.51 (.51)	.34 (.45)
Fathers’ Ratings	.45 (.48)	.51 (.50)	.38 (.45)
Teachers’ Ratings	.39 (.59)	.53 (.64)	.26 (.50)
Observers’ Ratings	4.18 (.47)	4.24 (.50)	4.11 (.44)
Grade 5
Mothers’ Ratings	.39 (.47)	.44 (.52)	.28 (.41)
Fathers’ Ratings	.35 (.46)	.41 (.48)	.29 (.42)
Teachers’ Ratings	.31 (.51)	.47 (.58)	.16 (38)
Observers’ Ratings	4.14 (.46)	4.20 (.45)	4.08 (.44)

We conducted repeated-measures general linear modeling (GLM) analysis to examine whether there were significant developmental changes in the levels of hyperactive-impulsive behaviors across 1^st, 3^rd, and 5^th grades, and whether the changes differed by gender (i.e., time by gender interactions). The results indicated that reports by mothers, fathers, and teachers showed significant declines in children’s hyperactive-impulsive behaviors regardless of gender, with F (1, 844) = 54.06, p< .05 for mother ratings, F (1, 472) = 61.67, p< .05 for father ratings, and F (1, 779) = 7.13, p < .05 for teacher ratings. In contrast, there were no significant changes across 1^st, 3^rd, and 5^th grades based on observer ratings, F (1, 779) = .38, p = .53.

We estimated zero-order bivariate Pearson correlations between all of the study variables which included 4 items for the home context and 3 items for the school context at each measurement time (available upon request).Inter-item correlations ranged from .20 to .41 for the home context, from .22 to .64 for the school context in 1^st grade; from .27 to .49 for the home context and from .21 to .64 for the school context in 3^rd grade; and from .33 to .52 for the home context and from .16 to .58 for the school context in 5^th grade. Correlations between parent ratings ranged from .20 to .41 in 1^st grade, from .27 to .49 in 3^rd grade, and from .33 to .52 in 5^th grade. Inter-item correlations for teacher ratings were .64 in 1^st grade, .64 in 3^rd grade, and .58 in 5^th grade.

Figure 1 presents the longitudinal factor model of context and informant effects. Different informants represent possible systematic errors that are referred to as method factors (Campbell & Fiske, 1959). Informant effects were measured by the degree of covariation between the unique factors for the variables reported by the same informant at a given measurement time (Kenny & Kashy, 1992).Stabilities within the same informants were estimated through autocorrelations over time (between the adjacent times) for unique factors for the same variable provided by the same informant (see McArdle & Nesselroade, 1994). First, we tested the significance of informant effects in studying temporalstability and cross-context consistency of hyperactivity-impulsivity by comparing two models — one with and one without informant effects. If there are no systematic informant variances, then the fit of the simpler model (the one without estimating informant effects) should not be significantly worse than the one estimating informant effects. Second, we compared the models with and without within-time cross-context correlations in order to appreciate the significance of cross-context consistency. If the home and the school contexts were completely independent, then the model fit should not significantly degrade for the simpler model (the one without estimating cross-context correlations) compared to the more complex model which estimates cross-context correlations.

Figure 1.

Longitudinal factor model of context and informant effects for hyperactivity-impulsivity from 1^st grade to 5^th grade. CBCL = Child Behavior Checklist; TRF = Teacher Report Form; COS = Classroom Observation System; M = Mother; F = Father; T = Teacher; O = Observer; G1 = 1^st Grade; G3 = 3^rd Grade; G5 = 5^th Grade.

We fitted a model that specified both informant effects and cross-context correlations. The correlations between unique factors (within a measurement time)represented informant effects, representing variances due to different sources of informants: mother, father, and teacher in 1^st, 3^rd, and 5^th grades. In addition, to consider the longitudinal stability of informant effects, correlations between unique factors of the corresponding variables for the same informant between two adjacent occasions were estimated. The longitudinal factor model with informant effects — “the informant effect model” —fit the data well, χ² (165) = 505.77, p = .00, CFI = .94, RMSEA = .05. In a subsequent model, the informant effects were fixed to zero. This model without informant effects provided a mediocre fit, χ² (182) = 946.35, p = .00, CFI = .87, RMSEA = .07. Compared to the model with informant effects, the model fit for the model without informant effects degraded significantly, Δχ² = 440.58, Δ df = 17, p< .05, and ΔCFI = .07.

Next, we compared the model with both informant effects and cross-context correlations with two alternative models that fixed cross-context correlations to zero in order to test the context specificity —whether the home and the school contexts are independent of each other. In this series of models, if we fixed one of the cross-context correlations to zero, the variance for the home context in 3^rd grade became small yet negative (e.g., -.005). In order to avoid such a Heywood case(i.e., the estimated error term for an indicator for a latent variable is negative), the variance of the home context factor in 3^rd grade and the covariance between the home and the school context factors in 3^rd grade had to be fixed to zero. Consequently, we were not able to test the significance of the cross-context correlations in 3^rd grade. We first tested the significance of the zero-order correlation between the home and the school contexts at 1^st grade. This model without the cross-context correlation in 1^st grade provided a mediocre fit, χ² (168) = 830.76, p = .00, CFI = .88, RMSEA = .06. Compared to the model that estimates 1^st grade correlation between the home and the school contexts – with the variance of the 3^rd grade home context and the 3^rd grade cross-context covariance fixed to zero, χ² (167) = 508.06, p = .00, CFI = .94, RMSEA = .05– the model fit without the 1^st grade cross-context correlation had a significantly worse fit, Δχ² = 322.70, Δ df = 1, p< .05, and ΔCFI = .06.

We examined the significance of cross-context consistency between the home and the school contexts in 5^th grade by fixing the correlation between the two context factors to zero. In doing so, within-context stability was kept in the model order to control for possible confounding effects of stability when examining cross-context consistency. This model without the cross-context correlation in 5^th grade yielded an acceptable fit, χ² (168) = 516.88, p = .00, CFI = .94, RMSEA = .05. Compared to the model with the cross-context correlation in 5^th grade, the model fit did not significantly degrade by assuming a correlation of zero between the home and the school contexts in 5^th grade, as indicated by a trivial change in CFI, Δχ² = 8.82, Δ df = 1, p< .05, and ΔCFI = .002.

As shown in Table 3 and Figure 2, a closer examination of the significant coefficients in the best-fitting model – the informant effect model – indicated that all of the within-context stability coefficients were significant and substantial. There was a strong stability in the home context between 1^st grade and 3^rd grade (β = .97, p< .05), and between 3^rd grade and 5^th grade (β = .90, p< .05). Similarly, there was significant stability, with weaker magnitude compared to the home context, in hyperactivity-impulsivity measured in the school context between 1^st grade and 3^rd grade (β = .95, p< .05), and between 3^rd grade and 5^th grade (β = .69, p< .05). With respect to cross-context consistency, the concurrent correlation between the home and the school contexts in 1^st grade was high and significant (r = .71, p< .05). In contrast, the association between hyperactivity-impulsivity measured in the home context and hyperactivity-impulsivity measured in the school context was significant but low in 5^th grade (r = .25, p< .05), after controlling for the context-specific stabilities (i.e., autoregressive effects of 3^rd grade context on 5^th grade context). It should be noted that the variances of home and school context factors as well as the covariance between them (r = .55, p = .20) in 3^rd grade became non-significant after controlling for informant effects and context-specific stabilities. This may be in part due to extremely high percentages of variances that were explained by the auto-regressive effects of the 1^st grade factors (i.e., 94% for the home context and 90% for the school context).

Table 3.

Unstandardized parameter estimates, standard errors, and critical ratios for longitudinal factor models of hyperactivity-impulsivity from 1^st Grade to 5^th Grade

	Model without informant effects			Model with informant effects
Variable	Estimate	Std. Error	Critical Ratio	Estimate	Std. Error	Critical Ratio
Factor loadings
CBCL M1→Home G1	1=			1=
CBCL M2→Home G1	1.29*	.08	15.34	1.32*	.09	14.47
CBCL F1→Home G1	1.21*	.09	13.37	1.25*	.10	12.33
CBCL F2→Home G1	.74*	.07	10.07	.71*	.08	8.92
TRF T1 →School G1	1=			1=
TRF T2 →School G1	1.09*	.05	20.00	1.12*	.06	19.01
COS O1 →School G1	.55*	.07	8.04	.74*	.09	7.94
CBCL M1→Home G3	1=			1=
CBCL M2→Home G3	1.22*	.07	18.33	1.27*	.08	16.76
CBCL F1→Home G3	.76*	.06	11.92	.77*	.07	10.62
CBCL F2→Home G3	1.13*	.07	15.31	1.19*	.09	13.64
TRF T1 →School G3	1=			1=
TRF T2 →School G3	1.13*	.06	20.20	1.15*	.06	19.47
COS O1 →School G3	.32*	.04	8.84	.40*	.05	8.52
CBCL M1→Home G5	1=			1=
CBCL M2→Home G5	1.21*	.07	16.98	1.25*	.08	16.12
CBCL F1→Home G5	.80*	.07	12.17	.77*	.07	11.00
CBCL F2→Home G5	1.10*	.08	14.61	1.09*	.08	13.38
TRF T1 →School G5	1=			1=
TRF T2 →School G5	.86*	.06	15.06	.85*	.06	15.53
COS O1 →School G5	.21*	.04	5.77	.20*	.05	4.00
Factor stabilities
Home G1→Home G3	1.01*	.08	13.96	1.01*	.08	13.33
Home G3→Home G5	.88*	.06	15.11	1.02*	.09	11.63
School G1→School G3	.75*	.05	15.47	.94*	.06	15.57
School G3→School G5	.64*	.05	13.56	.82*	.07	12.33
Factor covariances
Home G1 School G1	.09*	.01	10.74	.08*	.01	9.80
Home G3 School G3	.02*	.00	4.55	.01	.00	1.28
Home G5 School G5	.02*	.01	3.62	.01*	.01	2.82
Factor variances
Home G1	.11*	.01	8.84	.10*	.01	8.00
School G1	.22*	.02	12.05	.14*	.02	8.34
Home G3	.01*	.00	2.71	.01	.01	1.33
School G3	.12*	.01	9.30	.02	.01	1.38
Home G5	.02*	.00	3.82	.02*	.01	4.07
School G5	.13*	.02	8.96	.12*	.05	2.39

Note. CBCL = Child Behavior Checklist; TRF = Teacher Report Form; COS = Classroom Observation System; M = Mother; F = Father; T = Teacher; O = Observer; G1 = 1^st Grade; G3 = 3^rd Grade; G5 = 5^th Grade. The “=” symbol means a parameter is fixed.

^*p< .05.

Figure 2.

Structural equation model showing standardized estimates of stability coefficients and cross-context correlations for hyperactivity-impulsivity.

We observed a general tendency for the context-specific stability to increase when considering informant effects that may have been confounded with context effects (see Table 3). Specifically, when we compared the longitudinal factor models with and without informant effects, stabilities for the home context increased from β = .95 to β = .97 for 1^st grade–3^rd grade but slightly decreased from β = .92 to β = .90 for 3^rd grade–5^th grade. Stabilities for the school context increased from β = .73 to β = .95 for 1^st grade–3^rd grade, and from β = .64 to β = .69 for 3^rd grade–5^th grade.

An interesting trend was found for changes in the cross-context consistency (i.e., correlations between the home context and the school context latent factors) between models with and without informant effects. It appears that the cross-context correlations decrease over time while at the same time informant effects become less responsible for context-specificity as children age. Specifically, the correlation between the home and the school contexts increased from r = .62 to r = .71 in 1^st grade and decreased from r = .37 to r = .25 in 5^th grade. By estimating informant effects, the overlapping variances between the home and the school contexts notably increased (from r²= .38 to r²= .50) in 1^st grade, whereas the common variance between the two contexts decreased in 5^th grade (from r² = .14 to r²= .06).Finally, as for longitudinal stability of informant effects, autocorrelations over time for the unique factors ranged from .21 to .28 for mother reports and from .13 to .24 for father reports.

Although we did not expect any specific gender differences in the structural relationships among the longitudinal measures of hyperactivity-impulsivity scores measured for the home and the school contexts, we explored possible differences between boys and girls in terms of the magnitude of within-context stabilities and cross-context consistencies using two-group structural equation models. In general, boys and girls did not seem to differ significantly with respect to within-context stabilities and cross-context consistencies.¹

Discussion

We used multi-method multi-informant scores based on a large, representative national sample to model context-specific stability (i.e., home vs. school) and cross-context consistency in hyperactivity-impulsivity while considering informant effects that may confound both context-specific stability and cross-context consistency. Our findings suggested (1) stronger longitudinal stability of hyperactivity-impulsivity within a context compared to cross-context consistency, (2) higher within-context stability in the home context than the school context, and (3) systematic variances that were introduced by sharing a common measurement method (e.g., parent vs. teacher rating) and influenced the estimates of within-context stability and cross-context consistency.

Several findings of this study are notable as they add to the current knowledge base regarding context specificity and informant/method effects in child temperament. First, it is noteworthy that the magnitude of the estimates of cross-context consistency of hyperactive-impulsive behavior was exceeded by the magnitude of the estimates of temporal stability within each context (regardless of whether or not informant effects were considered). Research examining genetic and environmental sources of variance in hyperactivity-impulsivity commonly reports significant genetic influences. High heritability for hyperactivity-impulsivity also has been indicated regardless of measurement instrument, informant, and context (Derks et al., 2008; Kuntsi & Stevenson, 2001; Silberg et al., 1996). Particularly, high heritability has been indicated for hyperactivity-impulsivity across middle childhood (Derks et al., 2008). For example, heritability estimate of 88% was indicated in a recent study of 8-year-old twins for hyperactivity-impulsivity (McLoughlin, Ronald, Kuntsi, Asherson, & Plomin, 2007). These genetic influences are thought to represent, at least in part, a biological basis for temperament traits. However, our findings demonstrated that context-specific variance in behavior is systematic and not trivial even for the stable, trait-like temperament behavior of hyperactivity-impulsivity. Just as important, these context effects may only be modestly confounded with systematic effects of informants. The correlations between hyperactive-impulsive behavior in the home and the school contexts were significant and substantial (average r = .50) yet far from unity. In comparison to prior analyses (see Kim, Mullineaux, Allen, & Deater-Deckard, in press) that examined other temperament behaviors such as attention regulation (part of a broader effortful control dimension) and frustration/anger (part of a negative affectivity dimension), hyperactivity-impulsivity (part of a surgency/extraversion dimension) seems to have lower levels of consistency across diverse contexts than attention (average r = .68) but higher levels of consistency across contexts than anger (average r = .38). Taken together, the findings lend support for prior research suggesting that the particular pattern of context stability may depend on the particular aspect of temperament or personality in question (e.g., Majdandzic & van den Boom, 2007).

Second, greater temporal stability would be expected for the home-context because it is defined using the same informants over time (mothers and fathers) and represents the same stable home environment for the vast majority of children (e.g., Hanson, 1975). Furthermore, parental perceptions are based on a larger sample of daily experiences and their views of the child may be less likely to change over time (e.g., Majdandzic & van den Boom, 2007; Rothbart et al., 2001). In contrast, the school context is defined using different informants in the same “role” (i.e., teacher or observer) across the time points, and the classrooms obviously are different at each time point. From a methodological point view, lower levels of temporal stability for the school context factor compared to the home context factor may be in part due to the fact that the school context factor included observations of temperament that have shown to hold lower stability than questionnaires. Our data were consistent with such an expectation showing that the within-context stability was higher for the home context (average stability coefficient = .94) than for the school context (average stability coefficient = .69) before controlling for informant effects. The same pattern was found after controlling for informant effects (average stability coefficient = .94 for the home context and average stability coefficient = .82 for the school context). More generally, within-context stability estimates were higher than the stability estimates based on simple zero-order correlations, as we used the latent factor SEM approach that increased reliabilities of the variables by controlling for measurement errors as well as method variances due to informants. Therefore, these findings suggest that even after taking into consideration any systematic effects that result from a particular informant, children’s expression of hyperactivity/impulsivity was influenced by context-specific effects due to factors related to environmental settings (e.g., Kraemer et al., 2003).

Interestingly, there was evidence for differences in developmental changes in the stability of hyperactivity-impulsivity across the home and the school contexts. Specifically, the variances explained by autoregressive effects (e.g., regression effects of 1^st grade home context factor on 3^rd grade home context factor) were high and fairly comparable between the two contexts in 3^rd grade (94% for home context and 90% for school context). However, in 5^th grade, variances explained by autoregressive effects were fairly stable and still high for the home context (81%), whereas such variances dropped by almost half in magnitude for the school context (47%). This notable reduction may be due in part to modest decreases in measurement reliability especially in fifth grade, as seen in inter-rater reliabilities for the observation measures as well as inter-item correlations for the teacher ratings that composed the school context factor. Furthermore, when the variance attributable to potential informant effects were taken into consideration, the within-context stability increased notably for the school context behaviors (the average change in the stability coefficient = .13), but not for the home context (the average change in the stability coefficient = 0). Such findings seem to support the commonly assumed but rarely tested assumption that the magnitude of informant effects on estimates of stability of behavior will be stronger as a function of the number of informants being used. In other words, variances in stability estimates that can be attributed to informant bias are expected to be greater when larger numbers of informants are involved.

Third, there were significant informant effects that influenced stability within a context. The overall fit of the longitudinal factor model for hyperactivity-impulsivity significantly improved when the informant effects were estimated. Longitudinal stability within a context increased in general once the informant effects were taken into account, suggesting that children’s hyperactive-impulsive behaviors were substantially stable over time. Our results are in support of the recent findings of Cole and his colleagues who demonstrated that failure to include residual correlations that reflect shared method variance can change the meaning of the extracted latent variables and generate potentially misleading results (Cole, Ciesla, & Steiger, 2007).Our finding further implies that if we ignore informant effects in testing temporal stability of a psychological construct, we may obtain biased estimates of stability as they would beattenuated.

Fourth, the cross-context consistency between the home and the school contexts within a measurement time changed when we controlled for confounding informant effects. Specifically, the variances that overlapped between the two contexts increased from 38% to 50% in 1^st grade. The common variances decreased from 14% to 6% in 5^th grade after controlling for informant effects as well as context-specific autoregressive effects. These findings suggest that context-specific effects may become over- or under-estimated thus making children’s behaviors look less or more consistent across different contexts (e.g., Zeman & Shipman, 1996) when informant effects are not appropriately addressed. Our findings lend support for the argument by prior researchers who emphasized the importance of the psychological assessment of children being a multi-informant and multi-method process (Frazier & Youngstrom, 2006; Kendall & Morris, 1991). By collecting data from multiple informants and modeling explicitly the informant/method variances, researchers can better control the potential bias in estimating context-specific effects arising from confounds with informant effects.

One of the limitations of our secondary data analysis was that we were constrained by the available data—the items that were consistently available across all four occasions. This limited our ability to construct more comprehensive home and school context factors of hyperactivity-impulsivity. As the estimates of stability and consistency are expected to be dependent upon the method of assessment, it is important to note that the CBCL/TRF items are rather narrow descriptions of the child’s behavior, compared to measures of temperament impulsivity and activity level (e.g., CBQ). The limitation with data availability also prevented us from constructing latent factors that represent informant effects, because we had fewer than three indicators for each informant.

This limitation aside, the present study illustrates an approach for deriving multi-method composite scores while also considering context-specific behavior and controlling for systematic and transient informant effects. Our longitudinal factor model has broad applicability and is appropriate for use in other existing studies that measure child behavior across parent, teacher/caregiver, tester, observer, and self ratings. The findings demonstrate how “perspective” (i.e., characteristics of the informant) and “context” (i.e., factors related to circumstances that might influence the subject’s expression of the trait) interface with the other to influence testing of the stability of the “trait” (Kraemer et al., 2003).It would be informative to apply the approach we used to answer questions about personality stability later in life when self-reports are available. Researchers are encouraged to consider both context and informant effects in studying the stability of personality traits among individuals using self-reports and others’ ratings (such as family member and peer ratings).

Furthermore, the current findings illuminate potentially different patterns of stability and change for key indicators of Rothbart et al.’s (2001) three child temperament dimensions. Specifically, temperament behaviors of a negative affectivity dimension (represented by frustration/anger) seem to be the most specific to the influences and changes in the environmental settings (e.g., Kim et al., in press), followed by temperament behaviors of a surgency/extraversion dimension (represented by hyperactivity-impulsivity behaviors). Temperament behaviors of an effortful control dimension (represented by attention regulation) seem to be the least specific to the influences and changes in the child’s environment (e.g., Kim et al., in press).More broadly, our results affirm the significant context effects on individual differences in the development of temperament and personality, by illustrating significantly differential behaviors across different contexts that were not simply attributable to systematic informant bias with respect to hyperactivity-impulsivity – an aspect of temperament that is highly stable and easily observable by others. Our understanding of longitudinal stability in personality and behavioral development will be greatly enhanced by considering factors that contribute to context-specificity and context-consistency as well as informant-specific effects.