Abstract
Using both traditional composites and novel profiles of anger, we examined associations between infant anger and preschool behavior problems in a large, longitudinal data set (N = 966). We also tested the role of life stress as a moderator of the link between early anger and the development of behavior problems. Although traditional measures of anger were largely unrelated to later behavior problems, profiles of anger that dissociated typical from atypical development predicted behavior problems during preschool. Moreover, the relation between infant anger profiles and preschool behavior problems was moderated such that, when early life stress was low, infants with atypical profiles of early anger showed more preschool behavior problems than did infants with normative anger profiles. However, when early life stress was high, infants with atypical and normative profiles of infant anger did not differ in preschool behavior problems. We conclude that a discrete emotions approach including latent profile analysis is useful for elucidating biological and environmental developmental pathways to early problem behaviors.
From early in life, young children who respond to blocked goals with high levels of anger are believed to be at greater risk for aggressive, delinquent, and general externalizing problems relative to children who show less anger (Cole, Teti, & Zahn-Waxler, 2003; Denham et al., 2002). Yet, given normative increases in anger across infancy and toddlerhood, even in typically developing children (Braungart-Rieker, Hill-Soderlund, & Karrass, 2010), it is difficult to determine precisely which children are at risk. Research in other domains has delineated risk status by differentiating typical from atypical development early in life and identifying contextual moderators that exacerbate or buffer risk. We took each of these approaches in the current research. We followed a sample of infant twins longitudinally to characterize normative and atypical profiles of anger and to examine early life stress as a moderator of the link between infant anger and preschool behavior problems.
Early Anger and the Development of Behavior Problems
Anger is one of the earliest developing emotions in humans. Evolutionary and functionalist emotion theorists have described anger as an action-oriented response to goal interference linked to discrete patterns of expression in the face and body (Izard, 1977; Lewis, 1990). These expressions of anger, including bodily tensing, an arched back, furrowed brows, and/or a squaring of the mouth, are reliably elicited as early as 2 months of age by removing or withholding anticipated positive rewards (Alessandri, Sullivan, & Lewis, 1990; Izard, 1977). Early displays of anger in this context are consistent with the conceptualization of anger as motivating the persistent pursuit of a blocked goal (Campos, Mumme, Kermoian, & Campos, 1994; Levenson, 2003; Stein & Jewett, 1986). Some dissent exists regarding the presence of discrete anger early in life and perspectives vary on the degree of overlap that exists between displays of anger and other emotions, such as sadness (Camras, 1992; Izard, 1977; Oster, 2005), though nearly all perspectives report observable anger by six months of age.
Despite consistency in its function across the lifespan, anger evinces substantial change. Normatively, expressed anger is relatively low in early infancy, increases late in infancy and through the second year of life, and then decreases across toddlerhood. (Braungart-Rieker et al., 2010; Denham, Lehman, Moser, & Reeves, 1995). Periods of stability in early anger are also present between 2.5 to 5 months, 5 to 7.5 months, and 7.5 to 22 months of age (Malatesta, Culver, Tesman, & Shepard, 1989). Overall, then, there is mixed evidence for mean-level stability and change in early anger expressions, suggesting the possible existence of individual differences in developmental profiles. Therefore, our first goal was to describe individual variability in developmental profiles of anger. Although decidedly exploratory, the extant literature suggests at least three distinct profiles of anger in the second half of infancy: a prototypical profile of anger that increases over time and more atypical profiles of generally high and generally low anger reactivity. These three profiles were thus hypothesized based on previous work; however, additional profiles are certainly plausible (e.g., decreasing anger).
High, stable levels of anger and difficulties regulating anger during infancy and childhood are believed to precede the development of aggressive and externalizing problems (Cole et al., 2003; Denham et al., 2002; Eisenberg et al., 2001). In addition, increases in aggressive behaviors, which are separate from but related to anger relative to goal pursuit, mirror those for observed anger between 10 and 50 months of age, suggesting possible overlap in the development of anger and problem behaviors (Alink et al., 2006). In some cases, high levels of anger have also been associated with the development of internalizing problems in children (Eisenberg et al., 2001; Lemery, Essex, & Smider, 2002), although this may be at least partially due to internalizing and externalizing symptom comorbidity early in life (Gillom & Shaw, 2004). Stably high levels of anger therefore appear to be dysregulated in that they impede rather than aid goal pursuit (Cole, Michel, & Teti, 1994), placing individuals at risk for subsequent behavior problems. Atypically low anger may also be dysregulated in that it interferes with goal persistence (Belsky, Friedman, & Hsieh, 2001).
These findings underscore the importance of distinguishing between normal-range variation in anger and non-normative, more extreme anger tendencies, which may represent facets of risk for the development of behavior problems. Therefore, we examined infant anger in multiple contexts at both 6 and 12 months of age and its association with parent-reported behavior problems at 36 months of age. Furthermore, we characterized infant anger using both continuous measures as well as discretely specified developmental profiles of anger between 6 and 12 months of age. This dual analytic track permitted us to fully explore and better understand adaptive and maladaptive consequences infant anger reactivity.
Life Stress and Risk for Behavior Problems
Though some direct links among emotion and behavior problems are expected, relations between emotions and developing behavioral problems are often more complex. Contextual factors are critical in shaping developmental outcomes (Cicchetti & Aber, 1998). Normative life stressors (e.g., death in the family, moving to a new home, etc.) are notable contextual factors for early development given that they are experienced by most children. Investigations of adolescents report that these types of nonsevere stressors are key predictors of increases in behavior problems (Attar, Guerra, & Tolan, 1994; Vaux & Ruggiero, 1983). High levels of life stress positively predict externalizing problems, including aggression, hyperactivity, and conduct problems, between 7 and 13 years of age (Jackson & Warren, 2000).
Life stressors also place children and adolescents at increased risk for developing internalizing problems (Swearingen & Cohen, 1985). At least one study suggests that such effects cannot be explained by the presence of parental psychopathology, which may lead to both negative, stressful experiences and an increased risk for maladaptive outcomes in offspring. This work showed that when stressful life events were accounted for, parents’ mental illness was no longer associated with adolescents’ depressive symptoms (Gore, Aseltine, & Colten, 1992).
Early life stress is also associated with anger early in life; children who experience a greater number of stressful life events display more anger when goals are blocked (Wang, Trivedi, Treiber, & Sneider, 2005). Life stress may generally disrupt the ability of young children to regulate escalating negative emotions, including anger, leading to the development of behavior problems (Compas, Connor-Smith, Saltzman, Thomsen, & Wadsworth, 2001). From this vantage point, early stress may either disrupt children’s ability to up-regulate anger that is “too low” to maintain goal pursuit or to down-regulate “too-high” anger that may interfere with the goal attainment. Empirical support for the notion that life stress may be detrimental for both high and low anger children is lacking. Stress does exacerbate risk for behavior problems in instances where development is already non-normative (i.e., profiles of development that do not match the most common patterns; Hauser-Cram, Warfield, Shonkoff, & Krauss, 2001) and there is separate evidence that preschoolers who expressed extremely high or extremely low levels of anger in response to goal blockage were reported as having greater numbers of externalizing problems than moderately expressive preschoolers (Cole, Zahn-Waxler, Fox, Usher, & Welsh, 1996). Hence, it is possible that risk for behavior problems is the product of interactions between early life stress and atypical anger profiles, regardless of overall anger levels. This possibility underscores the importance of better understanding early profiles of anger in the context of family stress. Thus, our final aim was to test the interaction between developmental profiles of infant anger and family stress on future behavioral problems.
Current Study
In sum, we address several gaps in the literature regarding early anger development by characterizing typical and atypical profiles of anger during infancy and testing the role of early life stress as a moderator of the link between infant anger and preschool behavior problems. Though some of our analyses were exploratory in nature, hypotheses for each of our aims can be derived from the literature. As previously stated, we expected that at least three profiles of anger would be discernible during infancy. In addition, we anticipated that early anger would be positively associated with behavior problems in preschool, but that levels of life stress would moderate this relation. Namely, we expected that greater early life stress would exacerbate risk for behavior problems linked to atypical infant anger reactivity.
Method
Participants
The sample was drawn from a longitudinal twin study examining genetic and environmental influences on emotional development across infancy and early childhood (Schmidt et al., 2013). Participants for the parent study were recruited from the greater Madison, Wisconsin, area through a variety of methods including state birth records, mothers of twins clubs, television publicity, birth announcements in newspapers, flyers in doctors’ offices, the Internet, and referrals from participants. For the parent study, 989 families were contacted for recruitment, which was ongoing throughout the study. One hundred and forty one families were enrolled but did not participate in any phase of the study. Infants completed laboratory assessments at 6 months and 12 months of age. At the time of the 6-month assessment, 56 families chose not to participate, 3 families had withdrawn from the study, and 495 families were not yet enrolled. Thus the 6-month sample included 294 families; 566 infants provided usable data (292 girls). At the time of the 12-month assessment, 115 families chose not to participate, a total of 17 families had withdrawn from the study, and 160 families were not yet enrolled. Thus, the 12-month sample included 456 families; 881 infants provided usable data (451 girls). Nearly 500 infants (n = 481) provided observational data from laboratory episodes at both 6 and 12 months of age. Smaller numbers of infants had data from only the 6-month (n = 85) or the 12-month assessment (n = 400). The sample includes those infants with observational data from anger episodes at either the 6- or 12-month assessment1 (N = 966; 51.1% girls).
Consistent with local demographics, most mothers self-reported their race as Caucasian (95%, African-American = 1.7%, Asian-American = 0.9 %, American Indian = 0.4%, Hispanic = 1.4%, Other Race = 0.6%) as did fathers (92.8%, African-American = 2.7%, Asian-American = 0.9 %, American Indian = 0.3%, Hispanic = 2.1%, Other Race = 1.2%). The largest number of mothers (31.4%) reported a college degree as their highest level of education (Grade School = 0.1%, Some High School = 1.3%, High School Graduate = 14.5%, Some College, 29.6%, Some Graduate Training = 14.1%, Graduate Degree = 9.0%). The largest number of fathers reported either some college (27.5%) or a college degree (27.6%) as their highest level of education (Grade School = 0.3%, Some High School = 2.4%, High School Graduate = 19.1%, Some Graduate Training = 11.5%, Graduate Degree = 11.6%). Of those families who chose to report their annual income, the greatest number (43.1%) reported gross household incomes of over $60,000 (2.3% earned $15,000 or less, 3.5% earned $16,000–$20,000, 8.4% earned $21,000–$30,000, 13.3% earned $31,000–40,000, 13.8% earned $41,000–$50,000, and 15.5% earned $51,000–$60,000). One hundred and one families did not report on family composition: of those who did, 94.9% reported that children lived with both biological parents.
Procedure
Laboratory assessments occurred when infants were 6 (M = 5.83, SD = 0.60) and 12 (M = 11.89, SD = 1.42) months of age; infants participated in a number of episodes designed to elicit discrete emotions. We focused on episodes designed to elicit anger in infants. Following assessments, mothers and fathers completed a packet of questionnaires and mailed them back to the laboratory. Parents also completed mailed questionnaires when children were 36 months of age. Table 1 provides an outline of data collection, procedures, assessments, and measures.
Table 1.
Age | Assessment Type | Contexts/Measures | Constructs |
---|---|---|---|
6 months | Behavioral visit | Arm Restraint Car Seat |
Anger, Sadness Anger, Sadness |
Questionnaires | Parenting Stress Index | Life Stress | |
12 months | Behavioral visit | Arm Restraint Car Seat |
Anger, Sadness Anger, Sadness |
36 months | Questionnaires | Child Behavior Checklist | Child Behavior Problems |
Behavioral Assessment
Gentle arm restraint
At both 6 and 12 months of age, infant anger was assessed during the Gentle Arm Restraint episode from the Laboratory Temperament Assessment Battery (Lab-TAB; Goldsmith & Rothbart, 1996). For this, the infant was seated in a high chair and presented with a novel toy with which s/he was allowed to play for 15–30 seconds. The parent, having received instructions from the experimenter, then stood behind the infant and held the infant’s forearms to his/her side to prevent engagement with the toy for 30 seconds. The infant was then allowed to reengage with the toy for 30 seconds before a second restraint trial began. The second trial proceeded similarly, ending with the infant being allowed to play with the toy.
Car seat
Anger was also assessed at 6 and 12 months of age during the Car Seat Lab-TAB episode. For this, the parent buckled the infant into a standard but restrictive car seat and then stood outside of the child’s view. The parent was instructed to leave the infant in the car seat for 30 seconds and refrain from speaking to or comforting him/her. Episodes ended with the parent moving back into the infant’s direct line of sight and comforting him/her as needed.
Behavioral coding procedures
Consistent with Lab-TAB scoring procedures (Goldsmith & Rothbart, 1996), trained coders assigned ratings of anger and sadness for each infant in the Arm Restraint and Car Seat episodes. Ratings were assigned in 5-second epochs, during which the maximum intensity was scored for the following: observed facial anger (0 = no anger; 3 = anger of the highest intensity), bodily anger (0 = no struggle or resistance; 4 = continuous high-intensity struggle), and distress vocalizations (0 = no vocal distress; 5 = full intensity cry/scream). Latency (in seconds) to the first anger behavior was also recorded.
The following sadness behaviors were similarly scored: maximum intensity of facial sadness (0 = no sadness; 3 = sadness of the highest intensity), bodily sadness (0 = absent, 1 = present), and latency (in seconds) to the first sadness behavior in the episode. Anger and sadness variables were coded separately to allow for the possibility that infants may have shown both anger and sadness in the same 5-second epoch. All coders were required to achieve a minimum reliability (κ = 0.70) with a master coder before coding independently. Roughly 10% of episodes were double-coded to establish reliability and prevent coding drift. Mean inter-coder reliability for anger variables was κ = 0.78. Mean inter-coder reliability for sadness variables was κ = 0.75.
Latency scores were reversed to parallel the directionality of other behaviors; greater scores reflected a more rapid display of anger or sadness. Within-episode average composites were formed across epochs for each of the remaining behaviors. Speed scores and average composites were then z-scored and combined into single measures2 of anger (mean r = 0.59) and sadness for each episode (mean r = 0.45). Note that this procedure results in two composites of anger: (i.e., Car Seat and Arm Restraint) at each age and two composites of sadness at each age. Anger and sadness composites were moderately correlated at both 6 months (Car Seat: r = 0.53, Arm Restraint: r = 0.43) and 12 months (Car Seat: r = 0.50, Arm Restraint: r = 0.40).
Questionnaires
Early life stress
Given our interest in the role of non-severe, or more “typical” early life stressors, the life events domain of the Parenting Stress Index (PSI; Loyd & Abidin, 1985) was selected as a measure of early-life stress. When children were 6 months old, the primary caregiver reported whether the family had experienced each of 20 possible stressful life events that ranged from regular hassles (e.g., encountering more stressors than anticipated) to major life events (e.g., death of a member of the immediate family). Items were weighted by the severity of the life stressor; for example, endorsing an item asking about an illness in the immediate family (the child along with his/her parents and siblings) receives a score of 1 while endorsing an item asking about divorce receives a score of 7. Item scores were then summed to create an overall scale score. Endorsing all items would result in a score of 80. As a parent report measure, the PSI is generally sensitive to the perceptions and memories of parents rather than children. However, by focusing on the life events domain, we capture objective, discrete life events that impact all members of a family (e.g., children move when parents move). Thus, this scale is a family-level measure of cumulative life stress.
Behavior problems
Behavior problems were assessed at age 36 months via the Child Behavior Checklist (Achenbach & Rescorla, 2000). The primary caregiver reported the degree to which various statements (e.g., “shows panic for no good reason”) were true of their child’s behavior on a 3-point scale (0 = not at all true; 2 = very true or often true). A scale score of internalizing problems (n items = 32; α = 0.82) and a scale score of externalizing problems (n items = 34; α = 0.90) was formed for each child. Given that scores for internalizing and externalizing scales were highly correlated (r = 0.58) and that internalizing and externalizing behaviors are highly correlated early in life (Gillom & Shaw, 2004), the Total Problems scale score was used (n items = 98, α = 0.94). The Total Problems scale included all items from the internalizing and externalizing scales along with items from the Other Problems scale (n items = 32, α = 0.79). The mean correlation among the three scales was r = 0.68.
Missing Data
Because recruitment was ongoing throughout the duration of the study (i.e., some participants joined at 12 months), a greater number of children participated in the 12 month than in the 6 month laboratory assessment (Table 1). An analysis of patterns of missing data suggested that data were missing completely at random (Little’s MCAR χ2 (131) = 148.23, p > 0.10).
Profiles were created using a statistical procedure (described below) that includes Full Information Maximum Likelihood (FIML) estimation procedures. That is, all available data were used to derive individual likelihoods of profile group membership. Because of this, all individuals with life stress measures had an available profile assignment and a complete-cases analysis strategy was employed. Additionally, multilevel analyses were used to account for the correlated nature of the data (i.e., twins within families).
Results
Analyses were conducted in accordance with study aims and hypotheses. After examining descriptive statistics for all variables, profiles of infant anger were derived from 6 and 12 month observations of anger and validated by testing for profile-based differences in composite measures. Bivariate associations between preschool behavior problems and measures of anger were examined for both traditional (i.e., levels of observed anger quantified as composites) and novel (i.e., anger profiles) measures of infant anger prior to multivariate analyses. Finally, life stress was tested as a moderator of the relation between infant anger and preschool behavior problems.
Descriptive Statistics and Age-based Change in Variables
Descriptive statistics are shown in Table 2. Substantial variability was observed for all measures across assessments, indicating the presence of broad individual differences. T−tests compared cotwins on all study variables to ensure random labeling of twins (i.e., Twin A vs. Twin B) within family; all cotwin differences were nonsignificant. Additional t−tests compared boys and girls on all variables. Girls (M = −0.02, SD = 0.60) showed less anger than boys (M = 0.05, SD = 0.53) at the 12-month assessment (t(879) = −1.93, p = 0.05, d = 0.12). No other sex differences were present.
Table 2.
units | n | Minimum | Maximum | Mean | SD | t | |
---|---|---|---|---|---|---|---|
6-month Observed Anger | z−score | 566 | −1.82 | 2.07 | 0.02 | 0.62 | −0.40 |
12-month Observed Anger | z−score | 881 | −1.98 | 1.45 | 0.01 | 0.57 | 0.57 |
36-month Behavior Problems | raw score | 219 | 0.00 | 0.75 | 0.23 | 0.16 | 0.92 |
Life Stress | raw score | 455 | 0.00 | 29.00 | 9.66 | 5.93 | −0.13 |
Note: t is the product of a t−test comparison of twins
A paired samples t-test suggested that observations of anger (t(480) = 0.23, p > 0.10) evinced mean level stability between 6 and 12 months of age. Additionally, bivariate correlations suggested a small amount of rank-order stability in anger over time (r = 0.16, p < 0.01).
Heterogeneity in Profiles of Anger Development
We tested for heterogeneous profiles of anger during infancy using latent profile analysis (LPA). LPA identifies latent profiles of individuals who differ from the overall group by relaxing the assumption that all individuals are drawn from a single population. Relaxing assumptions in this way allows mean levels of anger to vary around separate means and enables identification of profiles reflecting unique patterns of observed anger across episodes. Analyses began with a single-profile solution and progressed until adding additional profiles no longer improved fit over the previous model (total models run = 4). Because the LPA procedure was intended to describe behavioral variability both across contexts (Arm Restraint and Car Seat) and across time (6- and 12-month assessments), composite scores from individual episodes were used. Hence, each analysis included 4 variables, reflecting multiple indices of anger over time, on which profile groups were based: Arm Restraint anger at 6 months of age, Car Seat anger at 6 months of age, Arm Restraint anger at 12 months of age, and Car Seat anger at 12 months of age. A multilevel framework was used to account for the non-independence of the twin data.
Examination of fit statistics (Table 3) suggested that a 3-profile solution fit the data best. The first profile included children with low scores in both anger-eliciting episodes at 6 and 12 months of age (n = 96). We labeled this the low anger profile. The second profile included children with decreasing scores of anger from 6 to 12 months of age, but whose scores remained high relative to the scores of other children (n = 83). We labeled this the high anger profile. The most common profile included children who showed moderate levels of anger across all episodes. However, children in this group showed relative increases in displays of anger in both episodes between the 6 and 12 month assessments (n = 787). We named this class the increasing profile. Gender was unrelated to profile membership (χ2 [2] = 1.60, p > 0.10). For subsequent analyses using anger profiles, children were assigned to the profile group for which they had the highest probability of membership (M probability of membership in assigned group = 0.82).
Table 3.
Number of Profiles in Model | AIC | BIC | χ2 | Entropy | Average Class Assignment Probability |
---|---|---|---|---|---|
2 | 5705.17 | 5770.52 | 90.80** | 0.72 | 0.80 |
3 | 5646.57 | 5734.29 | 70.60** | 0.58 | 0.81 |
4 | 5633.77 | 5754.86 | 22.80 | 0.55 | 0.86 |
Note: χ2 is derived from the Vuong-Lo-Mendell-Rubin Likelihood Ratio Test
p<.01, The three-profile model was determined to have the best fit.
Internal validation of the latent profiles
We conducted a test of internal validation of profile group differences in observed anger in each of the laboratory episodes at each assessment. Our analysis was conducted as a 2-level linear mixed model (Level 1: Family, Level 2: Twins within families) to account for nonindependence in the twin design.
The first analysis showed that, as suggested by Figure 1, profiles differed in levels of anger observed in the Car Seat (F(2, 503) = 25.43, p < 0.01, η2p = 0.09) and Arm Restraint (F(2, 536) = 330.22, p < 0.01, η2p = 0.55) episodes at 6 months. Follow-up contrasts with Bonferroni correction suggested that, in both episodes, the high anger profile showed the highest levels of anger while the low anger profile showed the lowest levels of anger. Similarly, at 12 months, profiles significantly differed in levels of anger observed in both the Car Seat (F(2, 774) = 15.14, p < 0.01, η2p = 0.04) and Arm Restraint (F(2, 832) = 356.21, p < 0.01, η2p = 0.46) episodes. Follow up contrasts with Bonferroni correction suggested that, for both episodes, the low anger profile showed less anger than both the high anger profile and the increasing anger profile. The high and increasing anger profiles were not significantly different.
Anger and 36-month Behavior Problems
Correlations including traditional (continuous) measures of anger revealed that greater anger at 6 (r = 0.38, p < 0.01) but not 12 (r = 0.05, p > 0.10) months was linked to more behavior problems at 36 months of age. To ensure that associations truly reflected links to observed anger and not more general levels of negative emotionality, we created a directionality composite that accounted for levels of sadness observed during the two anger episodes. Sadness and anger were significantly correlated at both 6 months (r = 0.39, p < 0.01) and 12 months of age (r = 0.35, p < 0.01). Modeled after the procedures of Essex and colleagues (2011), directionality composites were formed by halving the difference between the standardized anger and sadness composites. Given that coding procedures did not require sadness and anger to be mutually exclusive, children who expressed high levels of negative emotion overall (i.e., both anger and sadness) rather than only anger would receive a directionality score near zero. In contrast, children who displayed greater relative anger would receive positive scores, with more positive values indicating greater anger; children who displayed greater relative sadness would receive negative scores, with more negative values indicating greater sadness. Thus, directionality composites reflect propensities for anger (relative to sadness) that are independent from overall levels of negative emotion (sadness plus anger). Most importantly, they allow a test of additional confidence that findings are linked specifically to infant anger. Substituting the directionality composite into bivariate analyses revealed that accounting for levels of sadness left associations between anger and behavior problems unchanged (6 months: r = 0.24, p < 0.01; 12 months: r = −0.00, p > 0.10). That is, it remained the case that only greater levels of 6-month anger were associated with more 36-month behavior problems.
Anger profiles were also significantly associated with behavior problems at 36 months of age (F(2, 218) = 3.23, p < 0.05, η2p = 0.03). Children in the normative, increasing anger group showed fewer behavior problems at age 36 months than did children in the high anger group (t = −8.38, p < 0.05, d = 0.60). This difference remained significant upon correction for multiple comparisons (Bonferroni).
Life Stress as a Moderator of the Link between Early Anger and Problem Behaviors
In our final set of analyses, we tested early life stress as a moderator of the association between observed anger in infancy and behavior problems during the preschool years. Traditional and profile measures of anger were tested in separate analyses. A 2-level mixed model was used to account for the nested nature of the data (twins within families).
First, we tested the model that included traditional measures of observed anger at 6 and 12 months of age. At 6 months of age, greater anger was associated with more behavior problems at 36 months of age (B = 0.10, SE(B) = 0.03, p < 0.01). Life stress did not predict preschool behavior problems (B = −0.01, SE(B) = 0.00, p > 0.05) and there was no significant interaction between anger and life stress (B = −0.01, SE(B) = 0.01, p > 0.05). At 12 months of age, neither anger (B = −0.01, SE(B) = 0.03, p > 0.10) nor life stress (B = −0.00, SE(B) = 0.00, p > 0.10) predicted behavior problems during preschool and no significant interaction was present (B = −0.01, SE(B) = 0.01, p > 0.10).
A second 2-level linear mixed model tested whether early life stress moderated links between profiles of infant anger and preschool behavior problems3. Consistent with the suggestions of Aiken and West (1991), anger profiles were dummy coded such that the increasing group, the normative developmental profile based on its frequency in the current sample and consistency with expectations based on the extant literature, served as the referent group. Continuous variables were centered prior to the creation of interaction terms.
As shown in Table 4, there was a significant interaction between life stress and both of the dummy variables (ΔR2 = 0.10) 4. To probe the nature of this moderation, we examined the association between early anger profiles and later behavior problems when life stress was re-centered at low (−1 SD) and high (+1 SD) levels (Aiken & West, 1991; Cohen & Cohen, 1983). Re-centering the life stress variable allowed us to examine its effect on the association between anger profiles and behavior problems as a continuous variable while eliminating the need to create arbitrary “high stress” and “low stress” groups. Results showed that at low levels of life stress, children in both the low anger (B = 0.34, SE(B) = 0.09, p < .01) and the high anger profiles (B = 0.20, SE(B) = 0.06, p < .01) showed greater behavior problems than children in the increasing anger profile (Figure 2). In contrast, at high levels of life stress, neither the low (B = −0.18, SE(B) = 0.10, p > .05) nor the high (B = 0.04, SE(B) = 0.06, p > .05) anger groups significantly differed from the increasing group in levels of behavior problems5.
Table 4.
B | SE(B) | b | t | ΔR2 | |
---|---|---|---|---|---|
Step 1. | 0.12** | ||||
Low Anger1 | 0.10 | 0.05 | 0.19 | 2.00* | |
High Anger2 | 0.12 | 0.04 | 0.28 | 2.91** | |
Life Stress | −0.01 | 0.00 | −0.17 | −1.73† | |
Step 2. | 0.10** | ||||
Low Anger1 | 0.08 | 0.05 | 0.15 | 1.57 | |
High Anger2 | 0.12 | 0.04 | 0.27 | 2.98** | |
Life Stress | −0.00 | 0.00 | −0.03 | −0.28 | |
Low Anger1 BY Life Stress | −0.04 | 0.01 | −0.28 | −3.02** | |
High Anger2 BY Life Stress | −0.01 | 0.01 | −0.20 | −2.04* |
Note:
Dummy variable tests low anger profile relative to increasing anger profile reference group
Dummy variable tests high anger profile relative to increasing anger profile reference group.
p < .10
p < .05
p < .01.
Discussion
Our overarching goal was to refine existing knowledge about early anger development and the ways that anger in infancy may serve as a risk factor for preschool behavior problems. We identified three unique profiles of anger during infancy: high anger, low anger, and increasing anger. We found that differences in preschool behavior problems could be predicted by novel anger profiles and early life stress; preschool behavior problems were inconsistently linked to traditional measures of anger. Remarkably for such a large sample, observations of more extreme, high levels of infant anger were not consistent predictors of later behavior problems. However, findings showed that early life stress moderated preschool behavior problems when anger measures reflected cross-context and cross-time variability. When early life stress was low, infants with atypical profiles of anger in infancy showed more preschool behavior problems than did typically-developing infants. At high levels of early life stress, infants with atypical profiles of anger showed levels of preschool behavior problems similar to typically-developing infants. Notably, these findings were not present in analyses using traditional measures of anger, which did not differentiate typical and atypical developmental profiles.
Individual Differences in Continuous and Categorical Measures of Anger
Consistent with hypotheses, we found evidence for three profiles during infancy. Profiles were formed from multi-measure composites of anger in two contexts at 6 and 12 months of age. A substantial proportion of infants (19%) showed atypical profiles of anger during this period. Atypical profiles of anger development, which do not adhere to the typical developmental course as defined in the literature, have largely gone unnoticed, and highlighting these atypical profiles is a major contribution of this work.
Characterizing individual differences in profiles of early anger also helped clarify the importance of early anger for later behavior problems. We found that overall levels of early anger, when quantified using traditional composite measures, did not consistently predict parent-reported behavior problems at age 36 months. In contrast, more novel profiles of anger that described cross-context and cross-time variability did show group differences in behavior problems. Children with typical profiles of anger showed fewer behavior problems than children with atypical anger profiles between 6 and 12 months of age, although subsequent analyses showed that this association was further moderated by early life stress.
We emphasize that expressions of anger, especially during infancy, do not necessarily reflect maladaptive tendencies. Anger typically increases during the 6–12 month period (Braungart-Rieker et al., 2010), suggesting that increases in anger are normative rather than problematic early in life. Our findings were consistent with these studies. We found that the majority of infants (81%) showed increases in displays of anger between 6 and 12 months of age in two laboratory contexts and, at the 12-month assessment, these infants were indistinguishable from other infants who were classified as stably high in anger. This overlap between “typical” and putatively-at-risk infants was precisely the type of underlying process that we suggested might be obscuring the relationship between levels of anger at 12 months and behavior problems two years later. Stated differently, high levels of anger at 12 months may not be predictive of later outcomes because, at this age, putatively at risk and typically developing children display similar levels of anger.
Profiles of Infant Anger, Early Life Stress, and Behavior Problems
Consistent with expectations, we found that levels of stress in the early environment moderated the association between anger in infancy and preschool behavior problems. Given low levels of early stress, the most important risk factor for preschool behavior problems was not levels of anger during infancy but whether the developmental profile of anger was normative or atypical. Children with non-normative profiles of anger in infancy had greater behavior problems at 36 months compared to those with more typical profiles of infant anger. This risk-related distinction between normative and atypical profiles of early anger was not seen at high levels of early family stress. Similarly, this moderated effect was not evident in analyses that used continuous measures of anger, underscoring the importance of differentiating individual differences in early emotion, particularly as it relates to normative and atypical development.
It is somewhat surprising finding that early life stress did not directly predict increased behavior problems in young children. It is possible that, in a typically-developing sample of young children, there is diminished overall variability in behavior problems which make significant effects more difficult to detect. It is also possible that our measure of life stress is one that is distal from the true mechanism linking life stress with behavior problems, making main effects more difficult to observe. However, this lack of a significant main effect may be better explained by the presence of the significant interaction between early life stress and behavior problems. That is, the interaction suggests that the development of behavior problems are dependent not only on life stress, but also on early anger. Statistically, the presence of moderation has been known to obscure the presence of main effects and make them more difficult to interpret (Aiken & West, 1991). This may be particularly true in cases where effects are nonlinear, suggesting that moderate levels of stress are adaptive while abnormally low or abnormally high exposure to stress may inhibit the development of regulatory mechanisms that help to keep behavior problems at bay.
Our results suggest that risk factors for developing behavior problems are distinct at different levels of life stress. Previous work has shown that the presence of risk factors such as low socioeconomic status, parent mental illness, or marital discord increase risk for behavior problems (Sameroff, Seifer, Barocas, Zax, & Greenspan, 1987). Each of these risk factors is captured to some extent in our measure of stressful life events. We point out that the PSI taps non-severe, relatively common life stressors rather than extreme or severe conditions of stress (e.g. maltreatment). Thus, a contribution of this work is the finding that atypical anger profiles may also serve as risk factors for —and alternate pathways toward—negative outcomes. When levels of life stress were low, the risk associated with atypical profiles of development were more tightly linked to childhood outcomes. However, atypical profiles did not appear to exacerbate the effects of high stress.
The mechanisms by which early life stressors might impact children’s functioning during infancy are the subject of a wide range of investigations across numerous fields of psychology. Animal and human models posit that stress may “get under the skin” in early development by disrupting the development of physiological systems, contributing to overall wear-and-tear on the body, altering the development of neural circuitry, and programming individual-level tendencies for hypo- or hyperactive stress responses (McEwen, 2012; Shirtcliff & Ruttle, 2010). Additionally, an environment of early stress may indirectly impact children by impeding parents’ abilities to interact with their children in ways that contribute to the development of independent regulation. Namely, greater poverty in the early environment has been linked to greater stress and less maternal sensitivity during parent child interactions (National Institute of Child Health and Human Development Early Child Care Research Network, 2005); less sensitive parents may be less likely to provide the scaffold that is necessary for children to learn to appropriately regulate negative emotions such as anger (Kopp, 1982). While these mechanisms are likely to play important roles in the link between anger and early behavior problems, direct tests of these possibilities will be important for future research. Similarly, the identification of more refined biological and social mechanisms of early risk remains a critical aim for developmental research.
To the extent that stress and atypical development represent independent risk factors, a lack of association between anger profiles and behavior problems at high levels of stress was somewhat surprising. Conceivably, facets of early life that we did not study, such as parent sensitivity and responsiveness (Campbell, Shaw, & Gilliom, 2000; Zeanah, Boris, & Larrieu, 1997) might further moderate the impact of early stressors on children’s outcomes. For example, despite continued adversity and stress in the family, youth with highly responsive parents during infancy were at decreased risk for disruptive behavior disorders relative to youth with unresponsive mothers (Wakschlag & Hans, 1999). Thus, under conditions of high stress, the degree to which parents can maintain sensitive, responsive orientation towards children may be an additional predictor of child outcomes than children’s anger profiles or life stress. Examinations of anger profiles within various parenting contexts will be an important avenue for future research.
Theories about how the social environment interacts with temperamental characteristics, such as anger, to predict mental health outcomes largely center on the idea that genetic and/or biological factors can shape individual sensitivity to environmental influences (Reiss, Leve, & Neiderhiser, 2013). We might ask whether our results are consistent with these theories. Our results do not implicate a single profile of early anger that is highly “susceptible” according to existing definitions. Additionally, while associations between profiles and behavior problems differed under conditions of high and low stress, these were not clear conditions of both disrupted and enriched development, which is a central tenet of biological susceptibility. That is, the identification of highly sensitive profiles depends on measures that include a full range of positive and negative environmental conditions. Although our measure of environmental stress enabled parents to report very high levels of early stress, they did not tap enriched environments on the same spectrum. Rather, our “low stress” environment better reflected a lack of environmental stressors.
Limitations
The current study is not without limitations. First, unequal numbers of infants in each of the profiles led to different degrees of power to detect significant differences across pairings of trajectory classes. Additionally, clinical assessments of parents and with children at follow-up are unavailable for the current sample. Thus, although we can test associations between early anger and parent-reported preschool behavior problems, the degree to which any trajectory is associated with subsequent diagnoses is unknown. Finally, although we do not view the twin design as a limitation, we recognize that there may be concerns that twin samples may not represent the general population on one or more aspects of behavior that may bias results in some systematic way. In fact, there are differences between twins and nontwins early in life in factors such as gestational age at birth, birth weight, and rate of language acquisition (Plomin, DeFries, McClearn, & McGuffin, 2008). However, many of these differences decline and disappear by the early school years. Results from twin studies largely generalize to nontwins (Andrew et al., 2001), including on measures relevant to the current work, such as early temperament (Goldsmith & Campos, 1990), personality (Johnson, McGue, Krueger, & Bouchard Jr., 2004) and psychopathology (Christensen, Vaupel, Holm, & Yashin, 1995).
Conclusion
Individual differences in profiles of anger are present during infancy and are associated with variability in behavior problems at age 36 months. In addition, at low – but not high - levels of life stress, non-normative patterns of infant anger predicted greater parent-reported preschool behavior problems. Our results provide additional evidence that stress in the early family context, even when not severe, is related to developmental outcomes. Importantly, this work underscores the need to consider individual differences, both in early experiences and early profiles of anger, in studies of developing behavior problems in children.
Acknowledgments
Data collection for this project was supported by R01 MH50560 from the National Institute of Mental Health (PI: Goldsmith). The writing of this manuscript was partially supported by T32 MH018931 (PI: Davidson) and K01 MH100240 (PI: Brooker) from the National Institute of Mental Health. Infrastructure support was also provided by P30 HD03352 and P50 MH084051. Salary support was received by N. Aksan from the Toyota Motor Company.
We thank the families who participated in this study and the staff members who helped with the recruitment of study participants and data collection.
Footnotes
Results are unchanged when limited to those individuals with data at both 6 and 12 months.
The formation of each composite was also supported by a principal components analysis (PCA). Additional details of each PCA are available from authors.
For completeness, and to ensure that analyses were not biased by global levels of negative affect, we also tested a model that substituted the directionality composites for traditional measures of anger. Similar to analyses using traditional measures, greater anger at 6 months of age was associated with more behavior problems at 36 months of age (B = 0.18, SE(B) = 0.08, p < 0.05). Life stress did not predict preschool behavior problems (B = −0.00, SE(B) = 0.00, p > 0.10) and there was no significant interaction between the directionality composite and life stress (B = −0.02, SE(B) = 0.01, p > 0.10). At 12 months of age, neither the directionality composite (B = −0.09, SE(B) = 0.07, p > 0.10) nor life stress (B = −0.00, SE(B) = 0.00, p > 0.10) predicted behavior problems during preschool and no significant interaction was present (B = −0.01, SE(B) = 0.01, p > 0.10).
An R2 calculation is not a part of the estimation procedure for mixed models; therefore, we calculated R2 as the decrease in the proportion of error variance between models that did and did not include the life stress X anger class interactions. Alternatively, rerunning the analyses as a Stepwise regression, which does not account for twin relatedness but allows for the estimation of R2, returns an identical estimate of ΔR2 and a probability level of p < .01.
Recoding the dummy variable to test differences between low and high anger profiles revealed no differences in behavior problems at low levels of stress (B = −0.14, SE(B) = 0.10, p > .10) and a trend toward more behavior problems in the high anger profile than the low anger profile at high levels of life stress (B = 0.22, SE(B) = 0.11, p = .05).
Contributor Information
Rebecca J. Brooker, Montana State University
Kristin A. Buss, The Pennsylvania State University
Kathryn Lemery-Chalfant, Arizona State University.
Nazan Aksan, University of Iowa.
Richard J. Davidson, University of Wisconsin–Madison
H. Hill Goldsmith, University of Wisconsin–Madison.
References
- Achenbach TM, Rescorla LA. Manual for the ASEBA Preschool Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families; 2000. [Google Scholar]
- Aiken LS, West SG. Multiple Regression: Testing and Interpreting Interactions. Thousand Oaks, CA: Sage Publications; 1991. [Google Scholar]
- Alessandri SM, Sullivan MW, Lewis M. Violation of expectancy and frustration in early infancy. Developmental Psychology. 1990;26(5):738–744. [Google Scholar]
- Alink LRA, Mesman J, Van Zeijl J, Stolk MN, Juffer F, Koot HM, Van IJzendoorn MH. The Early Childhood Aggression Curve: Development of Physical Aggression in 10- to 50-Month-Old Children. Child Development. 2006;77(4):954–966. doi: 10.1111/j.1467-8624.2006.00912.x. [DOI] [PubMed] [Google Scholar]
- Andrew T, Hart DJ, Snieder H, de Lange M, Spector TD, MacGregor AJ. Are Twins and Singletons Comparable? A Study of Disease-related and Lifestyle Characteristics in Adult Women. Twin Research and Human Genetics. 2001;4(06):464–477. doi: 10.1375/1369052012803. [DOI] [PubMed] [Google Scholar]
- Attar BK, Guerra NG, Tolan PH. Neighborhood disadvantage, stressful life events and adjustments in urban elementary-school children. Journal of Clinical Child Psychology. 1994;23(4):391–400. [Google Scholar]
- Belsky J, Friedman SL, Hsieh K-H. Testing a Core Emotion-Regulation Prediction: Does Early Attentional Persistence Moderate the Effect of Infant Negative Emotionality on Later Development? Child Development. 2001;72(1):123–133. doi: 10.1111/1467-8624.00269. [DOI] [PubMed] [Google Scholar]
- Braungart-Rieker JM, Hill-Soderlund AL, Karrass J. Fear and anger reactivity trajectories from 4 to 16 months: The roles of temperament, regulation, and maternal sensitivity. Developmental Psychology. 2010;46(4):791–804. doi: 10.1037/a0019673. [DOI] [PubMed] [Google Scholar]
- Campbell SB, Shaw DS, Gilliom M. Early Externalizing Behavior Problems: Toddlers and Preschoolers at Risk for Later Maladjustment. Development and Psychopathology. 2000;12(03):467–488. doi: 10.1017/s0954579400003114. [DOI] [PubMed] [Google Scholar]
- Campos JJ, Mumme DL, Kermoian R, Campos RG. A functionalist perspective on the nature of emotion. Monographs of the Society for Research in Child Development. 1994;59(2–3):284–303. [PubMed] [Google Scholar]
- Camras LA. Expressive development and basic emotions. Cognition & Emotion. 1992;6(3–4):269–283. [Google Scholar]
- Christensen K, Vaupel JW, Holm NV, Yashin AI. Mortality among twins after age 6: Fetal origins hypothesis versis twin method. British Medical Journal. 1995;310(6977):432–436. doi: 10.1136/bmj.310.6977.432. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cicchetti D, Aber JL. Contextualism and developmental psychopathology. Development and Psychopathology. 1998;10(02):137–141. doi: 10.1017/s0954579498001540. [DOI] [PubMed] [Google Scholar]
- Cohen J, Cohen P. Applied multiple regression/correlation analyses for the behavioral sciences. 2nd ed. Hillsdale, NJ: Lawren; 1983. [Google Scholar]
- Cole PM, Michel MK, Teti LO. The Development of Emotion Regulation and Dysregulation: A Clinical Perspective. Monographs of the Society for Research in Child Development. 1994;59(2–3):73–102. [PubMed] [Google Scholar]
- Cole PM, Teti L, Zahn-Waxler C. Mutual emotion regulation and the stability of conduct problems between preschool and early school age. Development and Psychopathology. 2003;15(01):1–18. [PubMed] [Google Scholar]
- Cole PM, Zahn-Waxler C, Fox NA, Usher BA, Welsh JD. Individual differences in emotion regulation and behavior problems in preschool children. Journal of Abnormal Psychology. 1996;105(4):518–529. [PubMed] [Google Scholar]
- Compas BE, Connor-Smith JK, Saltzman H, Thomsen AH, Wadsworth ME. Coping with stress during childhood and adolescence: Problems, progress, and potential in theory and research. Psychological Bulletin. 2001;127(1):87–127. [PubMed] [Google Scholar]
- Denham SA, Caverly S, Schmidt M, Blair K, DeMulder E, Caal S, Mason T. Preschool Understanding of emotions: Contributions to classroom anger and aggression. Journal of Child Psychology & Psychiatry & Allied Disciplines. 2002;43(7):901–916. doi: 10.1111/1469-7610.00139. [DOI] [PubMed] [Google Scholar]
- Denham SA, Lehman EB, Moser MH, Reeves SL. Continuity and change in emotional components of infant temperament. Child Study Journal. 1995;25(4):289–308. [Google Scholar]
- Eisenberg N, Cumberland A, Spinrad TL, Fabes RA, Shepard SA, Reiser M, Guthrie IK. The Relations of Regulation and Emotionality to Children’s Externalizing and Internalizing Problem Behavior. Child Development. 2001;72(4):1112–1134. doi: 10.1111/1467-8624.00337. [DOI] [PubMed] [Google Scholar]
- Gillom M, Shaw DS. Codevelopment of externalizing and internalizing problems in early childhood. Development and Psychopathology. 2004;16(02):313–333. doi: 10.1017/s0954579404044530. [DOI] [PubMed] [Google Scholar]
- Goldsmith HH, Campos JJ. The structure of temperamental fear and pleasure in infants: A psychometric perspective. Child Development. 1990;61:1944–1964. [PubMed] [Google Scholar]
- Gore S, Aseltine RH, Jr, Colten ME. Social Structure, Life Stress, and Depressive Symptoms in a High School-Age Population. Journal of Health and Social Behavior. 1992;33(2):97–113. [PubMed] [Google Scholar]
- Hauser-Cram P, Warfield ME, Shonkoff JP, Krauss MW. Children with disabilities: A longitudinal study of child development and parent well-being. Monographs of the Society for Research in Child Development. 2001;66(3):1–131. [PubMed] [Google Scholar]
- Izard CE. Human emotion. New York: Plenum Press; 1977. [Google Scholar]
- Jackson Y, Warren JS. Appraisal, Social Support, and Life Events: Predicting Outcome Behavior in School-Age Children. Child Development. 2000;71(5):1441–1457. doi: 10.1111/1467-8624.00238. [DOI] [PubMed] [Google Scholar]
- Johnson W, McGue M, Krueger RF, Bouchard TJ., Jr Marriage and personality: A genetic analysis. Journal of Personality and Social Psychology. 2004;86(2):285–294. doi: 10.1037/0022-3514.86.2.285. [DOI] [PubMed] [Google Scholar]
- Kopp CB. Antecedents of self-regulation: A developmental perspective. Developmental Psychology. 1982;18(2):199–214. [Google Scholar]
- Lemery KS, Essex MJ, Smider NA. Revealing the Relation between Temperament and Behavior Problem Symptoms by Eliminating Measurement Confounding: Expert Ratings and Factor Analyses. Child Development. 2002;73(3):867–882. doi: 10.1111/1467-8624.00444. [DOI] [PubMed] [Google Scholar]
- Levenson Robert W. Autonomic specificity and emotion. In: Davidson RJ, Scherer KR, Goldsmith HH, editors. Handbook of Affective Sciences. New York, New York: Oxford University Press; 2003. pp. 214–224. [Google Scholar]
- Lewis M. The Development of Intentionality and the Role of Consciousness. Psychological Inquiry. 1990;1(3):230–247. [Google Scholar]
- Loyd BH, Abidin RR. Revision of the Parenting Stress Index. Journal of Pediatric Psychology. 1985;10(2):169–177. doi: 10.1093/jpepsy/10.2.169. [DOI] [PubMed] [Google Scholar]
- Malatesta CZ, Culver C, Tesman JR, Shepard B. The development of emotion expression during the first two years of llife. Monographs of the Society for Research in Child Development. 1989;54(1–2):1–104. [PubMed] [Google Scholar]
- McEwen BS. Brain on stress: How the social environment gets under the skin. Proceedings of the National Academy of Sciences. 2012;109(Supplement 2):17180–17185. doi: 10.1073/pnas.1121254109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- National Institute of Child Health and Human Development Early Child Care Research Network. Duration and developmental timing of poverty and children’s cognitive and social development from birth through third grade. Child Development. 2005;76(4):795–810. doi: 10.1111/j.1467-8624.2005.00878.x. [DOI] [PubMed] [Google Scholar]
- Oster H. The repertoire of infant facial expression: An ontogenetic perspective. In: Nadel J, Muir D, editors. Emotional Development. New York, New York: Oxford University Press; 2005. pp. 261–292. [Google Scholar]
- Plomin R, DeFries JC, McClearn GE, McGuffin P. Behavioral Genetics. 5th edition. New: Worth Publishers; 2008. [Google Scholar]
- Reiss D, Leve LD, Neiderhiser JM. How genes and the social environment moderate each other. American Journal of Public Health. 2013;103(S1):S111–S121. doi: 10.2105/AJPH.2013.301408. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sameroff AJ, Seifer R, Barocas R, Zax M, Greenspan S. Intelligence quotient scores of 4-year-old children: Social-environmental risk factors. Pediatrics. 1987;79(3):343–350. [PubMed] [Google Scholar]
- Schmidt NL, Van Hulle CA, Brooker RJ, Meyer LR, Lemery-Chalfant K, Goldsmith HH. Wisconsin Twin Research: Early Development, Childhood Psychopathology, Autism, and Sensory Over-Responsivity. Twin Research and Human Genetics. 2013;16(Special Issue 01):376–384. doi: 10.1017/thg.2012.105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shirtcliff EA, Ruttle P. Immunological and neuroendocrine dysregulation following early deprivation and stress. Attachment and Early Disorders of Development, Klett-Cotta, Stuttgart, Munich. 2010 [Google Scholar]
- Stein NL, Jewett JJ. A conceptual analysis of the meaning of negative emotions: Implications for a theory of development. In: Izard CE, Read PB, editors. Measuring emotions in infants and children. Vol. 2. New York, New York: Cambridge University Press; 1986. pp. 238–267. [Google Scholar]
- Swearingen EM, Cohen LH. Life events and psychological distress: A prospective study of young adolescents. Developmental Psychology. 1985;21(6):1045–1054. [Google Scholar]
- Vaux A, Ruggiero M. Stressful life change and delinquent behavior. American Journal of Community Psychology. 1983;11(2):169–183. doi: 10.1007/BF00894365. [DOI] [PubMed] [Google Scholar]
- Wakschlag LS, Hans SL. Relation of maternal responsiveness during infancy to the development of behavior problems in high-risk youths. Developmental Psychology. 1999;35(2):569–579. doi: 10.1037//0012-1649.35.2.569. [DOI] [PubMed] [Google Scholar]
- Wang X, Trivedi R, Treiber F, Sneider H. Genetic and environmental influences on anger expression, John Henryism, and stressful life events: The Georgia cardiovascular twin study. Psychosomatic Medicine. 2005;67(1):16–23. doi: 10.1097/01.psy.0000146331.10104.d4. [DOI] [PubMed] [Google Scholar]
- Zeanah CH, Boris NW, Larrieu JA. Infant Development and Developmental Risk: A Review of the Past 10 Years. Journal of the American Academy of Child & Adolescent Psychiatry. 1997;36(2):165–178. doi: 10.1097/00004583-199702000-00007. [DOI] [PubMed] [Google Scholar]