Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Mar 1.
Published in final edited form as: Infancy. 2011 Mar;16(2):198–210. doi: 10.1111/j.1532-7078.2010.00036.x

Toddlers’ Duration of Attention towards Putative Threat

Elizabeth J Kiel 1, Kristin A Buss 2
PMCID: PMC3045210  NIHMSID: NIHMS200839  PMID: 21373365

Abstract

Although individual differences in reactions to novelty in the toddler years have been consistently linked to risk for developing anxious behavior, toddlers’ attention towards a novel, putatively threatening stimulus while in the presence of other enjoyable activities has rarely been examined as a precursor to such risk. The current study examined how attention towards an angry-looking gorilla mask in a room with alternative opportunities for play in 24-month-old toddlers predicted social inhibition when children entered kindergarten. Analyses examined attention to threat above and beyond and in interaction with both proximity to the mask and fear of novelty observed in other situations. Attention to threat interacted with proximity to the mask to predict social inhibition, such that attention to threat most strongly predicted social inhibition when toddlers stayed furthest from the mask. This relation occurred above and beyond the predictive relation between fear of novelty and social inhibition. Results are discussed within the broader literature of anxiety development and attentional processes in young children.

Individual differences in responses to novelty (e.g., distress, facial and bodily fear, avoidance) during early childhood have been linked to later anxiety-spectrum outcomes such as social inhibition (Buss, Davidson, Kalin, & Goldsmith, 2004; Buss & Goldsmith, 1998; Calkins & Fox, 1992; Crockenberg & Leerkes, 2004; Kagan, Reznick, Clarke, Snidman, & Garcia-Coll, 1984; Kagan, Snidman, Zentner, & Peterson, 1999; Rubin, Burgess, & Hastings, 2002). Although sustained attention to putatively threatening novelty relates to anxious behavior in the first two years of life (Crockenberg & Leerkes, 2006), we know less about how attention towards putative threat at the end of the infancy period predicts anxious behavior later in childhood above and beyond other indices of fearfulness. The current study addressed these gaps and contributed to the extant literature by providing a longitudinal examination between toddlerhood and kindergarten entry of the relation between attention to putative threat and social inhibition in the context of other fear variables: fear of novelty and proximity to the object of attention.

Upon encountering a distressing novel stimulus, young children may avert their gazes or engage in distraction. Individual differences in these behaviors emerge early in infancy and seemingly regulate arousal. As early as 6 months of age, disengaging attention from the subjective threat relates to decreased distress in the moment, whereas maintained attention towards it relates to sustained distress (Crockenberg & Leerkes, 2004; Johnson, Posner, & Rothbart, 1991; but see Buss & Goldsmith, 1998) and predicts anxious behavior in the toddler period (Crockenberg & Leerkes, 2006). Toddler-aged children have increased motoric skill and independence in exploring their environments (Kopp, 1982), so they are capable of using more sophisticated distraction techniques such as involvement with other activities. Given this, toddlers’ sustained attention towards putative threat may indicate ineffective regulation or reliance on coping patterns (e.g., vigilance) predictive of future anxious behavior.

Future anxious behavior may be displayed as social inhibition, which characterizes children who display reticence, shyness, and similar behaviors with their peers and predicts risk for later social anxiety (Hirshfeld-Becker et al., 2008; Kagan et al., 1984). Attention to putative social threat occurs in socially inhibited children. Specifically, in preschool and school-aged children, “onlooking” behavior with peers while avoiding interaction is a primary feature of inhibition/reticence (Coplan, Rubin, Fox, Calkins, & Stewart, 1994; Fox, Henderson, Rubin, Calkins, & Schmidt, 2001; Hane, Cheah, Rubin, & Fox, 2008; Rubin et al., 2002). Whether attention towards putative threat earlier in development predicts social inhibition remains unknown.

To provide a more robust test, the predictive relation between attention to putative threat and later social inhibition should be considered above and beyond and in interaction with other early indicators of vulnerability towards anxiety. Fear of novelty, often studied as the broader construct of fearful/inhibited temperament (Kagan et al., 1984) and highlighted in situations that involve uncertainty but do not universally elicit distress (Buss, 2010; Buss et al., 2004), is a robust predictor of later social inhibition and anxiety. The relation between fear of novelty and social inhibition may indicate stability in an inhibited disposition. Although fear of novelty is typically characterized as expressions of fear and avoidance behavior, fearful tendencies may also be exhibited as vigilance for novel, potentially threatening objects in the environment (i.e., attention to threat). Not all temperamentally fearful children would be expected to be vigilant, but vigilance may serve as an additional risk for social inhibition for children also displaying high levels of fear expressions and avoidance indicative of fear of novelty. Thus, fear of novelty may interact with attention in relation to social inhibition. Crockenberg and Leerkes (2006) found that infant reactivity to novelty predicted anxious behavior in toddlerhood at low levels but not at higher levels of looking away from a novel toy. These results suggest that, when toddlers display higher levels of fear of novelty, increased duration of looking towards a potential threat would predict social inhibition.

The maximum proximity to which children approach novel stimuli has also been identified as predictive of later patterns of avoidance characteristic of social inhibition. Maintaining distance from novel stimuli may serve to decrease distress in the short-term (Buss & Goldsmith, 1998) but may ultimately reinforce withdrawal and avoidance of uncertainty (Thompson & Calkins, 1996). Proximity to novelty has moderated how other behaviors predict anxiety, with distress and activity predicting anxious behavior only in the context of maintained distance from novelty (Crockenberg & Leerkes, 2006). Likewise, proximity may interact with attention in relation to social inhibition. Specifically, some children may attend to novelty while avoiding it, indicating an anxious response predictive of later inhibition, whereas other children may show greater durations in looking towards a stimulus while actively exploring it, indicating more uninhibited or bold behavior. Thus, fear of novelty and proximity to the threatening stimulus should be included when predicting social inhibition, and examination of moderating effects is warranted.

In summary, greater durations of attention towards potentially threatening novelty in infancy, especially in the presence of more enjoyable activities, may interfere with the regulation of distress and predict anxious behavior in the toddler years. Early attention towards threat may indicate risk for later social problems because social inhibition in school-aged children is manifested through maintained attention towards subjectively threatening social situations (e.g., Hane et al., 2008). The current study aimed to examine the missing developmental link in this line of research: whether individual differences in the duration of toddlers’ attention towards a novel, putatively threatening stimulus in the presence of more enjoyable options for play predicts social inhibition in kindergarten. This predictive relation was examined alongside other potential influences on anxiety development (i.e., fear of novelty, proximity to the threatening stimulus). We hypothesized that increased attention towards the stimulus would predict social inhibition above and beyond the other variables. We also hypothesized that attention would interact with other fear-related variables, such that attention would be most related to social inhibition at higher levels of fear of novelty and lower levels of proximity to the putatively threatening stimulus.

Method

Participants

One hundred and ten toddlers (Mage = 25.10 months, SDage = 1.63 months; 48 female) and their primary caregivers (91% mothers), recruited from published birth records, participated in a laboratory visit. Participants were middle class (Hollingshead: M = 47.88, SD = 11.14) and 90% Caucasian. Eighty-two families participated in a kindergarten follow-up assessment involving parent- and teacher-report of children’s social inhibition.

Procedure

Toddler laboratory visit

Mothers completed consent and a demographics questionnaire, and the experimenter provided a detailed explanation of the procedure. The experimenter then led the mother and toddler to a room (the Risk Room) with four relatively neutral activities (i.e., tunnel, trampoline, balance beam, black box) and one putatively threatening stimulus, an angry-looking gorilla mask on a pedestal in the corner of the room. One of the walls adjoining this corner contained a one-way mirror behind which the camera recorded the episode. Toddlers’ looks to the mask were easily discernable, as they had to use a different angle to look at the mask than at the mirror or any other activities. The experimenter asked the mother to sit in a chair in the opposite corner and to minimize interactions with her toddler. The experimenter told the toddler to play in the room “however you like” and left the child to play for 3 minutes. Then, the experimenter returned and asked the child to engage with each of the activities, ending with the gorilla mask.

Toddlers also participated in two novelty tasks, Clown and Puppet Show. Adapted from previous studies of toddler wariness (e.g., Nachmias, Gunnar, Mangelsdorf, Parritz, & Buss, 1996), these episodes elicit individual differences in fearfulness. For Clown, the toddler began seated in the mother’s lap, and an unfamiliar female experimenter (different than who facilitated the Risk Room) entered at the opposite end of the room (about 10 feet away), dressed in a clown costume. With a friendly demeanor, she introduced herself and invited the child to join three one-minute activities: popping bubbles, playing catch, and playing with musical instruments. For Puppet Show, the child began seated in the mother’s lap in the opposite corner of a small wooden stage with a curtain. From behind the stage, the second female experimenter conducted a puppet show with two friendly animal puppets who invited the child to play three one-minute games: catch, fishing, and presentation of a sticker.

Kindergarten assessment

In the fall, mothers completed a new consent form and questionnaire assessing their children’s symptoms and adjustment. Mothers provided written permission to contact their children’s kindergarten teachers. Teachers were contacted by mail in the fall and again in the spring to complete the same questionnaire.

Measures

Attention to threat

Trained coders documented the target of toddlers’ attention each second of the Risk Room. Coders scored attention to threat only when the child directly looked at the gorilla mask or when they spoke to the parent about it. A master coder double-scored approximately 15% of cases, yielding adequate reliability (92% agreement, κ = .79). The proportion of time attending to the mask across the Risk Room episode (except when the experimenter asked the child to engage with the gorilla mask) yielded the measure of attention to threat (AT).

Proximity to threat

The maximum proximity to the mask demonstrated by each child across both phases of the Risk Room was coded on a 3-point scale (0 = always beyond 2 feet, 1 = within 2 feet, 2 = touches mask). Reliability computed on approximately 15% of cases between coders and a master coder was found to be adequate (89% agreement, κ = .78).

Fear of novelty

As part of a larger study, we have previously established a fear of novelty (FN) composite, composed of the behaviors described subsequently. Analyses establishing this composite have been reported previously (Buss, 2010). Behaviors were scored on a second-by-second basis throughout both Clown and Puppet Show so that latency (total number of seconds until the behavior first occurred) and duration (total number of seconds the behavior was displayed throughout the episode) could be derived. Before coding independently, coders were required to establish minimum training reliability (90% agreement, κ = .65) with a master coder. Inter-rater reliability was computed throughout coding to prevent coder drift. Reliability was computed on 15% of cases as percent-agreement because kappa calculations may be biased due to the large number of zero values in second-by-second micro-coding (Lantz & Nebenzahl, 1996). Reliability was found to be adequate (Clown: 86 – 89%, Puppet Show: 86 – 91%).

Facial fear expressions were scored according to the AFFEX system (Izard, Dougherty, & Hembree, 1983), based on brows (raised and slightly drawn together), eyelids (raised or tense), and mouth (open with corners pulled straight back). Bodily fear was scored when the child displayed decreased activity or tenseness. Freezing was scored as displays of bodily rigidity (e.g., limbs or body appears stiff or frozen in an awkward position) that lasted at least 2 seconds. Coders were trained to attend to differences between freezing and natural orienting to stimuli. Proximity to mother was scored when children were within 2 feet or touching their mothers. The final composite for each episode was formed as the mean of latency to freeze (reversed) and durations of freezing, facial fear, bodily fear, and proximity to mother, adjusted for total length of the episode for each child, yielding a proportion score. Because these episodes had similar demand characteristics, and in order to provide increased reliability of observation (rather than relying on a single context), the mean of the Clown and Puppet Show composites (r = .36, p < .001) yielded the final FN variable.

Social inhibition

Parent and teacher versions of the Health Behavior Questionnaire (HBQ; Armstrong, Goldstein, & the McArthur Working Group on Outcome Assessment, 2003) provided assessments of children’s social inhibition. Parents and teachers rated 140 and 115 items, respectively, describing children’s behaviors (0 = never or not true, 1 = sometimes or somewhat true, 2 = often or very true). The current study focused on the Social Inhibition scale (e.g., “Shy with other children”; 3 items, αparent = .77 and αteacher = .69). The parent and teacher versions of the Social Inhibition scale have previously been found to have adequate test-retest reliability (rs = .68, .79 respectively), modest cross-informant agreement (r = .22), convergent validity (i.e., positive associations with internalizing problems and negative associations with school adaptation) and discriminant validity in differentiating groups of children at different levels of risk for maladaptation (Essex et al., 2002).

Results

Preliminary Analyses

Missing data and attrition

Two of the 110 toddlers could not be scored for AT due to technical problems. Of these 108 participants, 26 families were lost to attrition at the kindergarten assessment, primarily because they had moved and could not be located. Of the 82 families who participated in the kindergarten assessment, all had complete parent-report data, and 62 had at least one teacher-report questionnaire (nfall=53, nspring = 52, nboth = 43). Missing teacher data occurred primarily because teachers never responded to initial or follow-up letters. The 26 toddlers whose families did not complete the follow-up did not differ from those who did in terms of SES, AT, or FN, but they did have lower scores on proximity to threat (t[106] = − 3.39, p < .01). The 46 participants missing both teacher-report questionnaires did not differ from the 62 who had at least one on any variables (all ps > .20).

Because missing data likely occurred at random (Little’s MCAR test: χ2[21] = 22.30, p = .38) and restricting analyses to participants with complete data would decrease the sample size and likely result in parameter bias, we followed current guidelines (Howell, 2007; Jeličić, Phelps, & Lerner, 2009; Widaman, 2006) and derived values for missing data using multiple imputation. Across a high number of imputations, multiple imputation provides relatively unbiased results because it preserves the characteristics of the sample, moreso than listwise deletion or other methods (Jeličić et al., 2009; Widaman, 2006). Data were imputed across 20 imputations (Graham, Olchowski, & Gilreath, 2007) using SES, gender, AT, FN, proximity to mask, and available kindergarten data. Thus, all 108 participants had data for all analyses.

Data reduction

Teacher-reported social inhibition scores were related across fall and spring (r[106] = .54, p < .001) so the mean of these reports was taken to comprise teacher-reported social inhibition. Teacher-report subsequently related to parent-reported social inhibition (r[106] = .38, p < .001). We used the mean of teacher- and parent-report to facilitate examination of social inhibition assessed by multiple reporters. Although not reported, subsequent analyses were also conducted separately by reporter, and findings were comparable.

Descriptive statistics

Table 1 presents descriptive statistics and bivariate relations among variables. No gender differences existed for any of the variables (all ps > .40), and SES (Hollingshead Index) did not relate to social inhibition (r[106] = −.06, ns). Gender and SES were thus not considered in further analyses.

Table 1.

Descriptive Statistics and Bivariate Relations for Primary Variables

Variable Mean (SD) Range 2 3 4 5
1. SES 47.91 (11.17) 13.00 – 66.00 .07 .10 .20* −.06
2. Attention 0.15 (0.11) 0.00 – 0.48 -- .05 −.03 .18
3. Fear of novelty 23.83 (18.38) 1.16 – 85.12 -- −.11 .23*
4. Proximity to threat 0.72 (0.91) 0.00 – 2.00 -- −.10
5. Social inhibition 0.62 (0.30) 0.00 – 1.67
Open in a new tab

Note. All statistics were computed after imputation of missing data (n = 108 for all analyses). SES was derived as the Hollinghead Index. Attention was the proportion of seconds toddlers attended to the gorilla mask. Fear of novelty comprised the mean of behaviors scored as durations or latencies in seconds. Proximity to threat was scored on a 0 to 2 scale with higher scores indicating closer proximity. Social inhibition scores ranged from 0 to 2 with higher scores indicating more inhibition. SES, Attention, Fear of novelty, and Proximity to mask were measured at 24 months. Assessment of Social inhibition occurred during children’s kindergarten year.

*

p < .05,

**

p < .01.

Primary Analyses

A hierarchical multiple regression was performed to examine whether AT predicted social inhibition above and beyond or in interaction with other risks for anxiety. AT and FN were centered at their means prior to analyses. Proximity to threat had a meaningful zero-point, so it was not centered. Table 2 summarizes results of this regression.

Table 2.

Hierarchical Regression Model Predicting Kindergarten Social Inhibition

Model Coefficient
Variable ΔR2 R2 95% CI F b (SE) 95% CI β t
Step 1 .09* .09 −.01, .19 3.35*
    Fear of novelty 0.01 (0.00) 0.00, 0.01 0.22 2.28*
    Proximity to threat −0.03 (0.03) −0.09, 0.04 −0.04 −0.46
    Attention to threat 0.49 (0.27) −0.04, 1.02 0.17 1.82†
Step 2 .05* .14 .03, .25 3.36*
    Fear of novelty 0.01 (0.00) 0.00, 0.01 0.23 2.49*
    Proximity to threat −0.02 (0.03) −0.08, 0.04 −0.07 −0.71
    Attention to threat 0.90 (0.32) 0.28, 1.53 0.31 2.86**
    Attention to threat X Fear of novelty −0.02 (0.02) −0.05, 0.02 −0.08 −0.88
    Attention to threat X Proximity to threat −0.68 (0.30) −1.28, −0.09 −0.25 −2.28*
Simple Slopes
    Attention to threat at Proximity to threat = 0 0.90 (0.32) 0.28, 1.53 0.31 2.86**
    Attention to threat at Proximity to threat = 1 0.22 (0.30) −0.38, 0.80 0.08 0.46
    Attention to threat at Proximity to threat = 2 −0.47 (0.51) −1.47, 0.54 −0.16 −0.92
Open in a new tab

Note. CI = Confidence Interval. df = (3, 104), and (5, 102) for Steps 1 and 2, respectively. Simple slopes of Attention were examined by recentering proximity at the values listed in the table.

†

p < .10,

*

p < .05,

**

p < .01

Step 1 contained the main effects of FN, proximity to threat, and AT and, as a whole, explained significant variance in social inhibition. AT and FN each independently predicted social inhibition.

Step 2 contained the two-way interactions between AT and each of the other fear variables. This step accounted for a significant change in the variance in social inhibition explained by the model. An interaction existed between AT and proximity to threat, above and beyond the still significant effect of FN. We examined simple slopes of AT predicting social inhibition across recentered values of proximity to threat (Table 2, Figure 1). With proximity centered at 0 (always beyond 2 feet from the mask), AT predicted social inhibition. AT did not predict social inhibition with proximity centered at 1 (came within 2 feet but did not touch the mask), or with proximity centered at its maximum value of 2 (touching the mask). Thus, AT at 24 months predicted social inhibition in kindergarten above and beyond other known risks for anxiety observed at 24 months, and this relation only occurred when toddlers refrained from approaching the target of their attention.

Figure 1.

Figure 1

Open in a new tab

When they also refrained from approaching it, toddlers’ increased attention towards a putatively threatening novel stimulus predicted kindergarten social inhibition. ** p < .01.

Discussion

Previous research has shown that attention to novel stimuli relates to anxious behavior in the first years of life as well as concurrently in older children. This relation has not been examined between late infancy and kindergarten, both important developmental periods for assessing individual differences in risk for anxious behavior. The current study found that increased attention towards a putatively threatening stimulus in the presence of other enjoyable activities predicted social inhibition in kindergarten. Moreover, children’s proximity to the stimulus moderated this association: attention predicted social inhibition when children remained at a distance from it but not when children approached it closely.

The relation between attention to putative threat and later social inhibition is consistent with previous research showing that children described as socially inhibited or reticent display “onlooker behavior” on the periphery of peer groups (Coplan et al., 1994; Fox et al., 2001; Rubin et al., 2002). This type of behavior may be thought of as a social version of orientation to potential threat (i.e., negative social cues, rejection). Despite the non-social nature of the looking behavior observed in toddlers, it appears to have implications for social behavior when children enter a developmental period marked by increased demands for peer interaction. It should be noted that the current study assessed social inhibition generally and not specifically with novel peers, as has been done in previous studies. Future research would augment this work by examining this specific relation.

Several studies have established the regulatory function of attention (Crockenberg & Leerkes, 2004; Johnson et al., 1991), suggesting that children who break their attention away from threat may be more adeptly regulating their internal arousal. We did not explicitly examine regulation, although establishing the regulatory nature of the type of attention examined presently might provide further information about the mechanisms by which early attention relates to later social inhibition. Moreover, although maintaining distance from novelty has been identified as a strategy fearful children use to regulate their arousal, doing so reinforces later withdrawal and avoidance of new situations. Although the relation between proximity to the mask and later social inhibition reported here was not statistically significant, it was in the expected negative direction. Similarly, it could be that maintaining vigilance towards novelty regulates arousal in the moment but ultimately fosters maladaptive behavioral patterns. Further identifying the regulatory processes related to attention would likely clarify whether attention might be targeted in early interventions to prevent social problems indicative of later anxiety.

The low teacher response rate and the use of a three-item scale for the measure of social inhibition were limitations in the current study. We used the most current recommendations for imputation of missing data and multiple reporters of social inhibition, but these results should be replicated with additional assessments of social inhibition.

Conclusions

Toddlers’ greater durations of attention to an avoided novel, potentially threatening stimulus, despite the presence of enjoyable activities, predicted social inhibition in kindergarten. This predictive relation occurred above and beyond an association between fear of novelty and social inhibition. Thus, in conjunction with more general distress to novelty, observed attention to threat in the toddler years may help identify children at risk for the development of anxiety-spectrum problems.

Acknowledgments

This study was funded by a grant to K.B. from NIH (R01 MH075750) and E.K. was funded by a predoctoral NRSA grant from NIH (F31 MH077385) when she was at University of Missouri-Columbia. We express our appreciation to the families and toddlers who participated in this project and to the Emotion Development Labs at University of Missouri-Columbia and The Pennsylvania State University for help with this project.

Contributor Information

Elizabeth J. Kiel, University of Mississippi Medical Center

Kristin A. Buss, The Pennsylvania State University

References

  1. Armstrong JM, Goldstein LH The MacArthur Working Group on Outcome Assessment. Manual for the MacArthur Health and Behavior Questionnaire. MacArthur Foundation Research Network on Psychopathology and Development (David J.Kupfer, Chair), University of Pittsburgh; 2003. [Google Scholar]
  2. Buss KA. Which fearful toddlers should we worry about? Effects of context on observed fear and risk for anxiety. 2010 doi: 10.1037/a0023227. Manuscript submitted for publication. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Buss KA, Davidson RJ, Kalin NH, Goldsmith HH. Context-specific freezing and associated physiological reactivity as a dysregulated fear response. Developmental Psychology. 2004;40:583–594. doi: 10.1037/0012-1649.40.4.583. doi: 10.1037/0012-1649.40.4.583. [DOI] [PubMed] [Google Scholar]
  4. Buss KA, Goldsmith HH. Fear and anger regulation in infancy: Effects on the temporal dynamics of affective expression. Child Development. 1998;69:359–374. doi: 10.2307/1132171. [PubMed] [Google Scholar]
  5. Calkins SD, Fox NA. The relations among infant temperament, security of attachment, and behavioral inhibition at twenty-four months. Child Development. 1992;63:1456–1472. doi: 10.2307/1131568. [PubMed] [Google Scholar]
  6. Coplan RJ, Rubin KH, Fox NA, Calkins SD, Stewart SL. Being alone, playing alone, and acting alone: Distinguishing among reticence and passive and active solitude in young children. Child Development. 1994;65:129–137. doi: 10.2307/1131370. [PubMed] [Google Scholar]
  7. Crockenberg SC, Leerkes EM. Infant and maternal behaviors regulate infant reactivity to novelty at 6 months. Developmental Psychology. 2004;40:1123–1132. doi: 10.1037/0012-1649.40.6.1123. doi: 10.1037/0012-1649.40.6.1123. [DOI] [PubMed] [Google Scholar]
  8. Crockenberg SC, Leerkes EM. Infant and maternal behavior moderate reactivity to novelty to predict anxious behavior at 2.5 years. Development and Psychopathology. 2006;18:17–34. doi: 10.1017/S0954579406060020. doi: 10.1017/S0954579406060020. [DOI] [PubMed] [Google Scholar]
  9. Essex MJ, Boyce WT, Goldstein LH, Armstrong JM, Kraemer HC, Kupfer DJ The MacArthur Assessment Battery Working Group. The confluence of mental, physical, social and academic difficulties in middle childhood II: Developing the MacArthur Health and Behavior Questionnaire. Journal of the American Academy of Child and Adolescent Psychiatry. 2002;41:588–603. doi: 10.1097/00004583-200205000-00017. [DOI] [PubMed] [Google Scholar]
  10. Fox NA, Henderson HA, Rubin KH, Calkins SD, Schmidt LA. Continuity and discontinuity of behavioral inhibition and exuberance: Psychophysiological and behavioral influences across the first four years of life. Child Development. 2001;72:1–21. doi: 10.1111/1467-8624.00262. doi: 10.1111/1467-8624.00262. [DOI] [PubMed] [Google Scholar]
  11. Graham JW, Olchowski AE, Gilreath TD. How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prevention Science. 2007;8:206–213. doi: 10.1007/s11121-007-0070-9. doi: 10.1007/s11121-007-0070-9. [DOI] [PubMed] [Google Scholar]
  12. Hane AA, Cheah C, Rubin KH, Fox NA. The role of maternal behavior in the relation between shyness and social reticence in early childhood and social withdrawal in middle childhood. Social Development. 2008;17:795–811. doi: 10.1111/j.1467-9507.2008.00481.x. [Google Scholar]
  13. Hirshfeld-Becker DR, Micco J, Henin A, Bloomfield A, Biederman J, Rosenbaum. J. Behavioral inhibition. Depression and Anxiety. 2008;25:357–367. doi: 10.1002/da.20490. doi: 10.1002/da.20490. [DOI] [PubMed] [Google Scholar]
  14. Howell D. The treatment of missing data. In: Outhwaite W, Turner SP, editors. The Sage Handbook of Social Science Methodology. London: Sage Publications, Ltd.; 2007. pp. 208–224. [Google Scholar]
  15. Izard CE, Dougherty LM, Hembree EA. A system for identifying affect expressions by holistic judgments (Affex) Newark: University of Delaware, Computer Network Services and University Media Services; 1983. [Google Scholar]
  16. Jeličić H, Phelps E, Lerner RM. Use of missing data methods in longitudinal studies: The persistence of bad practices in developmental psychology. Developmental Psychology. 2009;45:1195–1199. doi: 10.1037/a0015665. doi: 10.1037/a0015665. [DOI] [PubMed] [Google Scholar]
  17. Johnson MH, Posner MI, Rothbart MK. Components of visual orienting in early infancy: Contingent learning, anticipatory looking, and disengaging. Journal of Cognitive Neuroscience. 1991;4:335–344. doi: 10.1162/jocn.1991.3.4.335. doi: 10.1162/jocn.1991.3.4.335. [DOI] [PubMed] [Google Scholar]
  18. Kagan J, Reznick JS, Clarke C, Snidman N, Garcia-Coll C. Behavioral inhibition to the unfamiliar. Child Development. 1984;55:2212–2225. doi: 10.2307/1129793. [Google Scholar]
  19. Kagan J, Reznick JS, Gibbons J. Inhibited and uninhibited types of children. Child Development. 1989;60:838–845. doi: 10.2307/1131025. [PubMed] [Google Scholar]
  20. Kagan J, Snidman N, Zentner M, Peterson E. Infant temperament and anxious symptoms in school age children. Development and Psychopathology. 1999;11:209–224. doi: 10.1017/s0954579499002023. doi: 10.1017/S0954579499002023. [DOI] [PubMed] [Google Scholar]
  21. Kopp CB. Antecedents of self-regulation: A developmental perspective. Developmental Psychology. 1982;18:199–214. [Google Scholar]
  22. Lantz CA, Nebenzahl E. Behavior and interpretation of the κ statistics: Resolution of the two paradoxes. Journal of Clinical Epidemiology. 1996;49:431–434. doi: 10.1016/0895-4356(95)00571-4. [DOI] [PubMed] [Google Scholar]
  23. Nachmias M, Gunnar M, Mangelsdorf S, Parritz RH, Buss K. Behavioral inhibition and stress reactivity: The moderating role of attachment security. Child Development. 1996;67:508–522. doi: 10.2307/1131829. [PubMed] [Google Scholar]
  24. Rubin KH, Burgess KB, Hastings PD. Stability and social-behavioral consequences of toddlers’ inhibited temperament and parenting behaviors. Child Development. 2002;73:483–495. doi: 10.1111/1467-8624.00419. doi: 10.1111/1467-8624.00419. [DOI] [PubMed] [Google Scholar]
  25. Thompson RA, Calkins SD. The double-edged sword: Emotion regulation in high risk children. Development and Psychopathology. 1996;8:163–182. [Google Scholar]
  26. Widaman KF. Missing data: What to do with or without them. Monographs of the Society for Research in Child Development. 2006;71(3):42–64. doi: 10.1111/j.1540-5834.2006.00404.x. [Google Scholar]

RESOURCES