Abstract
Identifying patterns of fearful behaviors early and accurately is essential to identify children who may be at increased risk for psychopathology. Previous work focused on the total amount of fear by using composites across time. However, considering the temporal dynamics of fear expression might offer novel insights into the identification of children at risk. One hundred and twenty-five toddlers participated in high- and low-fear tasks. Data were modeled using a novel two-step approach. First, a hidden Markov model estimated latent fear states and transitions across states over time. Results revealed children’s behavior was best represented by six behavioral states. Next, these states were analyzed using sequence clustering to identify groups of children with similar dynamic trajectories through the states. A four-cluster solution found groups of children varied in fear response and regulation process: “external regulators” (using the caregiver as a regulation tool), “low reactive” (low reaction to stimulus), “fearful explorers” (managing their own internal state with minimal assistance from the caregiver), and “high fear” (fearful/at-caregiver state regardless of task). The combination of analytic tools enabled fine-grained examination of the processes of fearful temperament. These insights may help prevention programs target behaviors that perpetuate anxious behavior in the moment.
1 |. INTRODUCTION
Anxiety-related disorders are among the most prevalent disorders for children and adolescents (Merikangas et al., 2010), affecting 30% of adolescents in the United States (Kessler, Petukhova, Sampson, Zaslavsky, & Wittchen, 2012). Buss (2011) identified a distinct pattern of fearful behavior, coined dysregulated fear (DF), which increases the risk for social withdrawal and anxiety symptoms compared with the traditional model of behavioral inhibition. Even though there is a growing body of research examining DF and its relation to poor outcomes, as a field, we still do not understand how the behavior of DF children puts them at increased risk for poor adjustment. Research has suggested that the mechanism behind DF behaviors could be extreme reactivity (Buss, Davidson, Kalin, & Goldsmith, 2004) or failure to regulate (Buss, 2011; Buss, Davis, Ram, & Coccia, 2018); however, more fine-grained analyses are necessary to understand the dynamics of DF and what is driving these behaviors in children. Research examining the effects has largely used composites without examining the dynamic nature of emotion expression and regulation and of particular importance for this paper, the expression and regulation of fear. Aggregate composites, though useful and readily available, do not capture the temporal dynamics inherent in emotion expression; Buss and colleagues’ previous work in this area has highlighted the importance of context when examining fearful behavior. This study will take that nuanced approach a step further by examining the dynamic expression of fearful behaviors on a moment-to-moment basis to both explore the groups of behaviors visible in DF and to identify other possible groups of fear behavior that may be of interest to researchers.
1.1 |. Dysregulated fear and children’s outcomes
Anxiety symptoms are tied to a host of negative outcomes for young children and adolescents including their overall health and social and emotional well-being. In particular, children with anxiety have fewer close friendships, are more likely to be bullied by peers, and have less academic success compared with their less anxious peers (Rubin, Coplan, & Bowker, 2009). Not only do these children have social and academic difficulties, but also children with anxiety have poor health outcomes (Kidwell, Nelson, & Van Dyk, 2015). More specifically, anxious children experience more chronic illness, asthma, and allergies (Chavira, Garland, Daley, & Hough, 2008). Therefore, efforts to identify contributors to anxiety trajectories are of particular importance. One potentially important predictor of anxiety highlighted in the literature is extreme fearful temperament, which is associated with later anxiety throughout development and demonstrated to be a crucial marker for anxiety (Degnan, Almas, & Fox, 2010; Kagan & Fox, 2007; Pérez-Edgar & Fox, 2005).
Fearful temperament has commonly been studied as behavioral inhibition. Behavioral inhibition is characterized by children withdrawing from, avoiding, or reacting fearfully to novel situations (Garcia-Coll, Kagan, & Reznick, 1984; Kagan & Fox, 2007; Kagan, Reznick, Clarke, Snidman, & Garcia-Coll, 1984). However, not all children who exhibit behavioral inhibition develop anxiety symptoms. Indeed, several studies have shown variability in the risk of anxiety problems for children with fearful temperaments. Buss and colleagues argue that this is because children displaying fearful temperament comprise a diverse group (Buss, 2011; Kiel & Buss, 2013; Morales, Pérez-Edgar, & Buss, 2015), and more specifically that there are multiple types of fearful temperament (Shackman et al., 2013). Previous examinations of fearful temperament average behaviors across task not accounting for the context in which fear was expressed. Buss and colleagues found that the heterogeneity of fearful temperament was more visible when context was taken into account. In particular, Buss (2011) identified children who displayed a pattern of DF behavior characterized by high fear in low-threat situations. Though these low-threat situations are novel, they are situations that encourage engagement and pleasure from majority of toddlers (e.g., a novel but engaging friendly puppet show). This type of behavioral profile is the hallmark characteristic of DF, where the expression of fear does not match the context and is different from behavioral inhibition. Furthermore, previous research has shown that DF is stable over time suggesting a temperamental type (Buss et al., 2013).
Dysregulated fear status is an important predictor of children’s social difficulties in preschool and kindergarten (Buss, 2011). Previous research has shown that compared with children characterized by other types of fearful temperament, children with high DF are at higher risk. More specifically, children with DF were more likely to demonstrate symptoms of social anxiety compared with children showing behavioral inhibition but not DF (Buss et al., 2013). Previous research has individually highlighted reactivity and regulation as key dimensions of DF. Specifically, children identified as DF have also demonstrated higher basal and reactive cortisol and higher sympathetic cardiac activity during baseline (Buss et al., 2004), and a pattern of heart rate variability that suggests difficulty regulating fear (Buss et al., 2018). With DF emerging as an important predictor for later social anxiety over and above other inhibited temperament profiles, research has begun to examine the mechanisms under-lining this behavioral profile. For instance, research has shown that children with DF show distinct attention patterns as well, where they are more likely to show bias away from threat compared with those who did not exemplify this behavioral pattern. Furthermore, if children with DF showed a bias toward threat they were more likely to be associated with social withdrawal compared with others without this behavioral profile (Morales et al., 2015). Although these studies provide clues as to the underlying mechanisms of DF, more research is needed that examines children’s reactivity and regulation simultaneously to elucidate how this process may be multifaceted. It could be that by examining the temporal relations between these behaviors, we can better inform fear behavior profiles. By using these fine-grained analyses, these methods can provide additional insights into how fearful behaviors unfold from moment to moment, what pattern constitutes dysregulation and/or extreme reactivity, and aid in identification of the pattern associated with risk trajectories.
1.2 |. Static versus dynamic approaches
Previous research has largely used regression-based techniques that summarize several indices of fear expressions across a task into one aggregate score of fear (e.g., Buss, 2011; Buss et al., 2013; Fox, Snidman, Haas, Degnan, & Kagan, 2015; Morales et al., 2015). For example, Buss (2011) created an average composite that represented the proportion of time fear behavior was observed (i.e., latency to freeze, duration of facial fear, bodily fear, freezing, and close proximity to caregiver). Buss then used this composite of fear behavior observations to create a slope score for each individual for each task as an indicator of their fear regulation across tasks. Another example in recent work, Morales et al. (2015) used a principal components analysis to combine duration of facial fear, bodily fear, and freezing, time spent in proximity to caregiver, and latency to freeze for each of the six tasks. The authors then used these timing composites and instances of observed behavior in a latent profile analysis across all tasks to identify profiles of fear to test associations with attention bias toward or away from threat. However, these approaches do not capture how the sequence of behaviors develop during the interaction and how different fearful behavior trajectories might be differentially associated with a given outcome. More importantly, understanding how fearful behaviors in the moment co-occur for specific groups of children and their trajectories across these tasks (i.e., if they stay in high-fear states or transition over the course of the task) could further our understanding of how their moment-to-moment responses influence later anxiety outcomes. The current study explored children’s fearful behaviors on a moment-to-moment basis to investigate the dynamic nature of individual fear expressions and how behaviors interact and change over time.
By taking this more fine-grained approach, we are able to account for temporal dependencies (e.g., timing and order) among different types of fear expressions and study how children’s states of expression change or remain the same throughout high- and low-fear contexts. This information would otherwise be lost using a single score. For this analysis, we developed a novel two-stage approach that explicitly examines the trajectories and sequences of fearful behaviors and identifies meaningful groups based on those dynamic trajectories. Our method first uses a hidden Markov model (HMM) to examine latent fear states at each time point (Baum & Petrie, 1966; Stifter & Rovine, 2015), and combines it with a sequence clustering approach to identify and characterize groups of children that share common trajectories in fear states across longer timescales. A similar method was pioneered by Brinberg and colleges (Brinberg, Ram, Hülür, Brick, & Gerstorf, 2018) using state space grids, but is limited to bivariate categorical data. Our approach combining HMM and sequence clustering therefore allows us to build a typology of fearful expression and the micro-temporal patterns in their display, and map out the way those micro-patterns combine to form more complete processes that may indicate risk or protective factors.
The HMM approach (Baum & Petrie, 1966) allows for the temporal dynamic of fearful expressions and behaviors to be captured by using latent states. Specifically, an HMM estimates a set of latent momentary states based on current behaviors and the probabilities of changing into or out of those behaviors (Stifter & Rovine, 2015). For example, if children frequently show both facial fear and bodily fear together for only a few seconds before transitioning to a longer display of facial fear only, an HMM might distinguish two states: one characterized by both displays of fear and a high probability of change, and one with facial fear that is more likely to stay stable. In this way, the HMM transforms a complex multivariate sequence into a single categorical sequence in which a child moves sequentially from momentary state to momentary state.
The sequence of states from the HMM provides a comprehensible view of the fear state of a child at each moment. For instance, Figure 1 shows the complex multivariate trajectories of four children’s fear behaviors below the much more comprehensible HMM-derived state sequence for each. Yet, the goal of our analysis was to find groups that characterized the way that children transition through these states (e.g., from high fear to low) across time. Sequence clustering provides a convenient way to characterize this more global information. Sequence clustering is a technique from the data mining literature that distinguishes groups in the data based on the similarity of their trajectories across time (Caiado, Maharaj, & D’Urso, 2016). Importantly, this method examines the entire sequence to identify clusters of sequences that describe the way that children’s moment-to-moment behavior changes across time. For example, some children may stay comfortably at their caregiver for some time, then venture slowly toward a fearful object showing high levels of fear, while others may use proximity to the caregiver as a regulatory mechanism, running back and forth between high fear near the fearful object and lower fear in their caregiver’s lap. These profiles are difficult to determine with summary statistics but are easily captured by sequencing methods.
FIGURE 1.

Individual state behavior plot from the high-fear task (spider). Note. This plot depicts example plots for individual children for each cluster during the high-fear task (spider)
Children’s temperament is an important factor in shaping how they interact with their families and their larger environment. The insights gained by using different analytic tools to get at the fine-grained processes of what accounts for individual differences in fearful temperament profiles could then be used to tailor specific prevention programs to target behaviors that in the moment will perpetuate anxious behavior.
1.3 |. The current study
Data were modeled using HMMs, which find latent states of behavior that approximate underlying fear states and transforms the multivariate measurements of fear behaviors into a single sequence of transitions from one state to another over time. These state transition sequences were then grouped using optimal-match sequence clustering to identify similar dynamic trajectories through those states to categorize patterns in the way that children behave across time. The aim of the current study was to build on previous work and to examine the expression and intensity of fearful behaviors on a moment-to-moment basis to both explore the groups of behaviors visible in DF and to identify other possible groups of fear behavior. We hypothesized that: (a) different groups of children will be identified that have different patterns of reactivity and use different strategies to regulate emotion; and (b) the DF group will be identifiable, which shows poor regulation of fear even in low-fear conditions.
2 |. METHOD
2.1 |. Participants
The present study includes 125 children (62 girls/63 boys, Mage = 24.43 months, SDage = 0.47) who participated in a larger prospective longitudinal investigation of temperament and socio-emotional development oversampling for fearful/anxious toddlers. Children and their primary caregiver participated in observed tasks, and caregivers also completed questionnaires. Children were predominantly Caucasian (91.2%, 8% more than one race, and 0.8% African American) with most children residing in married two-parent households (96.8%). Family income ranged from <15,000 (2.4%) to >$60,000 (49.6%) with most families (84%) earning over $30,000. Primary caregiver education ranged from 12 to 20 years (M = 16.22, SD = 2.32). One child did not complete either the low- or high-fear observation task and was removed. This study was conducted in accordance with the ethical standards outlined in the Declaration of Helsinki of the American Psychological Association, the National Institutes of Health, and approved by the Pennsylvania State University Review Board. Written informed consent was obtained from all children’s primary caregiver before any assessment or data collection.
2.2 |. Procedure
Children and the primary caregiver (98% mothers) visited the laboratory for a 2-hr visit. The caregiver stayed with their child during the entire visit. Following consent, warm-up, and an overview of the visit with the parent, caregiver and child participated in several episodes designed to measure dysregulated fear (see Buss, 2011 for a review for full description of protocol). For the purpose of this study, we focused on one low-fear task (puppet) and one high-fear task (spider).
2.2.1 |. Low-fear task
The puppet task was novel and engaging but not fear-eliciting by design. During the episode, two friendly puppets (an elephant and a lion) played together (with balls and a fishing pole) while continually inviting the toddler to come play with them. Toddlers began each episode in their caregivers’ laps. The task lasted for about 3 min. However, if the child experienced lasting distress and could not be calmed, the stimulus was removed. At the end of the task, the puppets gave the toddler a sticker for playing or watching the show. The puppets then said goodbye, and the unfamiliar puppeteer came out from behind the stage and asked whether the toddler wanted to play with the puppets.
2.2.2 |. High-fear task
The high-fear task included a novel scenario but involved an object designed to elicit fear. Similar to the low-fear task, the high-fear task began with the toddler sitting in their caregiver’s lap, and then, the child was free to move once the episode started. The pair sat opposite a large fuzzy toy spider mounted on a remote toy car. After a brief time of not moving, the spider started to move toward the chair and then stopped at the midway point before retreating. On the second movement of the spider, the spider moved the entire distance to the chair where the parent and toddler were sitting before retreating. The episode lasted approximately 1 min before the experimenter entered and asked whether the child wanted to play with or touch the spider.
2.3 |. Measures
2.3.1 |. Behavioral coding
A second-by-second coding scheme was used to score children’s fear behaviors in the two tasks. These behaviors included facial fear (0 [no fear]–3 [very fearful]), bodily fear (0 [no fear]–3 [very fearful]), and proximity to the caregiver (0 [far from caregiver (beyond 2ft)]–2 [touching caregiver]). The AFFEX (Izard, Dougherty, & Hembree, 1983) coding scheme was used to code facial fear. Facial fear was then recoded into a binary (fearful/not fearful) variable due to the low variability in the intensity of facial fear. Facial fear was characterized as straight brows or brows that were slightly raised and drawn together, eyelids raised/tense, and/or an open mouth with corners pulled back. Bodily expressions of fear included behaviors such as diminished play or activity, freezing (limbs unmoving/rigid and arranged awkwardly for longer than two seconds), and trembling. Proximity to the caregiver was coded on a 3-point scale based on whether the child was at the caregiver (touching or immediately next to), near the caregiver (within 2ft), or far from caregiver (beyond 2ft). Proximity to caregiver has been used in previous examinations as a measure of external regulation (e.g., Buss, Brooker, & Leuty, 2008; Kopp, 1989). Behavioral coders were required to reach a minimum of 90% interrater agreement and have a kappa of at least 0.70 for each individual behavior. Interrater reliability was calculated by using the percentage agreement on at least 15% of cases for each episode (facial fear: 94%, K = 0.81; bodily fear: 96%, K = 0.87; and proximity to caregiver: 98%, K = 0.97).
2.3.2 |. Questionnaire data
To assess the validity of the clusters, we used relevant questionnaire data completed by the parent (see Table 1 for descriptives). These scales have been used in previous papers examining fearful temperament (e.g., Beekman et al., 2015; Buss et al., 2018; Carranza, González-Salinas, & Ato, 2013; Cho & Buss, 2017; Morales et al., 2015). We used subscales from the Toddler Behavior Assessment Questionnaire (TBAQ; Goldsmith, 1996). The scale ranged from 1 (never) to 7 (always). The subscales included the following: inhibitory control (e.g., “when asked to wait for something (like a toy or a snack; α = 0.82), how often did your child wait patiently?”; α = 0.81), object fear (e.g., “when a dog or other large animal approached your child, how often did s/he cling to you fearfully?”; α = 0.74), social fear (e.g., “if a stranger came to your house or your apartment, how often did your child ‘warm up’ to the stranger within 10 min?”; α = 0.82), and soothability (e.g., “when you were comforting your upset child, how often did s/he calm down quickly?”; α = 0.75).
TABLE 1.
Questionnaire subscale descriptives
| Subscale | Mean | SD | Min | Max |
|---|---|---|---|---|
| TBAQ | ||||
| Inhibitory control | 3.85 | 0.70 | 2.00 | 5.38 |
| Object fear | 2.48 | 0.82 | 1 | 5.50 |
| Social fear | 3.70 | 1.05 | 1.43 | 6.38 |
| Soothability | 5.29 | 0.66 | 3.44 | 6.89 |
| ITSEA | ||||
| Anxious worry | 0.22 | 0.25 | 0 | 1.43 |
| Separation distress | 0.92 | 0.40 | 0 | 1.83 |
| Inhibition to novelty | 1.02 | 0.55 | 0 | 2.00 |
Abbreviations: ITSEA, Infant and Toddler Social and Emotional Assessment; TBAQ, Toddler Behavior Assessment Questionnaire.
Next, we used the Infant and Toddler Social and Emotional Assessment (ITSEA; Carter, Briggs-Gowan, Jones, & Little, 2003) to assess indices of behavioral adjustment. The measure consisted of 169 items that were aggregated into 20 subscales that the parent reported on. The scale ranged from 0 (not true) to 2 (very true). We used the following subscales: anxious worry (e.g., “worries a lot or is very serious”; α = 0.54), separation distress (e.g., “cries or hangs onto you when you try to leave”; α = 0.76), and inhibition to novelty (e.g., “takes a while to feel comfortable in new places”; α = 0.85) in the current study.
2.4 |. Analytic plan
A HMM was fitted to the entire data set to approximate the set of underlying fear states experienced by the children. The model then reprocessed each child’s data to estimate the fear state of each child at each time point and provide a time series of fear states for each child. Put in another way, the HMM determined the level and type of fear expression in each second of a child’s experience, rather than seeing what is happening around it. Once the final state sequence for each child was confirmed, we used sequence clustering to identify groups of children with similar dynamic trajectories across their entire time series. We then used regression in exploratory analyses to validate the clusters. All analyses were conducted with the R software package (R Core Team, 2018). HMM was performed using depmixS4 (Visser & Speekenbrink, 2010), while sequence clustering, cluster evaluation, and visualization were performed with the TraMineR (Gabadinho, Ritschard, Müller, & Studer, 2011) and fpc (Hennig, 2018) R packages. Exploratory regression analyses used the lm() function within the stats package (R Core Team, 2018), which automatically dummy-codes categorical variables to test associations with the outcome variables. The clusters were all included in the same regression model for each respective temperament subscale tested.
3 |. RESULTS
3.1 |. Hidden Markov models
A HMM was fitted to the data to approximate the underlying fear state of each child at each time point. Latent fear states are determined based on the coded characteristics of a child’s fear expression and distance from caregiver at each time point, and the probabilities of transitioning from that state to each other state in the next time point. For example, Table 3 shows the probabilities of each fear expression in each state in our final model; Table 4 shows the transition probabilities. Both are interpreted in more detail below.
TABLE 3.
State characteristics and probabilities
| States | Facial fear | Bodily fear | Proximity to caregiver |
|---|---|---|---|
| Low-fear | None (0.64) | Low (0.89) | Far (1.0) |
| No-fear/at-caregiver | None (0.71) | None (0.99) | Touching (1.0) |
| No-fear/near-caregiver | None (0.67) | None (0.62) | Near (0.99) |
| No-fear/far-from-caregiver | None (1.0) | None (1.0) | Far (1.0) |
| Fearful/at-caregiver | Present (0.57) | Low (0.69) | Touching (1.0) |
| Fearful/far-from-caregiver | Present (1.0) | None (1.0) | Far (1.0) |
Note: Probability results from the HMM analyses are presented in parentheses.
TABLE 4.
State transitions
| 1 | 2 | 3 | 4 | 5 | 6 | |
|---|---|---|---|---|---|---|
| Puppet task (low fear) | ||||||
| 1. No-fear/far-from-caregiver | 0 | −5.05 | −7.45 | −4.14 | −4.36 | −4.25 |
| 2. No-fear/at-caregiver | 0 | 4.86 | 1.89 | −2.73 | −5.51 | 1.81 |
| 3. Fearful/at-caregiver | 0 | 3.78 | 7.42 | −7.17 | 0.58 | 3.23 |
| 4. Fearful/far-from-caregiver | 0 | −2.98 | −4.48 | 1.51 | −1.35 | −1.50 |
| 5. Low-fear | 0 | −5.24 | −3.39 | −1.43 | 2.74 | −1.52 |
| 6. No-fear-/near-caregiver | 0 | −0.84 | −1.97 | −1.79 | −1.22 | 2.74 |
| Spider task (high fear) | ||||||
| 1. No-fear/far-from-caregiver | 0 | 1.66 | 1.02 | 0.26 | 0.55 | 0.32 |
| 2. No-fear/at-caregiver | 0 | −0.14 | 0.57 | 1.09 | 2.61 | −1.22 |
| 3. Fearful/At-Caregiver | 0 | 0.33 | −0.55 | −4.72 | −2.45 | −2.70 |
| 4. Fearful/far-from-caregiver | 0 | 1.23 | −5.89 | 0.16 | 0.27 | −1.10 |
| 5. Low-fear | 0 | 3.76 | 0.38 | −0.16 | −0.02 | −1.46 |
| 6. No-fear/near-caregiver | 0 | 0.41 | 0.20 | 0.12 | −1.67 | −0.74 |
Note: No-fear/far-from-caregiver is the comparison state. The values presented in the table are log odds ratios. Negative log odds indicate a lower likelihood to transition, whereas positive log odds indicate a higher likelihood to transition.
To facilitate a better understanding of the fearful behaviors across tasks, Figure 1 shows the trajectories of four representative children in detail during the high-fear task (spider). The bottom of each plot shows the level of behavior for one child across time (e.g., bodily fear, facial fear, proximity to caregiver), with the level of behavior on the y-axis and seconds on the x-axis. The top of each plot shows the state changes in a color sequence across time. For example, in the top left plot, one can see the first 50 s is mostly characterized by close proximity to caregiver, the presence of facial fearful, and low bodily fear, which corresponds to the fearful/at-caregiver state. Then, the child exhibits no bodily fear, largely no face fear, and stayed close to the caregiver, which corresponds to the no-fear/at-caregiver state. Overall, from the figure, it can be seen that the external regulator cluster child showed bodily fear and face fear but used the caregiver as an successful external regulatory behavior over the course of the tasks when fear became prevalent. The low-reactive-cluster child showed no fear across the majority of the tasks. The fearful explorers cluster child reflected the heterogeneity of the sequences of the cluster. Specifically, the child initially showed a close proximity to the caregiver and then displayed no fear as they moved further away before peaking in bodily fear and remaining away from the caregiver as the child continued to explore. Last, the child in the high-fear cluster showed close proximity to caregiver and fear throughout the tasks.
3.1.1 |. Determining number of states
Like any mixture model, an HMM requires identification of the number of latent states before it can be fitted. To determine the number of states, we explored the fit indices of HMMs created with state counts ranging from two through eleven. Several fit statistics, including log likelihood, AIC, and BIC, are reported in Table 2. While previous work suggested that three to four states should be necessary, we examined models including up to eleven states to ensure that we found a best-fitting model. We created HMMs for both the low- and high-fear task, as well as across the two tasks by including the task type as a covariate in the HMM. As shown in Table 2, a six-state model consistently provided the best fit with distinct, interpretable states.
TABLE 2.
State fit statistics
| States | Log likelihood | AIC | BIC |
|---|---|---|---|
| 2 | −76596.78 (df = 17) | 153,227.6 | 153,372.3 |
| 3 | −65612.05 (df = 32) | 131,288.1 | 131,560.6 |
| 4 | −56681.73 (df = 51) | 113,465.5 | 113,899.7 |
| 5 | −48628.84 (df = 74) | 97,405.69 | 98,035.82 |
| 6 | −46503.8 (df = 101) | 93,209.6 | 94,069.65 |
| 7 | −41633.03 (df = 132) | 83,530.05 | 84,654.07 |
| 8 | −44042.99 (df = 167) | 88,419.98 | 89,842.04 |
| 9 | −41857.6 (df = 206) | 84,127.2 | 85,881.35 |
| 10 | −32537.85 (df = 249) | 65,573.7 | 67,694.01 |
| 11 | −35563.73 (df = 296) | 71,719.47 | 74,240 |
Note: AIC and BIC are comparative fit indices. Lower scores indicate better fitting models.
3.1.2 |. Interpreting the six state models
The characterizations and probabilities (with estimates closer to one being indicative of a higher probability of that behavior occurring in that state and closer to zero being a lower probability of that behavior occurring in that state) of each state are depicted in Table 3. These states can be distinguished based on the fear expressions of the child and the proximity of the child to the caregiver, implying that proximity may be acting as a tool to regulate fear. The first state, labeled “low-fear,” is characterized by no facial fear and low bodily fear when explicitly far from the caregiver, and indicates a state of low fear without intervention from the caregiver. Three “no-fear” states were also found, corresponding to a near lack of fearful expression when touching, near, or far from the caregiver, respectively. Fearful/at-caregiver is the first fearful state and is characterized by facial fear and low bodily fear when near or touching the caregiver, whereas the fearful/far-from-caregiver state is characterized by predominantly facial expressions of fear and no bodily fear when far from caregiver.
3.1.3 |. Predicting the transition probabilities
The initial probabilities indicated that children were most likely to start in the no-fear/at-caregiver state, which was expected given the structure of the tasks. We examined the transition probabilities (see Table 4) to investigate how children were moving between states based on whether they were in the low-fear (puppet) and high-fear (spider) tasks. During the low-fear task, children who began in no- or low-fear states were more likely to stay in those states than transition to fearful states. Similarly, children who began in fearful states were more likely to stay in fearful states during the low-fear task than transition into a low-fear or no-fear state. In summary, during the low-fear condition, children were more likely to begin in a low-fear state, showed fear soon after the introduction of the stimulus, and then quickly either reduced fear or left the caregiver’s side while still expressing fear. In comparison, during the high-fear task, children in the no-fear/at-caregiver state were more likely to transition to the low-fear state and children in the low-fear state were more likely to transition to the no-fear/at-caregiver state. Children who were in the fearful/at-caregiver state were less likely to transition to states that were far from their caregiver, and those children who were in the fearful/far-from-caregiver state were more likely to move to the no-fear/at-caregiver state. Overall, in the high-fear task, children were less likely to transition to far-from-caregiver states, and more likely to stay in fearful states for longer.
Figure 2 shows a state transition plot that depicts the probabilities of transitioning in and out of the HMM states based on the low- or high-fear task. Each state subplot shows the probabilities that a child in that state transitions from that state to another in the low-fear task (left side of each subplot) and high-fear task (right side) conditions. For example, a child in the no-fear/at-caregiver state (top middle) was more likely to transition out of the state into a fearful state in the high-fear condition, as shown by the line at the bottom of the plot rising toward the right. Children in the high-fear task also showed overall higher probability of changing state, with more transitions to high-fear and low-fear states, suggesting higher volatility in fear in this condition. Children in the high-fear condition also showed higher probabilities of transitioning directly back and forth between no fear/at caregiver and fearful/far from caregiver, suggesting that the caregiver may be acting more strongly as a regulating force. These characterizations are in line with expected behavior and provide a sensible check of the face validity of the HMM states.
FIGURE 2.

State transition plot. Note. This plot depicts children’s transition probabilities between hidden Markov model (HMM) states
Hidden Markov models results provided valuable information about the individual-level transition. However, we were also interested in grouping children based on their momentary trajectories. In order to distinguish between groups of trajectories over the course of the low-fear and high-fear tasks, we used a sequence cluster technique that upholds the temporal structure of the HMM. The combination of these two methods facilitated a multi-time-scale examination using momentary and sequential assessments, for example, investigation on a short timescale (e.g., child right now) versus a longer time view (e.g., using the cluster to get a better representation of the child throughout the whole task).
3.2 |. Sequence clustering of HMM state trajectories
3.2.1 |. Clustering approach
Optimal-match sequence clustering uses an optimization algorithm to calculate the distance between pairs (e.g., Brinberg et al., 2018; Needleman & Wunsch, 1970) of state sequences as a whole without breaking the sequence into smaller subsections, and identify clusters of sequences that have low distances among them. In this way, sequence clustering is a technique that allows the identification of typologies of sequences. Specifically, the optimal-match distance between two sequences is defined as the minimum change needed to transform one state sequence into another state sequence through insertion, deletion, and substitution. For example, a sequence ABAAC could be converted into AAAD by removing the B and substituting a D for the C. This is easier than converting it to DBCDDA, which requires three conversions and one deletion, and thus, ABAAC would be considered more similar (and therefore closer) to AAAD than to DBCDDA.
A k-medoids cluster analysis (using PAM; Hennig, 2018) was applied to the optimal-match distances. For some number k, this analysis searches for groups of state sequences such that sequences within a single group are similar to one another (minimal distance) and sequences in different groups are less similar (more distance). An optimal cluster solution therefore consists of clusters where elements of each cluster are very similar, but where clusters themselves are very different from each other.
3.2.2 |. Cluster fit indices
Based on the above HMM solution, we clustered the children’s dynamic state trajectories into groups based on the optimal-match similarity between their sequences. We tested solutions using between two and six clusters and found three- and four-cluster solutions to best fit the data (see Table 5). To access the fit of the cluster solution, we used several cluster validation indices. When assessing whether or not a cluster is valid, the important thing is to identify that the cases within a cluster are similar and those in different clusters are dissimilar (Halkidi, Batistakis, & Vazirgiannis, 2002; Halkidi, Vazirgiannis, & Hennig, 2016; Hennig, 2016; Kassambara, 2017; Meila, 2016). That is, a good clustering solution should find clusters such that trajectories within each cluster are similar, and where each cluster is well separated from other clusters (Halkidi et al., 2016; Kassambara, 2017). Although there is no specific null hypothesis test for a clustering solution, metrics of cluster goodness of fit may be thought of as similar in concept to an F-score in an ANOVA—they capture the relative ratio of within-cluster variation as compared to variation between clusters.
TABLE 5.
Cluster fit statistics
| Absolute validation indices | |||
|---|---|---|---|
| Number of clusters | |||
| Fit statistic | 3 | 4 | 5 |
| Dunn | 0.95 | 0.93 | 0.84 |
| Avg.silwidth | 0.21 | 0.17 | 0.13 |
| CH index | 114.79 | 93.22 | 76.48 |
| Entropy | 1.10 | 1.35 | 1.59 |
| Comparative validation indices | |||
| Cluster comparison | |||
| Fit statistic | 3 vs. 4 | 3 vs. 5 | 4 vs. 5 |
| VI | 0.37 | 0.99 | 0.68 |
| Corrected rand | 0.81 | 0.49 | 0.64 |
Note: For the absolute validation indices, higher scores on Dunn, Avg.silwidth, and CH index indicate better fit, whereas lower scores for entropy indicate a better fitting model. For the comparative validation indices, lower scores on the VI and higher scores on the Corrected Rand indicate better fit.
We examined several measures of cluster fit. First, we used a modified Dunn index. The Dunn indices are a measure of dissimilarity between clusters indicating whether clusters are far apart relative to the size of the cluster. Higher scores on the Dunn statistic is indicative of good distance between and separation of clusters, whereas low scores are indicative of poor cluster separation (Halkidi et al., 2002). Based on the above criteria, we have acceptable fit according to the Dunn index. A second measure of fit is the silhouette coefficient, which measures both how homogeneous a cluster is internally relative to the distance between clusters. Again, a higher value (closer to 1.0) is a good clustering solution, while those close to 0 suggest that the case lies between two clusters and closer to −1 indicates a misclassification (Kassambara, 2017). Though the silhouette coefficient for each cluster solution is relatively low, they still provide a valid method of comparing clustering solutions.
We also examined several other internal indices, such as the Calinkski–Harabasz index, which accounts for both within-cluster distance divided by between-cluster distance (similar to an F-test). For this index, higher values means the clusters are distinct and also compact, while small values means they are not (Halkidi et al., 2016). Our results pointed to separated and distinct clusters and compacted and homogeneous clusters for all three cluster solutions with three and four clusters fitting better than a five-cluster solution. We also quantified entropy, which in this context measures the reduction in disorder or gain in information provided by the cluster solution, while accounting for the complexity of the solution itself. Entropy here can be considered a trade-off between the parsimony of the model and the amount of information it explains—the more overlap between clusters and the more intricate the clustering solution, the larger the value of the index. A small entropy value therefore indicates good fit (Li, Ma, & Ogihara, 2004). Entropy for our cluster solutions was acceptable and highlighted the benefits of a three- or four-cluster solution.
Across all the absolute validation indices, the three- and four-cluster solutions fit better than the two-, five-, or six-cluster solutions. We also compared our clustering solutions to each other using comparative validation indices (Meila, 2016), which allowed direct comparisons of two-cluster solutions. The first, known as the corrected Rand index, ranges from −1 (no agreement between the clustering) to 1 (complete agreement between the clustering); the second, Meila’s variation index (VI), measures the unique contribution and loss or gain of each added cluster (Meila, 2007). Results comparing the three-, four-, and five-cluster solutions are also reported in Table 5. Results showed that the three- and four-cluster and four- and five-cluster solutions were considered to be in high agreement. In order to capture the important information in all three solutions, and to build on the previous conceptual literature that identifies a three-group solution, we selected the four-cluster solution, which we felt also more clearly delineated qualitative differences in the data.
3.2.3 |. Cluster characteristics
The individual state trajectories of children as divided into clusters are shown in Figure 3. Similar to Figure 1, each row is colored to represent a state at a given moment to provide an overview of each child’s individual trajectory (i.e., children’s trajectories for the high- and low-fear task are included as individual trajectories). Each of these clusters represents a specific pattern of fearful behavior across time; each line represents a single child’s trajectory across time. The figure therefore provides an excellent “at-a-glance” view of the common characteristics among the trajectories within each cluster.
FIGURE 3.

Individual sequence cluster plots across tasks. Note. Each row is an individual child’s sequence of hidden Markov model (HMM) states. Participants are counted twice (once for the low-fear task and once for the high-fear task) in these clusters for depiction purposes only
The first cluster showed a group of children we named “external regulators.” Children in this cluster may be using the caregiver as an external fear regulation tool. Specifically, sequences of behavior in this cluster were characterized by the no-fear/at-caregiver and fearful/at-caregiver states, with members beginning in the no-fear/at-caregiver state and then transitioning to the fearful/at-caregiver state before returning to the no-fear/at-caregiver state. In the lower part of the figure where the cluster showed a bit more heterogeneity, it becomes clear that as children moved farther from their caregiver, they began to display more fear, returning closer to caregiver and allowing their fear to subside before once again venturing out into the room and toward the stimulus. The top left panel of Figure 1 shows a single child from this cluster in more detail, which demonstrates this pattern. This child in the external regulators cluster was showing mostly no-fear/at-caregiver states with fearful/at-caregiver, the fearful/far-from-caregiver, and low-fear states interspersed. This highlights that children in the “external regulators” cluster repeatedly returned to the caregiver, which in turn reduced their fear which allowed them to venture back out.
The second cluster, coined “low reactive,” was characterized by a group of children who showed minimal reactivity in response to the stimulus. Sequences in the low-reactive group were dominated by the no-fear/far-from-caregiver state. Children moved quickly and progressively away from their caregiver, with minimal shows of fear and few returns. There was some heterogeneity where fear states made a brief appearance in their trajectories, but overall, members of this cluster were defined by their lack of fear and engagement in fear-free exploratory behavior. Indeed, in Figure 1, top right panel, the individual graphic shows a child mainly staying in the no-fear/far-from-caregiver state throughout the task.
The third cluster was named “fearful explorers” and was the most heterogeneous group in terms of the number of states and the transitions that occur in individual trajectories. The no-fear/far-from-caregiver state characterized the ends of many of the cluster members’ trajectories. However, throughout their individual trajectories, they showed two common patterns. First, many returned to their caregiver at one or more points in the sequence. While in some cases, these children moved close and allowed their fear to subside (into the no-fear/near-caregiver state), there are a large number of cases in which members of this cluster transition directly from the fearful/far-from-caregiver state to the low-fear/far-from-caregiver or no-fear/far-from-caregiver state without returning to the caregiver in between. In Figure 1 bottom left panel, one can see an individual child member of this cluster moving through the task. Although the child clearly displayed fear, and does occasionally return to the caregiver for support, much of their regulation of fear state happened entirely away from the caregiver. That is, the child seemed to be managing their own internal state with only minor assistance from the caregiver, hinting at the development of a mechanism for internal regulation of fear.
The fourth cluster was the high-fear group, indicating a dysregulated fear profile of behavior. This group across tasks was dominated by the fearful/at-caregiver state. The group of children showed some heterogeneity toward the bottom of the cluster depicted in Figure 3. Where some of the children moved away from the caregiver with no fear, then they quickly transitioned back into a fear state and moved quickly back to the caregiver. In Figure 1 (bottom right), one can see that the dominant state was the fearful/at-caregiver state. Table 6 shows how children transition between clusters between tasks. In this transition matrix, one can see that many children were in the high-fear cluster during the high-fear task but showed more heterogeneity in the lower fear conditions.
TABLE 6.
Matrices for transitions between clusters
| Spider | |||||
|---|---|---|---|---|---|
| Puppet | External regulators | Fearful explorers | High fear | Low reactive | Total |
| External regulators | 4 | 0 | 10 | 0 | 14 |
| Fearful explorers | 20 | 3 | 20 | 0 | 43 |
| High fear | 8 | 0 | 15 | 0 | 23 |
| Low reactive | 25 | 5 | 10 | 0 | 40 |
| Total | 57 | 8 | 55 | 0 | |
Note: Each number reflects the number of children in that cluster during the spider (high-fear) and puppet (low-fear) task.
3.3 |. Using the clusters to predict to children’s outcome
We used the typologies from the clusters derived above to test associations with children’s temperament at the same age (age 2) as reported by their caregiver. These were exploratory analyses to test the validation of the clusters and provide directions for future research. We used estimated cluster membership to predict parent-reported responses on the TBAQ using regression. Across all analyses, 1–2 participants on any of the below subscales were removed due to missingness. In these analyses, the fearful explorers cluster in the high-fear task was associated with social fear (β = −1.0, SE = 0.37, t = −2.69, p < .01), such that those in the fearful explorers cluster showed low levels of social fear. The high-fear cluster during the high-fear task was also marginally associated with social fear (β = 0.36, SE = 0.18, t = 1.97, p = .05), such that those in the high-fear cluster showed higher levels of social fear. There was also a marginally positive association for children in the fearful explorers cluster during the high-fear task and object fear (β = −0.59, SE = 0.30, t = −1.94, p = .05), such that children in the fearful explorers clusters were less likely to show object fear. However, the clusters were not related to children’s inhibitory control or soothability as measured by the TBAQ.
Next, we examined the associations between the four clusters and select subscales of the ITSEA. Preliminary results showed that being in the high-fear cluster during the low-fear task was positively associated with inhibition to novelty (β = 0.38, SE = 0.17, t = 2.17, p < .05), such that those in the high-fear cluster were more likely to demonstrate inhibition to novelty. In addition, those in the fearful explorers cluster during the high-fear task were negatively related to inhibition to novelty (β = −0.43, SE = 0.20, t = −2.13, p < .05); that is, they were less likely to demonstrate inhibition to novelty. Next, membership in the fearful explorers cluster during the high-fear task was negatively associated with the anxious/worry scale (β = −0.19, SE = 0.09, t = −2.05, p < .05), such that those in the fearful explorers cluster were less likely to be anxious/worried. Membership in the fearful explorers cluster during the high-fear task was also negatively related to separation distress (β = −0.31, SE = 0.15, t = −2.10, p < .05), such that those in the fearful explorers cluster were less likely to demonstrate separation distress. The results showed that being in the high-fear group across low- and high-fear tasks was associated with more anxious types of behavior, whereas being in the fearful explorer cluster was associated with less.
4 |. DISCUSSION
The literature has highlighted fearful temperament as a key predictor of later anxiety problems in children (Degnan et al., 2010; Pérez-Edgar & Fox, 2005); however, few have examined the temporal dynamics of the expression of fear on a moment-to-moment basis and none have used sequential methods to group children based on their momentary trajectories. In this way, the current study has addressed a crucial gap in our understanding of the timing and order of fearful behaviors and therefore contributed to our understanding of what is perpetuating DF behavior in the moment (fear reactivity vs. a regulation deficit). Building on previous work, we aimed to distinguish groups of children’s fearful behaviors and examine the different momentary trajectories of these groups. As hypothesized, our examination identified a pattern of behavior that aligned with a DF trajectory, where children sustained fearful behaviors across the entire low-fear task and identified several distinctive clusters of fearful behavior that have historically aggregated into an “normative” fear group.
Our HMM search procedure discovered that a six-state model fits the data best. The six states included a low-fear state, three different no-fear states differing in caregiver proximity (no-fear/at-caregiver, no-fear/near-caregiver, and no-fear/far-from-caregiver), and two fearful states, again differing by caregiver proximity (fearful/at-caregiver and fearful/far-from-caregiver). Next, we used sequence clustering to group these moment-to-moment state trajectories to find collections of similar patterns between children. When looking at Table 6, one can see that the number of children in these clusters varied by the task they are in, highlighting the essential role of context in the examination of fearful behaviors. The clusters included external regulation, low reactive, fearful explorers, and high fear.
The external regulation cluster was characterized by children using the caregiver as a way to regulate their fear. Specifically, we see that when children in this cluster were with their caregiver, they rarely expressed fear, and that fear tended to grow as soon as they stepped away. This external regulation strategy has support in the literature with parents helping children regulate their negative emotions. Indeed, research has shown that in unfamiliar or distressing situations, children were more likely to engage the parent at this age as a regulation strategy by searching for them or staying close by (Grolnick, Bridges, & Connell, 1996). In this cluster, children could be refusing to stray far from the caregiver, and instead using them as a source of comfort and reassurance when fear of the unfamiliar experience became overwhelming.
The low-reactive cluster was characterized by children who display little to no fear at all. One could argue that these children may be exhibiting a different type of dysregulation—a lack of adaptive fear responses—when in the presence of fearful stimuli. It could be that these children were not as sensitive to the stimulus in their environment. Research supports that children with low environmental sensitivity actually display less negative affect in comparison with children with high sensitivity to environmental stimuli (Kagan, 1997; Rothbart, Ahadi, & Evans, 2000; Rothbart, Posner, & Kieras, 2006). Previous examinations of children’s temperament have found a similar low-reactive group (Beekman et al., 2015), and this low-reactive group has also been identified in past analyses with this sample (Morales et al., 2015). It could be that these children identify the exuberant, bold, surgent types but without information regarding their approach behavior or positive affect expression, this potential explanation is only speculative. Indeed, it is also possible that children in this group did not find the stimulus fearful, and they may have reacted strongly to a different type of fearful stimulus.
The fearful explorers cluster was particularly interesting because these children expressed fear but were still willing to explore their environment away from their caregiver. The children in this cluster could be employing internal regulation strategies. Children during the preschool years are making big gains in their abilities to regulate their emotions and begin to demonstrate more variety beyond physical coping methods and include cognitive strategies (Stansbury & Sigman, 2000). For example, Stansbury and Sigman (2000) found that children as young as 3 years can engage in cognitive reappraisal strategies during frustrating situations. In addition, Grolnick et al. (1996) demonstrated that children may engage in exploratory behavior as a way to distract themselves from distressing situations or use self-soothing strategies (such as thumb sucking) when the caregiver was not engaged. It could be that children in the fearful explorers group were able to reframe the situation in such a way as to keep their expressions of fear low (e.g., trying to understand the spider as a toy), rather than using their parents as an external regulator. It is also possible that children in this group do not view their caregiver as a secure base from which to explore (Ainsworth, Blehar, Waters, & Wall, 2015). This idea raises questions about the dynamic interplay of emotion regulation and attachment across child development; more research will be needed to answer these questions.
Children in the high-fear cluster spend much of their time in fear states. During the high-fear task, this was not unexpected, with some children remaining in fear even while close to caregiver. However, those children who were in this cluster during the low-fear task, which is supposed to be unfamiliar but not fear-inducing (puppet show), were not normative. This cluster in the low-fear task was indicative of DF (Buss, 2011; Buss et al., 2004; Buss & Kiel, 2013). DF is when children express fear in situations where that expression is not warranted. DF has been related to numerous child anxious outcomes (Buss et al., 2013; Buss & Kiel, 2013). This is the first time the profile has been replicated on a micro-scale, showing the robustness of the DF construct and how the fearful behaviors are temporally related. Furthermore, these results provide key insights into whether DF is high reactivity or a regulation deficit. The results from the current study suggest that DF may be a deficit of regulation rather than high-fear reactivity due to the role the proximity to caregiver had in differentiating the high-fear-reactive clusters, specifically the high-fear and fearful explorer clusters.
Though the results are compelling, there are a few limitations that should be noted. First, the sample consisted of mostly Caucasian, middle-class families and therefore may lack generalizability to other demographic groups. Second, it could be that children’s fearful behaviors are on a continuum and therefore characterizing them into clusters may not provide a complete understanding of the phenomena. In future work, approaches such as multidimensional scaling might be helpful to develop a more continuous measure of children’s fearful behavior trajectories. In addition, we examined children’s temperament and adjustment using the TBAQ and ITSEA at the same age as the moment-to-moment observation assessment. In our examination, we used these questionnaire data as a way to explore relations with our clusters and validate that they predict related constructs. However, future research should look at predicting later child outcomes and look at the clusters’ longitudinal stability. Last, another important avenue for future work is to further examine how emotion reactivity and regulation processes differ on a moment-to-moment level.
Overall, the clusters that emerged from our exploratory analyses are consistent with groups of children identified in other studies. However, we were able to differentiate what previous examinations referred to only as normative (Buss et al., 2013; Morales et al., 2015) with two distinct clusters: fearful explorers and external regulators. While these groups both show normative regulation, they differ in the way that regulation occurs. By examining the moment-to-moment latent groupings of children’s fearful behaviors, we were able to see that members of one group were using their caregiver as a regulator, whereas members of the other were engaging in internal regulation strategies, patterns we were not able to discern when using the data in aggregate form. Particularly unique and important to the current approach was the ability to simultaneously look at between-group differences by examining the group trajectories in aggregate, and examining the patterns present within each child’s individual state transitions.
In sum, we were able to replicate DF on a micro-scale for the very first time, demonstrating the robustness of this construct. We were also able to provide insight into why children with DF behave the way that they do. The results from the current study suggest that it could be a regulation deficit rather than high-fear reactivity. In addition, we were able to further distinguish different types of fear groups based on children’s regulation strategy, which allowed us to provide deeper insight into how children in different fearful group behaviors differed in terms of timing and order. Therefore, this method provides us the utility of groups with the attention to temporal relations to provide a more complete picture of the processes within children’s fearful behaviors.
ACKNOWLEDGMENTS
This work was supported by funding from the National Institutes of Health (Grants are NIH R01MH075750 and NIH UL1TR002014).
Funding information
National Institute of Mental Health, Grant/Award Number: NIH R01MH075750; Clinical and Translational Science Institute, Penn State University, Grant/Award Number: CTSI UL1TR002014
Footnotes
CONFLICT OF INTEREST
The authors declare no conflicts of interest with regard to the funding source for this study.
REFERENCES
- Ainsworth MDS, Blehar MC, Waters E, & Wall SN (2015). Patterns of attachment: A psychological study of the strange situation. New York, NY: Psychology Press. [Google Scholar]
- Baum LE, & Petrie T (1966). Statistical inference for probabilistic functions of finite state Markov chains. The Annals of Mathematical Statistics, 37(6), 1554–1563. 10.1214/aoms/1177699147 [DOI] [Google Scholar]
- Beekman C, Neiderhiser JM, Buss KA, Loken E, Moore GA, Leve LD, … Reiss D (2015). The development of early profiles of temperament: Characterization, continuity, and etiology. Child Development, 86(6), 1794–1811. 10.1111/cdev.12417 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brinberg M, Ram N, Hülür G, Brick TR, & Gerstorf D (2018). Analyzing dyadic data using grid-sequence analysis: Interdyad differences in intradyad dynamics. The Journals of Gerontology: Series B, 73(1), 5–18. 10.1093/geronb/gbw160 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Buss KA (2011). Which fearful toddlers should we worry about? Context, fear regulation, and anxiety risk. Developmental Psychology, 47(3), 804–819. 10.1037/a0023227 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Buss KA, Brooker RJ, & Leuty M (2008). Girls most of the time, boys some of the time: Gender differences in toddlers’ use of maternal proximity and comfort seeking. Infancy, 13(1), 1–29. 10.1080/15250000701779360 [DOI] [Google Scholar]
- Buss KA, Davidson RJ, Kalin NH, & Goldsmith HH (2004). Context-specific freezing and associated physiological reactivity as a dysregulated fear response. Developmental Psychology, 40(4), 583–594. 10.1037/0012-1649.40.4.583 [DOI] [PubMed] [Google Scholar]
- Buss KA, Davis EL, Kiel EJ, Brooker RJ, Beekman C, & Early MC (2013). Dysregulated fear predicts social wariness and social anxiety symptoms during kindergarten. Journal of Clinical Child and Adolescent Psychology, 42(5), 603–616. 10.1080/15374416.2013.769170 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Buss KA, Davis EL, Ram N, & Coccia M (2018). Dysregulated fear, social inhibition, and respiratory sinus arrhythmia: A replication and extension. Child Development, 89(3), 214–228. 10.1111/cdev.12774 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Buss KA, & Kiel EJ (2013). Temperamental risk factors for pediatric anxiety disorders In Vasa RA, & Roy AK (Eds.), Pediatric anxiety disorders: A clinical guide (pp. 47–68). New York, NY: Springer; 10.1007/978-1-4614-6599-7_3 [DOI] [Google Scholar]
- Caiado J, Maharaj E, & D’Urso P (2016). Time-series clustering In Hennig C, Meila M, Murtagh F, & Rocci R (Eds.), Handbook of cluster analysis (pp. 241–263). Boca Raton, FL: Taylor & Francis Group LLC. [Google Scholar]
- Carranza JA, González-Salinas C, & Ato E (2013). A longitudinal study of temperament continuity through IBQ, TBAQ and CBQ. Infant Behavior and Development, 36(4), 749–761. 10.1016/j.infbeh.2013.08.002 [DOI] [PubMed] [Google Scholar]
- Carter AS, Briggs-Gowan MJ, Jones SM, & Little TD (2003). The Infant-Toddler Social and Emotional Assessment (ITSEA): Factor structure, reliability, and validity. Journal of Abnormal Child Psychology, 31(5), 495–514. 10.1023/A:1025449031360 [DOI] [PubMed] [Google Scholar]
- Chavira DA, Garland AF, Daley S, & Hough R (2008). The impact of medical comorbidity on mental health and functional health outcomes among children with anxiety disorders. Journal of Developmental and Behavioral Pediatrics, 29(5), 394–402. 10.1097/DBP.0b013e3181836a5b [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cho S, & Buss KA (2017). Toddler parasympathetic regulation and fear: Links to maternal appraisal and behavior. Developmental Psychobiology, 59(2), 197–208. 10.1002/dev.21481 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Degnan KA, Almas AN, & Fox NA (2010). Temperament and the environment in the etiology of childhood anxiety. Journal of Child Psychology and Psychiatry, 51(4), 497–517. 10.1111/j.1469-7610.2010.02228.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fox NA, Snidman N, Haas SA, Degnan KA, & Kagan J (2015). The relations between reactivity at 4 months and behavioral inhibition in the second year: Replication across three independent samples. Infancy, 20(1), 98–114. 10.1111/infa.12063 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gabadinho A, Ritschard G, Müller NS, & Studer M (2011). Analyzing and visualizing state sequences in R with TraMineR. Journal of Statistical Software, 40(4), 1–37. 10.18637/jss.v040.i04 [DOI] [Google Scholar]
- Garcia-Coll C, Kagan J, & Reznick JS (1984). Behavioral inhibition in young children. Child Development, 55(3), 1005–1019. 10.2307/1130152 [DOI] [Google Scholar]
- Goldsmith HH (1996). Studying temperament via construction of the Toddler Behavior Assessment Questionnaire. Child Development, 67(1), 218–235. 10.2307/1131697 [DOI] [PubMed] [Google Scholar]
- Grolnick WS, Bridges LJ, & Connell JP (1996). Emotion regulation in two-year-olds: Strategies and emotional expression in four contexts. Child Development, 67(3), 928–941. 10.1111/j.1467-8624.1996.tb01774.x [DOI] [PubMed] [Google Scholar]
- Halkidi M, Batistakis Y, & Vazirgiannis M (2002). Clustering validity checking methods: Part II. SIGMOD Record, 31(3), 19–27. 10.1145/601858.601862 [DOI] [Google Scholar]
- Halkidi M, Vazirgiannis M, & Hennig C (2016). Method-independent indices for cluster validation and estimating the number of clusters In Hennig C, Meila M, Murtagh F, & Rocci R (Eds.), Handbook of clustering analysis (pp. 595–618). Boca Raton, FL: Taylor & Francis Group LLC. [Google Scholar]
- Hennig C (2016). Clustering strategy and method selection In Hennig C, Meila M, Murtagh F, & Rocci R (Eds.), Handbook of clustering analysis (pp. 703–730). Boca Raton, FL: Taylor & Francis Group LLC. [Google Scholar]
- Hennig C (2018). fpc: Flexible procedures for clustering (R package version 2.1–11). Retrieved from https://CRAN.R-project.org/package=fpc
- Izard CE, Hembree EA, Dougherty LM, & Spizzirri CC (1983). Changes in facial expressions of 2- to 19-month-old infants following acute pain. Developmental Psychology, 19(3), 418–426. 10.1037/0012-1649.19.3.418 [DOI] [Google Scholar]
- Kagan J (1997). Temperament and the reactions to unfamiliarity. Child Development, 68(1), 139–143. 10.2307/1131931 [DOI] [PubMed] [Google Scholar]
- Kagan J, & Fox NA (2007). Biology, culture, and temperamental biases In Eisenberg N, Damon W, & Lerner RM (Eds.), Handbook of child psychology: Social, emotional, and personality development (pp. 167–225). Hoboken, NJ: John Wiley & Sons Inc; 10.1002/9780470147658.chpsy0304 [DOI] [Google Scholar]
- Kagan J, Reznick JS, Clarke C, Snidman N, & Garcia-Coll C (1984). Behavioral inhibition to the unfamiliar. Child Development, 55(6), 2212–2225. 10.2307/1129793 [DOI] [Google Scholar]
- Kassambara A (2017). Practical guide to cluster analysis in R: Unsupervised machine learning. CreateSpace Independent Publishing Platform. [Google Scholar]
- Kessler RC, Petukhova M, Sampson NA, Zaslavsky AM, & Wittchen H-U (2012). Twelve-month and lifetime prevalence and lifetime morbid risk of anxiety and mood disorders in the United States. International Journal of Methods in Psychiatric Research, 21(3), 169–184. 10.1002/mpr.1359 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kidwell KM, Nelson TD, & Van Dyk T (2015). Parenting stress and child physical health among a low-income sample: The moderating role of child anxiety. Journal of Health Psychology, 20(11), 1377–1387. 10.1177/1359105313512352 [DOI] [PubMed] [Google Scholar]
- Kiel EJ, & Buss KA (2013). Toddler inhibited temperament, maternal cortisol reactivity and embarrassment, and intrusive parenting. Journal of Family Psychology, 27(3), 512–517. 10.1037/a0032892 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kopp CB (1989). Regulation of distress and negative emotions: A developmental view. Developmental Psychology, 25(3), 343–354. 10.1037/0012-1649.25.3.343 [DOI] [Google Scholar]
- Li T, Ma S, & Ogihara M (2004). Entropy-based criterion in categorical clustering In Proceedings of the Twenty-first International Conference on Machine Learning (pp. 68–72). New York, NY: ACM; 10.1145/1015330.1015404 [DOI] [Google Scholar]
- Meila M (2007). Comparing clusterings—An information based distance. Journal of Multivariate Analysis, 98(5), 873–895. 10.1016/j.jmva.2006.11.013 [DOI] [Google Scholar]
- Meila M (2016). Criteria for comparing clusterings In Hennig C, Meila M, Murtagh F, & Rocci R (Eds.), Handbook of clustering analysis (pp. 619–635). Boca Raton, FL: Taylor & Francis Group LLC. [Google Scholar]
- Merikangas KR, He J-P, Burstein M, Swanson SA, Avenevoli S, Cui L, … Swendsen J (2010). Lifetime prevalence of mental disorders in U.S. adolescents: Results from the national comorbidity survey replication–adolescent supplement (NCS-A). Journal of the American Academy of Child and Adolescent Psychiatry, 49(10), 980–989. 10.1016/j.jaac.2010.05.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Morales S, Pérez-Edgar KE, & Buss KA (2015). Attention biases towards and away from threat mark the relation between early dysregulated fear and the later emergence of social withdrawal. Journal of Abnormal Child Psychology, 43(6), 1067–1078. 10.1007/s10802-014-9963-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Needleman SB, & Wunsch CD (1970). A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of Molecular Biology, 48(3), 443–453. 10.1016/0022-2836(70)90057-4 [DOI] [PubMed] [Google Scholar]
- Pérez-Edgar K, & Fox NA (2005). Temperament and anxiety disorders. Child and Adolescent Psychiatric Clinics of North America, 14(4), 681–706. 10.1016/j.chc.2005.05.008 [DOI] [PubMed] [Google Scholar]
- R Core Team (2018). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; Retrieved from https://www.R-project.org/ [Google Scholar]
- Rothbart MK, Ahadi SA, & Evans DE (2000). Temperament and personality: Origins and outcomes. Journal of Personality and Social Psychology, 78(1), 122–135. 10.1037//0022-3514.78.1.122 [DOI] [PubMed] [Google Scholar]
- Rothbart MK, Posner MI, & Kieras J (2006). Temperament, attention, and the development of self-regulation In McCartney K, & Phillips D (Eds.), Blackwell handbook of early childhood development (pp. 338–357). Malden, MA: Blackwell Publishing. [Google Scholar]
- Rubin KH, Coplan RJ, & Bowker JC (2009). Social withdrawal in childhood. Annual Review of Psychology, 60, 141–171. 10.1146/annurev.psych.60.110707.163642 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shackman AJ, Fox AS, Oler JA, Shelton SE, Davidson RJ, & Kalin NH (2013). Neural mechanisms underlying heterogeneity in the presentation of anxious temperament. Proceedings of the National Academy of Sciences of the United States of America, 110(15), 6145–6150. 10.1073/pnas.1214364110 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stansbury K, & Sigman M (2000). Responses of preschoolers in two frustrating episodes: Emergence of complex strategies for emotion regulation. The Journal of Genetic Psychology, 161(2), 182–202. 10.1080/00221320009596705 [DOI] [PubMed] [Google Scholar]
- Stifter CA, & Rovine M (2015). Modeling dyadic processes using hidden markov models: A time series approach to mother-infant interactions during infant immunization. Infant and Child Development, 24(3), 298–321. 10.1002/icd.1907 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Visser I, & Speekenbrink M (2010). depmixS4: An R package for hidden markov models. Journal of Statistical Software, 36(7), 1–21. 10.18637/jss.v036.i07 [DOI] [Google Scholar]
