Quantifying Severity of Preschool-Aged Children’s Internalizing Behaviors: A Daily Diary Analysis

Abstract

A subset of preschool-aged children meets criteria for impairing and persistent anxiety and depression. However, the overlap between normative emotional development and impairing symptoms complicates assessments of internalizing problems in early childhood. Given the benefits of early identification/prevention and avoiding over-pathologizing typical development, empirical information is needed to norm expression of internalizing behaviors. In this 14-day online diary study, 609 primary caregivers of 3–5-year-old children reported the frequency of children’s daily separation and social anxiety and depressive behaviors and impairment. Item response theory analyses quantified specific frequencies at which each behavior was psychometrically severe/rare. Patterns varied for each behavior; for example, distress when anticipating separation had to occur at least 10 times and sadness at least 35 times over 14 days to be considered severe. Most social anxiety behaviors had to occur approximately every other day to be considered severe. Parameters did not vary by child age or sex, and behaviors were significantly associated with impairment. These data provide empirical information for refining internalizing behavior assessment in preschool-aged children and can be used as benchmarks by child practitioners to assess the extent to which frequencies fall in the range of developmentally typical behavior versus those that may be more severe.

Symptoms and diagnoses of anxiety and depression have been identified across numerous community samples of preschool-aged (3–5-year-old) children (Bufferd et al., 2011; Dougherty, Leppert, et al., 2015; Franz et al., 2013; Lavigne et al., 2009; Wichstrom et al., 2012). These forms of internalizing psychopathology are associated with impairment for youth and their families (Barrios et al., 2019; Bufferd et al., 2011; Luby, Belden, et al., 2009; Towe-Goodman et al., 2014) and, for some children, persist over time and/or contribute to other disorders and difficulties (Bufferd et al., 2012; Dougherty, Smith, et al., 2015; Dougherty et al., 2013; Ezpeleta et al., 2020; Finsaas et al., 2018; Luby et al., 2014). Overall, internalizing symptoms and disorders contribute to suffering for children and families, and the continuity and escalation of these difficulties come with high personal and societal costs (Baxter et al., 2014; Greenberg et al., 2015). This body of research has illuminated that some young children experience clinically significant levels of persistent anxiety and depression and signifies important advances in the identification of early-emerging psychopathology and sequalae. Nevertheless, questions remain with regard to the extent to which symptoms of psychopathology can be distinguished from typical development (Bufferd et al., 2016). Given the high degree of overlap between normative emotions and behaviors in early childhood and symptoms of anxiety (e.g., separation and social fears) and depression (e.g., irritability, tearfulness) (Cole et al., 2008; Muris, 2010; Spring & Carlson, 2021; Vidal-Ribas et al., 2016), more data are needed to inform assessment efforts during this developmental period. The present study examines daily depressive and anxiety behaviors and associated impairment in a large, diverse community sample of preschool-aged children. These data will allow for the characterization and analysis of the spectrum of internalizing behavior that young children display and identify specific levels of behavior that may be more severe to generate benchmarks for use by child practitioners who are tasked with screening young children’s emotional development with few empirical guidelines.

Categorical approaches to the assessment of anxiety and depression in young children have yielded useful information about the proportion of children displaying symptoms at the higher severity range of these behaviors (Dougherty, Leppert, et al., 2015), and there have been important efforts to improve diagnostic criteria using developmentally-relevant information for young children (Egger & Emde, 2011; Luby, Mrakotsky, et al., 2003). Although developmentally sensitive diagnostic assessment can capture children’s behavior at greater severity, categorical approaches would not identify children’s behavior that falls short of these thresholds and can obscure relevant information about behavior that may still be impairing and/or indicative of burgeoning risk (Fatori et al., 2018; Keenan et al., 2008; Wesselhoeft et al., 2013). Further, these categorical thresholds, albeit developmentally sensitive, are not empirically derived and may not sufficiently capture severity of behavior (Dougherty et al., 2021). These limitations are particularly relevant to the preschool period when development is rapid, and normative variation in young children’s behavior is difficult to distinguish from clinically salient internalizing behavior (Bufferd et al., 2016). In contrast, long-established dimensional assessment approaches in young children measure a wider range of behavior and continue to be an important part of clarifying early-emerging emotional and behavioral difficulties (Arend et al., 1996; Moreland & Dumas, 2008; Rescorla et al., 2011; Wakschlag et al., 2015). Dimensional assessment accurately reflects the continuous nature of anxiety and depression as well as the overlap between normative emotional phenomena/personality and psychopathology identified by both structural (Achenbach, 2020; Kotov et al., 2021; Lahey et al., 2017; Olino et al., 2014) and genomic (Levey et al., 2020; Martin et al., 2018; Smoller et al., 2019; Wray et al., 2018) research.

Developmental considerations have been increasingly incorporated into the dimensional assessment of psychopathology in early childhood to improve the validity and utility of measurement in this age group. For example, Wakschlag and colleagues have developed and adapted both questionnaire (Blackwell et al., 2020; Wakschlag et al., 2014) and observational measures (Mian et al., 2015; Wakschlag, Briggs-Gowan, et al., 2008; Wakschlag, Hill, et al., 2008) to sharpen the characterization of the normal-abnormal spectrum of behavior in preschool-aged children. Among many advances, they have identified specific features of mood dysregulation/irritability and related behaviors (e.g., tantrums) in preschool-aged children as clinically significant indicators of risk for psychopathology; these features include irritable mood that escalates quickly and persists, difficulty returning to baseline from irritable mood or tantrums, daily tantrums, and low frustration tolerance (Wakschlag et al., 2014; Wakschlag et al., 2012; Wiggins et al., 2021). This work has generated critical information in moving toward the goal of characterizing the continuum of behavior in early childhood.

To further refine developmentally based and empirically derived assessment of psychopathology in preschool-aged children to generate specific, translational benchmarks of emotional development, more data are needed to address some challenges in this work. First, the DSM criteria and some questionnaire assessments include frequency descriptors of behaviors that are not operationally defined. For example, some behaviors must occur “often” or be “recurrent” or “excessive” to meet diagnostic criteria. Similarly, on questionnaire measures, informants (e.g., parents) may be asked to indicate whether a behavior occurs “sometimes” or “always”. Across diagnostic criteria and some questionnaire measures, these undefined parameters necessitate the use of subjective judgement in describing a child’s behavior, precluding comparison among children and limiting efforts to understand the level of behavior that is associated with concurrent and future psychopathology and impairment. This limitation is particularly problematic given the absence of norms for these behaviors. Second, various assessment approaches typically require periods of recall of a month or longer; such periods may make it difficult to remember details about the parameters (e.g., frequency, intensity) of children’s behavior that could inform more precise assessment. Finally, a majority of work documenting the range of behavior relevant to psychopathology has included externalizing behaviors (Belden et al., 2008; Hong et al., 2015; Wakschlag et al., 2014); although some of this work includes transdiagnostic constructs and behaviors such as irritability/tantrums, little work to date in this area has focused specifically on anxiety and depressive behaviors in preschool-aged children.

To address these challenges and limitations, a daily measure in which parents report the frequencies of children’s specific behaviors each day can be used to quantify the severity of internalizing behavior. A daily diary with open-ended questions about the frequencies of behaviors, rather than a priori cutoffs or undefined terms, would provide more precise empirical information about the spectrum of the frequency and severity of normative to problematic levels of anxiety and depressive behaviors. More precise parameters can be used to generate norms of internalizing behaviors in preschool-aged children and, importantly, provide specific, concrete guidelines for child practitioners (e.g., pediatricians) who decide whether to refer parents to services to help with children’s behavior. Data on more specific indicators of risk could improve the objectivity of decision making, especially given the overlap between normative development and internalizing behaviors. Further, these data can aid in the identification of children who may be at risk for clinically significant psychopathology while also contributing information that can reduce diagnostic inflation and over-pathologizing developmentally typical/expected difficulties (Francis & Widiger, 2012). Assessing specific, empirically-derived frequencies of behavior is an essential part of effective screening, particularly for behaviors that are relatively common during a particular developmental period (Wiggins et al., 2021). That said, although assessing frequencies of children’s behavior can be a straightforward, feasible, translational approach for screening for internalizing difficulties, particularly in busy primary care settings in which mental health difficulties are often missed (Bricker et al., 2004), screening based on frequency of behavior alone would not capture the full constellation of relevant factors (e.g., duration, intensity, and setting of behavior; distress/impairment for the child and family). The use of behavior frequencies in screening would provide guidelines for potential referral to improve early detection of emotional difficulties; such norms or benchmarks, like those for other developmental processes (e.g., motor development; language development), would permit improved identification of children who may be at risk for worsening internalizing difficulties (Marks et al., 2011).

Another benefit of a daily measure is the reduction of retrospective recall bias in parents’ recalling children’s behaviors, potentially improving the accuracy of the data obtained. Further, the assessment of behavior and related impairment within the same day may enhance recall for the actual level of distress and/or difficulties associated with the behavior(s) each day, improving our understanding of the ways in which internalizing behaviors impact children and families.

Using this daily diary approach in a sample of 291 parents of 3–5-year-old children, we (Bufferd et al., 2017, 2019) previously identified specific frequencies of anxiety and depressive behaviors that were considered psychometrically severe/rare. Certain behaviors had to occur more frequently than others to be considered severe. For example, fear about going to sleep without being near the caregiver had to occur ten or more times over 14 days to be considered severe/rare, whereas the child worrying that the caregiver would be hurt or leave home and not return only had to occur two or more times over 14 days; shyness around peers and new people were not psychometrically severe at any frequency, suggesting that these behaviors may be more developmentally typical than others. For depressive behaviors, sadness had to occur at least 31 times over 14 days (roughly twice per day or more) and irritability 23 times for the behaviors to be considered severe, whereas low interest/pleasure had to occur three or more times over the 14 days suggesting that anhedonia-like behavior may be less developmentally normative than mood-relevant behaviors. We also identified significant associations between anxiety and depressive behaviors and impairment assessed daily; most correlations were strong (greater than .50). Although these data reflect a first step toward developing more precise information about anxiety and depressive behaviors in preschool-aged children, additional data are needed in larger samples to 1) more accurately chart the range of normative variation, 2) identify the level at which anxiety and depressive behaviors may be clinically significant, and 3) denote the associations with daily impairment for children and their families. In addition, it is not yet known whether parameters for behavior vary based on demographic characteristics such as child age and sex; such information is needed to further refine assessment of common, yet potentially impairing internalizing behaviors.

The goal of the present study was to replicate previous work (N = 291) (Bufferd et al., 2017, 2019) in an independent, larger community sample (N = 609) of diverse 3–5-year-old children. We used a 14-day diary in which parents reported about their children’s daily separation and social anxiety and depressive behaviors, along with the extent to which the behaviors caused distress and/or impairment each day. We analyzed the frequency and severity of daily anxiety and depressive behaviors and examined whether item parameters varied by demographic characteristics (children’s age and sex) in preschool-aged children. In addition, we examined the extent to which daily behavior is associated with daily impairment. Based on our prior work, we hypothesized that higher base rate behaviors (e.g., shyness around new adults, irritability) will occur at greater frequencies to be considered psychometrically severe, whereas less normative behavior (e.g., worry about separation due to natural disaster; fatigue) will occur less frequently to be considered severe. Given potential improvement in emotion regulation and self-control through the preschool period, we hypothesize that items parameters will vary by age such that more dysregulated behaviors (e.g., irritability, tantrums) will occur at lower frequencies to be considered severe/rare in older children compared to younger children. However, we do not have any a priori hypotheses about potential sex differences given the absence of theoretical and empirical information.

Finally, we also hypothesized that severity/frequency of behaviors would be associated with daily impairment, including children’s and parents’ distress, and children’s difficulties in psychosocial functioning and relationships given ample evidence of links between internalizing symptoms and impairment (Luby, Belden, et al., 2009; Markon, 2010; Towe-Goodman et al., 2014). The data in the present study can 1) clarify benchmarks of behavior relevant to emotional development that may suggest risk for psychopathology as in other areas of development (e.g., motor, language) and 2) contribute to efforts to demarcate ranges of risk in dimensions of mental health constructs (Bufferd et al., 2016; Ruggero et al., 2019; Shah et al., 2020) as commonly done in medical testing; this information can be useful to both child practitioners and parents/caregivers. Further, these data respond to calls to identify risk for mental health difficulties earlier in life to support preventative efforts that reduce suffering and costs (Wakschlag et al., 2019; Wissow et al., 2021).

Method

Participants

Parents of 3–5-year-old children were invited to participate in a 14-day online daily diary study of anxiety and depressive behaviors and impairment. Participants were eligible if they had nightly Internet access, could read and speak English, were the primary caregiver with at least 50% custody of the child, and lived within 120 miles of the participating universities; in addition, children could not have any major medical or developmental disabilities. Flyers advertising the study were sent to local pediatricians, preschools/daycares, and community institutions (e.g., libraries) and posted on social media (e.g., on pages for parenting groups) inviting parents to complete an online screener for eligibility. We used both general and targeted flyers to recruit participants: general flyers listed all eligibility criteria, and targeted flyers listed these same criteria and also mentioned internalizing behaviors (“Do you have a child who seems shy, anxious, or has difficulty separating from his or her parent? Do you have a child who is moody, irritable, or sad?”). Targeted flyers were used in an attempt to increase recruitment of parents of children who displayed anxiety and/or irritability/sadness to assess the full range of internalizing behaviors.

Online screeners were used to assess eligibility criteria. However, all study eligibility criteria were confirmed with each participant by telephone prior to enrolling. We received a total of 1,717 screeners. Of the screeners from individuals we did not enroll, most did not respond to our contact (n = 404) or did not meet eligibility criteria (n = 484). In total, 628 eligible parents completed baseline measures; 19 of these parents did not start the diary measures, so the final sample included N = 609 participants. Most respondents (96.6%) were mothers. The sample was diverse: approximately 45% of the children were from racial/ethnic minority backgrounds. See Table 1 for participant demographics. Participants were recruited from a 120-mile radius around California State University San Marcos and University of Maryland. The Institutional Review Boards at both universities approved the study. Participants provided informed consent electronically and were financially compensated for participation.

Table 1.

Demographic Characteristics of the Study Sample

Child mean age: years (SD; range)	4.23 years (.78; 3–5)
Age 3 n (%)	257 (42.2%)
Age 4 n (%)	236 (38.8%)
Age 5 n (%)	116 (19.0%)
Child sex: female n (%)	278 (45.6%)
Child race/ethnicity: n (%)
African American/Black	47 (7.7%)
Asian	31 (5.1%)
Hispanic/Latinx	73 (23.0%)
Multi-ethnic/Other	122 (20.0%)
White	336 (55.2%)
Childcare/school:
Mean number of hours of preschool, daycare, and/or other childcare settings per week (SD; range)	22.49 hours (17.01; 0–60)
Parents’ marital status: n (%)
Married or living together	533 (87.5%)
Family incomea: n (%)
<$40,000	78 (12.9%)
$40,000 - $70,000	104 (17.2%)
$70,001 - $100,000	130 (21.5%)
>$100,000	294 (48.5%)
Parents’ employmentb: n working outside the home (%)
Primary caregiver/parent (96.6% mothers)	361 (59.8%)
Second caregiver/parent	535 (90.8%)
Parents’ educationc: n graduated college (%)
At least one parent graduated college	482 (79.9%)
Neither parent graduated college	121 (20.1%)

Note: N = 609.

^a0.5% of the sample (n = 3) did not indicate their family income level.

^bEmployment information was not reported for 0.8% (n = 5) of primary caregivers/parents and for 3.3% (n = 20) of the second caregivers/parents.

^cLevel of education was not reported for 1.0% (n = 6) of the sample.

The average number of diaries completed was 12.84 (SD = 2.39) out of 14; 580 (95.2%) participants completed at least 7 of the 14 diary entries, 556 (91.3%) completed at least 10 days, and 393 (64.5%) completed all 14 days. Rates of completion were consistent with or higher than other daily diary studies with children (Allen et al., 2010; Beidel et al., 1999; Beidel et al., 2000; Colasante et al., 2016).

Measures

Daily Anxiety and Depressive Behaviors.

An online daily diary was used to assess children’s daily separation and social anxiety and depressive behaviors. Items were derived from two validated, reliable measures, the Early Childhood Inventory (ECI) (Gadow & Sprafkin, 1997, 2000) and the Preschool Age Psychiatric Assessment (PAPA) (Egger et al., 1999) based on the symptoms of separation anxiety, social anxiety, and major depression diagnoses in the DSM (American Psychiatric Association, 2013). We also used this daily diary in pilot work (Bufferd et al., 2017, 2019). Parents reported the daily frequency of eight separation anxiety behaviors, four social anxiety behaviors, and twelve depressive behaviors for 14 days. The full text of diary items is available in the Open Science Framework (OSF) repository: https://osf.io/q5ucx/?view_only=05c26787cb6e4f7cb24a7a46ebdbc632. The frequency of each behavior was summed across the 14 days to create a total frequency across the dairy period. After examining the distribution and imputing data, frequency categories were created to generate item parameters for each behavior (see sections on Data Analysis for more information and Results for the specific items and frequency categories).

Daily Impairment and Distress.

To assess the extent to which anxiety and depressive behaviors contributed to impairment and/or distress/stress for children and their families each day, we asked three sets of questions derived from the ECI and the PAPA. First, when reporting the frequency of each anxiety and depressive behavior each day, parents also rated the extent to which each behavior was problematic that day (e.g., caused child and/or parent distress; child’s routine disrupted; relationships with others affected) on a 1–5 scale (1: not at all to 5: a great deal). These ratings were averaged across each set of separation (M = 1.09, SD = 0.13) and social (M = 1.08, SD = 0.13) anxiety and depressive behaviors (M = 1.24, SD = 0.18) across all 14 days. Second, parents provided an overall rating of impairment due to separation¹ (M = 1.21, SD = 0.30) and social (M = 1.14, SD = 0.24) anxiety and depressive (M = 1.59, SD = 0.49) behaviors that day on the same scale; these ratings were also averaged across all 14 days. Finally, parents rated stress for the child and themselves overall each day. These ratings were not specific to individual anxiety or depressive behaviors; they reflected general daily feelings of distress for children and parents. Parents rated three items each day on a 1–5 scale (1: not at all stressful to 5: extremely stressful), and scores were averaged across the 14-day diary period: parenting was difficult/stressful (M = 1.69, SD = 0.45), the extent to which the parent-child relationship was stressful (M = 1.63, SD = 0.46) and level of stress for the child (M = 1.57, SD = 0.46).

Procedure

Research staff confirmed eligibility for the study with interested parents by telephone. Parents who enrolled in the study were provided detailed instructions by telephone for completing the online daily diary. Participants were asked to complete each diary after their child’s bedtime and report about their child’s behavior and impairment that day. All participants started their first diary on a Monday evening and were asked to complete a diary entry each evening for 14 consecutive nights. Links were sent to participants at 7:30pm each evening in the participant’s local time zone. Study staff checked for completed diary entries each morning; if the diary was not completed, staff contacted participants to request completion of the diary by noon that day (based on their child’s behaviors from the previous day). If the parent could not complete the diary by noon, that diary entry was considered missed.

Data Analysis

As not all participants completed the full 14-day protocol, the total number of behaviors reported were imputed using the frequency of the specific behavior on all available days. We used Markov Chain Monte Carlo estimation to impute 25 complete datasets; estimates were pooled across the multiple imputations. To prepare the data for analysis, we recoded the frequency distribution for each item across the 14-day diary period into ordinal variables with four categories reflecting whether children displayed behaviors roughly below the 50^th percentile, between the 50^th and 80^th percentile; between the 81^st and 95^th percentile; and above the 95^th percentile for specific behaviors over the 14-day period (Table 2; the full text of diary items is available in the OSF repository: https://osf.io/q5ucx/?view_only=05c26787cb6e4f7cb24a7a46ebdbc632). Although we sought for uniformity in these categories, some behaviors had relatively restricted ranges; therefore, category ranges vary between items. These ordinal categories were used in the item analyses.

Table 2.

Frequencies of Each Behavior Item Across the 14-day Diary Period

		Category 1	Category 2	Category 3	Category 4
Separation Anxiety
Distress when anticipating separation	Frequency of Behavior	0	1–3	4–9	10+
	Percentile Range	< 54%	55–72%	73–95%	> 95%
Worry about caregiver’s safety/return home	Frequency of Behavior	0	1	2+
	Percentile Range	< 83%	83–94%	>94%
Worry about disaster separating child from caregiver	Frequency of Behavior	0	1	2+
	Percentile Range	< 93%	86–94%	>94%
Avoided going places without caregiver	Frequency of Behavior	0	1–2	3–6	7+
	Percentile Range	< 50%	50–76%	77–95%	> 95%
Worry about being left home without caregiver	Frequency of Behavior	0	1–2	3+
	Percentile Range	< 74%	74–92%	> 92%
Fearful of going to sleep without caregiver	Frequency of Behavior	0	1–6	7–13	14+
	Percentile Range	< 45%	45–80%	81–94%	>94%
Separation nightmares	Frequency of Behavior	0	1–2	3+
	Percentile Range	< 76%	76–93%	> 93%
Reported illness when separation anticipated	Frequency of Behavior	0	1–3	4+
	Percentile Range	< 80%	81–95%	> 95%
Social Anxiety
Shy around new people	Frequency of Behavior	0	1–2	3–6	7+
	Percentile Range	< 48%	48–61%	62–88%	> 88%
Shy with peers	Frequency of Behavior	0	1–3	4–6	7+
	Percentile Range	< 65%	65–85%	86–94%	> 94%
Shy with family members/familiar adults	Frequency of Behavior	0	1	2–3	4+
	Percentile Range	< 59%	59–87%	88–95%	> 95%
Distress/withdrawal in social situations	Frequency of Behavior	0	1–2	3–7	8+
	Percentile Range	< 66%	66–76%	77–95%	> 95%
Depression
Sadness	Frequency of Behavior	0–9	10–20	21–34	35+
	Percentile Range	< 56%	56%−82%	83%−96%	> 96%
Irritability	Frequency of Behavior	0–13	14–26	27–43	44+
	Percentile Range	< 54%	54–81%	82–96%	> 96%
Tantrums	Frequency of Behavior	0–4	5–11	12–26	27+
	Percentile Range	< 52%	52–80%	81–95%	> 95%
Low interest/pleasure	Frequency of Behavior	0	1–2	3–6	7+
	Percentile Range	< 64%	64–85%	86–95%	> 95%
Talked about death/suicide	Frequency of Behavior	0	1	2–4	5+
	Percentile Range	< 73%	73–86%	87–95%	> 95%
Low self-worth	Frequency of Behavior	0	1–3	4–9	10+
	Percentile Range	< 53%	53–84%	85–95%	> 95%
Fatigue	Frequency of Behavior	0	1	2–5	6+
	Percentile Range	<69%	69–82%	83–95%	> 95%
Appetite and/or weight changes	Frequency of Behavior	0	1	2–3	4+
	Percentile Range	< 58%	59–80%	81–94	> 94%
Sleep difficulties	Frequency of Behavior	0–1	2–5	6–7	8+
	Percentile Range	< 52%	52–87%	88–96%	> 96%
Change in activity level	Frequency of Behavior	0	1–4	5–10	11+
	Percentile Range	< 46%	46–84%	85–95%	> 95%
Difficulty concentrating/making decisions	Frequency of Behavior	0	1–3	4–9	10+
	Percentile Range	< 60%	60–81%	82–93%	> 93%
Tearfulness/sensitivity	Frequency of Behavior	0–9	10–20	21–33	34+
	Percentile Range	< 53%	53–82%	82–95%	> 95%

Frequencies displayed in bold indicate severity greater than the 95^th percentile based on item analyses (see Table 3) or the level at which the behavior is considered psychometrically severe/rare.

Dimensionality for the separation anxiety dimension, social anxiety, and depression dimensions was tested using confirmatory factor analysis (CFA) using the robust weighted least squares estimator in Mplus 8.5 (Muthén & Muthén, 1998–2018). Model fit was evaluated using the chi-square test, the comparative fit index (CFI), and the root mean square error of approximation (RMSEA); CFI values >0.90 and RMSEA values <0.08 suggest good fit (Lord, 1980; Reise & Waller, 1990). We report model fit information based on the average across imputations. Next, we conducted Item Response Theory (IRT) (Reise et al., 2005) analyses to examine item characteristics using a graded response model [GRM; (Samejima, 1970)] in Mplus using robust maximum likelihood estimation. The IRT models complement the concrete frequency metrics by providing information about (1) the youths’ severity required to lead to the observable behavior and (2) the connection to the specific instrumentation used in the assessment. Discrimination values reflect the extent to which items relate to the latent construct/provide sufficient information about the latent trait; higher discrimination values are associated with greater and more precise information (Baker, 2001). Difficulty parameters reflect the estimated severity level of the latent trait at which there is a transition in reporting the next higher category (e.g., reporting the behavior occurred five times across 14 days instead of zero to four times); the average of these parameters is also reported for each item as well as total test information assessed by the separation and social anxiety and depression items. For the difficulty parameter, 1.65 is the t-statistic that reflects the 80^th percentile, a commonly used index of clinical severity (Achenbach & Rescorla, 2000). Next, we examined whether model fit varied based on child age and sex to test whether item parameters might differ based on these demographic characteristics. These analyses relied on tests of measurement invariance using multiple group modeling. Finally, separation anxiety, social anxiety, and depression factor scores estimated from IRT calibration (averaged across 25 imputations) were extracted from the model; correlations between factor scores, demographic characteristics (child age, sex, race/ethnicity, average number of hours per week in preschool/daycare, and parental income, employment status, and education), and impairment were computed. To test for potential nonlinear associations between daily behavior frequency counts and daily impairment, graphical plots and spline models were examined for possible points of the distribution with changing magnitudes of the associations.

Results

Frequency distribution of each diary item.

The frequencies of each separation anxiety, social anxiety, and depressive behavior summed across the 14-day diary period are reported in Table 2. These frequencies correspond to percentile ranges such that children within each percentile group were reported by parents as demonstrating the behavior with the range of frequencies. For example, for the item “distress when anticipating separation from a caregiver”, children falling in the category up to the 54^th percentile were reported to demonstrate this behavior zero times across the 14-day diary period; youth between the 55^th and 72^nd percentiles demonstrated this behavior one to three times; children between the 73^rd and 95^th percentiles demonstrated the behavior four to nine times; and children who fell in the category reflecting more than the 95^th percentile demonstrated the behavior 10 or more times in the two-week period (i.e., nearly daily). As another example, for the sadness item, children falling in the category up to the 56^th percentile were reported to demonstrate sadness zero to nine across the 14-day diary period; youth between the 56^th and 82^nd percentiles demonstrated this behavior 10 to 20 times; children between the 83rd^rd and 96^th percentiles demonstrated the behavior 21 to 34 times; and children who fell in the category reflecting more than the 96^th percentile demonstrated the behavior 35 or more times in the two-week period (i.e., more than 2.5 times per day).

Dimensionality of diary items.

CFAs were conducted to test dimensionality of the separation and social anxiety and depressive behaviors and the fit of each one factor model (the range of values identified across imputations is available in the OSF repository: https://osf.io/q5ucx/?view_only=05c26787cb6e4f7cb24a7a46ebdbc632). Three separate models were run to permit interpretation of the parameters of item functioning. For separation anxiety, the original model with no modifications was not a good fit: χ²(20) = 134.16 (standard deviation (SD) = 1.80); CFI = .92 (SD = .002); RMSEA = .097 (SD = .001). The final model that included a residual covariance between two items (distress when anticipating separation and avoided going places without caregiver) demonstrated a good fit to the data: χ²(19) = 91.37 (SD = 2.47); CFI = .95 (SD = .002); RMSEA = .079 (SD = .001). Given the nature of these two items, it is reasonable that they would be related beyond the separation anxiety factor. Further, intraclass correlations (ICC) were estimated to evaluate the consistency in model parameters with and without the residual covariance path. For these models, the ICC was .976, showing a high degree of reliability for parameter estimates. Therefore, including this residual correlation improved model fit and did not appear to bias the results. For social anxiety, the original model with no modifications was an excellent fit to the data: χ²(19) = 0.51 (SD = .08); CFI = 1.00 (SD = .00); RMSEA= 0.00 (SD = .00).

For depression, the original model with no modifications was not a good fit to the data: χ²(54) = 540.75 (SD = 6.76); CFI = .808 (SD = .003); RMSEA = .122 (SD = .001). The final one factor model that best fit the data included residual correlations between four pairs of items theoretically expected to be associated beyond the factor: (1) sadness and tearfulness, (2) irritability and tantrums, (3) sadness and irritability, and (4) sleep and appetite. The relation of sadness and tearfulness may signify that young children often have difficulty regulating their sadness and could display sadness more externally as tearfulness in comparison to older children who may conceal their feelings and avoid becoming tearful (Cole et al., 1994; Southam-Gerow & Kendall, 2002). Similarly, irritability may reflect the underlying distress/mood dysregulation that contributes to outward displays of tantrums (Carlson et al., 2016; Copeland et al., 2015; Giesbrecht et al., 2010; Potegal & Davidson, 2003). The pairing of sadness and irritability is consistent with the option for sad or irritable mood to meet the mood criterion for depression in children and adolescents (Diagnostic and statistical manual of mental disorders (5th ed.), 2013); in addition, sadness and irritability may be especially sensitive indicators of depression in young children (Luby, Heffelfinger, et al., 2003). Finally, with regard to the pairing of sleep problems and appetite changes, we are not aware of research specifically linking these depressive behaviors in particular in children beyond the factor of depression. Both sleep and appetite difficulties are neurovegetative aspects of depression, and may reflect disruptions in circadian rhythms (Germain & Kupfer, 2008). There is evidence of associations between various biomarkers, sleep, and appetite (Caroleo et al., 2019; Lin et al., 2020) and links between inflammation and neurovegetative depressive symptoms in older youth and adults (Chu et al., 2019; Duivis et al., 2013; Jokela et al., 2016). Statistically, this model yielded a good fit to the data: χ²(50)=282.34 (SD = 3.94), CFI = .909 (SD = .002); RMSEA = .087 (SD = .001). For the depression models with and without the residual covariances, the ICC was .955, showing a high degree of reliability for parameter estimates. Overall, the separation and social anxiety and depressive items assessed daily in the present study appear to reflect a coherent underlying dimension.

Item characteristics.

We used GRMs to evaluate item functioning. Table 3 displays item discrimination values, difficulty parameters, and average difficulty parameters for each item. The discrimination values exceeded 1.00 for all but two items, indicating that most items provided acceptable amounts of information about the latent trait. Difficulty parameters indicate the extent to which the items assess range of severity; four separation anxiety items and four depression items had average difficulty parameters greater than 1.65, indicating that these items assessed information at high levels of severity (> 95^th percentile). Figure 1a–c displays the test information curves, which reveals the extent to which items provided information across the underlying severity of the traits. These data show that the items provide reliable information, corresponding to an alpha of .80 from approximately .75 SD to 3 SDs above the mean for separation anxiety (Figure 1a) and from approximately .5 SD below the mean to 3.5 SDs above the mean for depression (Figure 1c). These data demonstrate that the separation anxiety and depression items assess clinical severity well. However, the information curve for social anxiety suggests that these items provided relatively less reliable (< .80) information (Figure 1b), which may have been due to the relatively smaller number of behaviors assessed for this dimension.

Table 3.

Item Parameters for Each Behavior

	Discrimination	Difficulty			Average Difficulty
	a	b1	b2	b3
		Cat. 2	Cat. 3	Cat. 4
Separation Anxiety
Distress when anticipating separation	2.23	−0.40	0.87	2.40	0.96
Worry about caregiver’s safety/return home	2.10	1.20	2.15		1.68
Worry about disaster separating child from caregiver	1.35	1.67	2.59		2.13
Avoided going places without caregiver	1.56	−0.03	1.25	2.79	1.34
Worry about being left home without caregiver	1.48	0.73	2.54		1.64
Fearful of going to sleep without caregiver	1.20	−0.23	1.45	2.70	1.31
Separation nightmares	1.26	1.19	2.56		1.88
Reported illness when separation anticipated	1.27	1.37	2.80		2.09
Social Anxiety
Shy around new people	2.76	−0.05	0.96	1.82	0.91
Shy with peers	2.35	−0.13	0.94	1.97	0.93
Shy with family members/familiar adults	1.43	0.34	1.28	2.33	1.32
Distress/withdrawal in social situations	1.38	0.02	1.08	2.55	1.22
Depression
Sadness	2.28	−0.03	1.57	3.14	1.56
Irritability	1.47	0.00	1.90	3.69	1.86
Tantrums	0.76	−0.50	2.97	6.00	2.82
Low interest/pleasure	1.66	0.49	1.05	2.29	1.28
Talked about death/suicide	0.45	2.21	4.14	6.67	4.34
Low self-worth	1.11	0.10	1.41	3.25	1.59
Fatigue	1.30	0.81	1.45	2.86	1.71
Appetite and/or weight changes	1.31	0.36	1.57	2.91	1.61
Sleep difficulties	1.03	−1.25	1.51	3.26	1.17
Change in activity level	1.69	−0.16	1.11	2.39	1.11
Difficulty concentrating/making decisions	1.09	0.47	1.44	2.99	1.63
Tearfulness/sensitivity	2.03	−0.06	1.54	3.02	1.50

Note: Cat. = Category. Item thresholds (difficulty parameters) indicating severity greater than the 95^th percentile (i.e., t (608) > 1.65) are displayed in bold. Difficulty parameters reflect the level of severity required to endorse a specific item at a given response level/category. See Table 2 for frequency categories. Difficulty parameter b1 corresponds to Category 2, b2 to Category 3, and b3 to Category 4. For example, sadness had to occur 35 or more times over 14 days for the behavior to be considered psychometrically severe/rare, whereas worry about a disaster separating the child from the caregiver had to occur once over the 14 days for the behavior to be considered severe.

Figure 1a.

Test Information Curves for Latent Separation Anxiety Dimension

Total Information

Information = 5 ~ alpha = .80

Figure 1c.

Test Information Curves for Latent Depression Dimensions

Figure 1b.

Test Information Curves for Latent Social Anxiety Dimension

Table 4 displays each item’s location on the severity continuum of separation anxiety (4a), social anxiety (4b), and depressive (4c) behaviors based on average difficulty parameters. The severity continuum ranks each behavior according to its average difficulty parameter such that items at the top of the continuum suggest more problematic behavior than items at the bottom of the continuum. The four separation anxiety items (worry about disaster separating child from caregiver; reported illness when separation anticipated; separation nightmares; worry about caregiver’s safety) and the four depression items (talked about death/suicide, tantrums, irritability, fatigue) that fell above the 95^th percentile reflect the more potentially problematic markers of separation anxiety and depressive behaviors, respectively. No social anxiety items fell about the 95^th percentile, suggesting that these behaviors may be less problematic and/or may be more developmentally typical than other behaviors.

Table 4a.

Severity continuum of separation anxiety behaviors

Average Difficulty Parameter	Separation Anxiety Behaviors
2.13	Worry about disaster separating child from caregiver
2.09	Reported illness when separation anticipated
1.88	Separation nightmares
1.68	Worry about caregiver’s safety/return home
----------------------------------------------95th percentile-----------------------------------------
1.64	Worry about being left home without caregiver
1.34	Avoided going places without caregiver
1.31	Fearful of going to sleep without caregiver
0.96	Distress when anticipating separation

Table 4b.

Severity continuum of social anxiety behaviors

Average Difficulty Parameter	Social Anxiety Behaviors
----------------------------------------------95th percentile-----------------------------------------
1.32	Shy with family members/familiar adults
1.22	Distress/withdrawal in social situations
0.93	Shy with peers
0.91	Shy around new people

Table 4c.

Severity continuum of depressive behaviors

Average Difficulty Parameter	Depressive Behaviors
4.34	Talked about death/suicide
2.82	Tantrums
1.86	Irritability
1.71	Fatigue
----------------------------------------------95th percentile-----------------------------------------
1.63	Difficulty concentrating/making decisions
1.61	Appetite and/or weight changes
1.59	Low self-worth
1.56	Sadness
1.50	Tearfulness/sensitivity
1.28	Low interest/pleasure
1.17	Sleep difficulties
1.11	Change in activity level

Difficulty parameters also inform which response options reflect psychometric severity/rarity at or greater than the 95^th percentile on levels of each behavior (Table 3; see Table 2 for specific frequencies). For example, with regard to separation anxiety, distress when anticipating separation had to occur 10 more times (slightly less than once per day) over 14 days to be considered psychometrically severe/rare; the child feeling afraid to go to sleep unless they were near the caregiver had to occur daily to be considered severe/rare. With regard to social anxiety, shyness around new people and shyness with peers had to occur at least seven times over 14 days (every other day or more) to be considered severe/rare. With regard to depressive behaviors, sadness had to occur at least 35 times over 14 days (approximately 2.5 times per day) for the behavior to be considered severe/rare; irritability had to occur at least 27–43 times over 14 days (2–3 times per day) for the behavior to be considered severe/rare; and tantrums had to occur at least 12–26 times over 14 days (approximately 1–2 times per day) for the behavior to be considered severe/rare.

Other behaviors occurred less frequently to be considered severe. For example, the child worrying that the caregiver would be hurt or leave home and not return had to occur two or more times over 14 days to be considered severe/rare; shyness with family members and/or familiar adults had to occur four times over 14 days to be considered severe/rare. Talking about death and/or suicide only had to occur at least once in the 14 days to be considered severe/rare.

Item characteristics by demographic variables.

We examined model fit by child age group (3, 4, and 5 years old) and sex to test whether item parameters might differ based on these demographic characteristics. Tests of measurement invariance, based on average fit across the multiply imputed datasets, are presented in Table 5. We found no differences in model fit across age group or sex. These results suggest there is no evidence that the item parameters we identified differ for these subgroups.

Table 5.

Tests of Measurement Invariance Across Youth Age and Sex

Outcome	Group	Model	χ2	χ2 SD	df	CFI	CFI SD	RMSEA	RMSEA SD	ΔCFI	ΔRMSEA
Separation	Age	Configural	129.045	5.110	57	0.953	0.003	0.079	0.003
Separation	Age	Metric	144.065	4.428	71	0.952	0.003	0.071	0.002	−0.001	−0.008
Separation	Age	Scalar	171.671	4.330	91	0.947	0.003	0.066	0.002	−0.005	−0.005
Social	Age	Configural	4.432	0.178	6	1.000	0.000	0.000	0.000
Social	Age	Metric	20.367	0.494	12	0.993	0.000	0.059	0.002	−0.007	0.059
Social	Age	Scalar	44.415	0.545	26	0.984	0.001	0.059	0.001	−0.009	0.000
Depression	Age	Configural	378.817	3.802	150	0.913	0.002	0.087	0.001
Depression	Age	Metric	420.469	4.163	172	0.906	0.002	0.084	0.001	−0.007	−0.003
Depression	Age	Scalar	436.208	4.152	218	0.917	0.002	0.070	0.001	0.011	−0.014
Separation	Sex	Configural	108.711	2.900	38	0.951	0.002	0.078	0.002
Separation	Sex	Metric	114.038	3.173	45	0.953	0.002	0.071	0.002	0.002	−0.007
Separation	Sex	Scalar	116.711	3.571	55	0.958	0.002	0.061	0.002	0.005	−0.01
Social	Sex	Configural	6.154	0.210	4	0.998	0.000	0.042	0.002
Social	Sex	Metric	8.651	0.391	7	0.998	0.001	0.028	0.003	0.000	−0.014
Social	Sex	Scalar	16.333	0.397	14	0.998	0.000	0.023	0.002	0.000	−0.005
Depression	Sex	Configural	319.615	4.081	100	0.914	0.002	0.085	0.001
Depression	Sex	Metric	342.680	3.926	111	0.909	0.002	0.083	0.001	−0.005	−0.002
Depression	Sex	Scalar	332.223	3.674	134	0.922	0.002	0.070	0.001	0.013	−0.013

Model fit values are averaged across 25 imputation runs. SD indicates the variability in fit estimates across imputations. Change in fit reflected by Δ compares average estimate for the metric invariant model to the configural invariant model and the scalar invariant model to the metric invariant model.

Associations between diary items and demographic variables.

We extracted factor scores for separation anxiety, social anxiety, and depressive behaviors, and examined zero-order correlations with demographic variables (Table 6). Most demographic variables were not significantly related to factor scores with several exceptions. First, children whose primary caregiver worked outside the home had higher separation anxiety factor scores. Second, younger children and children with at least one parent who graduated college had higher social anxiety factor scores. Finally, using a one-way analysis of variance (ANOVA), children’s race/ethnicity was significantly related to depression factor scores, F(4, 604) = 2.67, p = .03. A post-hoc LSD test showed that white children had higher mean depression scores (M = .09, SD = .85) compared to Black/African American children (M = −.25, SD = .93; p = .01) and multi-ethnic children (M = −.11, SD = .84; p = .03). There were no other significant differences.

Table 6.

Pearson correlations between anxiety and depression factor scores and demographic variables

	Factor Scores Extracted from Diary Items
	Separation anxiety	Social anxiety	Depression
Demographic variables
Child age	.00	−.09*	.03
Child sexa	.06	.05	−.02
Time in preschool/daycare per week	.06	−.01	−.03
Parents’ marital statusb	−.05	.06	.07
Family income	−.08	.07	.03
Primary caregiver/parent employedc	.12**	−.02	−.04
Second caregiver/parent employedc	−.06	.00	−.02
Parents’ level of educationd	−.01	.13**	.06

Note. Pearson correlation coefficients were run using pairwise deletion.

^a0=male; 1=female;

^b0=not/never married; 1=married or living together;

^c0=does not work outside the home; 1=works outside the home;

^d0=neither parent graduated college; 1=at least one parent graduated college

^*p < .05;

^**p < .01

Associations between diary items and impairment.

We also examined zero-order correlations between the separation and social anxiety and depressive factors scores extracted from the daily diary and the average levels of daily impairment/distress (Table 7). Most factor scores were significantly correlated with most impairment ratings. The magnitudes of correlations were largest within domains (e.g., depression factor scores and depression impairment) versus across domains (e.g., depression factor scores and social anxiety impairment). All associations remained unchanged when including significant demographic covariates in the relevant models. Finally, we examined graphical plots and spline models of the associations between daily behavior frequency counts and daily impairment; we did not find any clear points of the distribution where there were changing magnitudes of the association between behavior frequency and impairment, suggesting a linear association.

Table 7.

Pearson correlations between anxiety and depression factor scores and daily-rated impairment

	Factor Scores Extracted from Diary Items
	Separation anxiety	Social anxiety	Depression
Average Daily-Rated Impairment
Separation Anxiety - Problematic	.70 ***	.28***	.30***
Separation Anxiety - Overall	.56 ***	.18***	.28***
Social Anxiety - Problematic	.37***	.66 ***	.35***
Social Anxiety - Overall	.39***	.55 ***	.36***
Depression – Problematic	.39***	.28***	.64 ***
Depression – Overall	.27***	.20***	.51 ***
Parenting Difficulties	.23***	.19***	.42***
Child Distress/Impairment	.27***	.25***	.43***
Parent-Child Relationship Distress	.06	.05	.18***
Overall Distress/Impairment	.23***	.20***	.42***

Note. Pearson correlation coefficients were run using pairwise deletion. Items within domains (e.g., separation anxiety behavior and separation anxiety-related impairment) are bolded.

Problematic impairment items refer to the extent to which each behavior within that domain caused distress/impairment each day.

Overall impairment for each domain refers to the extent to which all the behaviors within that domain caused distress/impairment each day.

Parenting stress, parent-child relationship distress, and overall impairment refer to distress/impairment overall each day (i.e., not in relation to a specific behavior/domain).

^***p<.001

Discussion

The present study assessed the frequency and psychometric severity of parent-reported separation anxiety, social anxiety, and depressive behaviors in a large, diverse community sample of 3–5-year-old children. We used a 14-day daily diary design that reduced parental retrospective recall and eliminated the need for parents’ (and researchers’) subjective judgment in evaluating the relative frequencies of children’s behavior. These data expand on the very limited literature in this area to provide more precise estimates of specific levels of internalizing behaviors that may indicate increased risk. Further, this approach yields empirically based information to refine assessment practices of child practitioners (e.g., pediatricians) who gauge the extent to which young children’s behaviors reflect developmentally normative patterns versus behaviors that may indicate present or impending clinically significant emotional and behavioral difficulties. Such data can improve identification of clinically concerning internalizing behavior without inadvertently pathologizing normative development.

The CFAs to evaluate the separation anxiety, social anxiety, and depression models indicated good fit for each model, suggesting that anxiety and depressive behaviors assessed daily reflected a uniform underlying dimension for each set of behaviors. Similar to our pilot work (Bufferd et al., 2017), discrimination values exceeded 1.00 for all but two items: tantrums and talking about death/suicide. Therefore, lower discrimination values identified across these two independent samples (N = 900 total) for these items suggest that tantrums and talking about death/suicide provide more limited information about the overall level of depressive behaviors compared to other behaviors. For anxiety behaviors, unlike our pilot work in which the discrimination value fell below 1.00 for distress when anticipating separation (Bufferd et al., 2019), we found that all anxiety items assessed in the present study exceeded this value. Overall, most anxiety and depression items assessed provided acceptable amounts of information about the latent traits.

Average difficulty parameters exceeded the threshold of 1.65 for four separation anxiety items (worry about caregiver’s safety/return home; worry about disaster separating child from caregiver; separation nightmares; reported illness when separation anticipated) and four depression items (irritability, tantrums, talked about death/suicide, fatigue), suggesting that these items best assessed information at higher levels of severity. Therefore, although these eight items may provide less information within the less severe/more normative range of traits, these items may also offer clinicians more information when evaluating higher levels of severity of behaviors relevant to separation anxiety and depression in preschool-aged children. Conversely, no social anxiety items fell above the 95^th percentile on the severity continuum, suggesting that these behaviors (e.g., shyness with peers and new people) may be less indicative of severity and/or may be more developmentally typical during the preschool period than other behaviors identified.

The difficulty parameters revealed the specific frequencies of behaviors that were considered psychometrically severe for each anxiety and depressive item assessed in the diary. For separation anxiety, distress when anticipating separation had to occur 10 times across the 14 days (just under once per day or more) and the child feeling afraid to go to sleep unless they were near the caregiver had to occur daily to be considered psychometrically severe/rare. The child avoiding going places without the caregiver occurred seven or more times (approximately every other day or more) to be considered severe. Other separation anxiety behaviors occurred less frequently to be considered severe; for example, reported illness when separation was anticipated had to occur at least four times and separation nightmares had to occur three times over the 14 days to be considered psychometrically severe/rare. For social anxiety, most behaviors (shy around new people, shy with peers, distress/withdrawal in social situations) had to occur approximately every other day to be considered severe, whereas shyness with family members/familiar adults had to occur at least four times or more over 14 days to be considered psychometrically severe/rare. In contrast, in a smaller sample assessed in our pilot work, shyness around new people and with peers were not severe at any frequency (Bufferd et al., 2019).

For depressive behaviors, sadness had to occur at least 35 times over 14 days (approximately 2.5 times per day or more) and tearfulness/sensitivity at least 34 times (2.5 times per day or more) for the behaviors to be considered severe/rare; irritability had to occur at least 27–43 times (2–3 times per day or more), and tantrums had to occur at least 12–26 times (approximately 1–2 times per day or more) over 14 days for the behavior to be considered severe/rare. The majority of the remaining depressive behaviors had to occur around five to ten times over the 14-day diary period (see Table 3) for the behaviors to be severe. Finally, talking about death and/or suicide only had to occur at least once in the 14 days to be considered severe/rare. This pattern of findings was consistent with our pilot study (Bufferd et al., 2017, 2019) and provide further empirical evidence that sadness, tearfulness/sensitivity, irritability, and tantrums are normative during the preschool developmental period. These data also overlap with Wakschlag and colleagues’ findings on irritability and temper tantrums/temper loss in identifying, for example, that tantrums occurring once or more per day are considered psychometrically severe (Wakschlag et al., 2012; Wakschlag et al., 2015).

Taken together, the results from the item analyses support our hypotheses that behaviors theorized to be more developmentally typical/higher base rate occurred at greater frequencies to be considered severe in relation to less common behaviors that occurred at lower frequencies to be considered severe. Although the behaviors identified as more common reflect existing developmental knowledge, the data in the present study empirically delineate specific frequencies of behaviors that may be clinically meaningful, and the overall pattern of findings is generally consistent with our pilot study with a separate sample. These data inform assessment by highlighting specific frequencies of internalizing behaviors that may be of concern and prioritizing particular behaviors when assessing risk, while also suggesting that behaviors occurring at frequencies below these thresholds may be less concerning.

We found that model fit did not vary by child age (3, 4, 5 years old) or sex (female and male) (these analyses were not conducted in the pilot study given the smaller sample size). Therefore, there was no evidence that anxiety or depressive behavior item characteristics varied by these demographic subgroups. We had expected to see differences in item parameters by age given that emotion regulation and self-control increase, on average, as children age through this developmental period (Eisenberg et al., 2010; Eisenberg & Sulik, 2012; Fox & Calkins, 2003; Gagne, 2017); therefore, we had hypothesized that behaviors reflecting more dysregulation (e.g., irritability, tantrums) would occur at lower frequencies to be considered severe/rare in older children compared to younger children in which these behaviors may be more common. However, behaviors assessed on a daily basis in the present study did not reflect these potential developmental changes within this period. We did not have a specific hypothesis regarding possible sex differences in item parameters; like children’s age, the models did not vary for girls and boys. The lack of sex differences is consistent with preschool psychopathology research in general in which sex differences in internalizing disorders are typically not identified in this age group (Bufferd et al., 2011; Egger & Angold, 2006), although emotional reactivity may differ for depressed and at risk boys versus girls (Luby, Essex, et al., 2009). This lack of differences identified in the present study with regard to children’s age and sex suggests that the frequencies of psychometric severity of anxiety and depressive behavior can be applied similarly across the preschool period and amongst girls and boys.

Severity/frequency of anxiety and depressive behaviors (as indexed by factor scores extracted from each model) were not related to most demographic variables with some exceptions. First, children whose primary caregiver worked outside the home had higher separation anxiety factor scores. There is evidence to the contrary in that researchers found an association between parental unemployment and increasing separation anxiety symptoms over the preschool period (Battaglia et al., 2016); this finding may reflect a family stressor more than a specific feature of parental employment. It is possible that children of parents who do and do not work outside the home have similar underlying levels of separation anxiety, but perhaps for children of parents who do work outside the home, there are more opportunities for separation from parents and, therefore, separation anxiety in children, to be observed. Second, younger children had higher social anxiety factor scores. This finding is consistent with our pilot work and studies in older children that identify that social anxiety and inhibition decrease over time, on average (Degnan & Fox, 2007). We would expect to see the same pattern with regard to separation anxiety (Costello et al., 2003), but this decrease in separation anxiety may be more likely to occur when children enter formal schooling, which had not yet occurred in our sample of 3–5-year-old children. In addition, children with at least one parent who graduated college had higher social anxiety factor scores compared to children whose parent(s) did not graduate college. We are not aware of research identifying associations between parental educational attainment and social anxiety in preschool-aged children. It is possible that parents with more education may be more prone to engage in over-involved or intrusive parenting, which is associated with higher levels of anxiety in children (Bayer et al., 2019; McLeod et al., 2007). Finally, white children had higher mean depression factor scores compared to both Black/African American children and multi-ethnic children. The research in this area has been conducted primarily with school-aged children and adolescents, and the findings with regard to depression and race/ethnicity in children have been mixed (Anderson & Mayes, 2010). In a study that examined depression in a large community sample of preschool-aged children, there were no racial/ethnic differences in rates of depression (Lavigne et al., 2009).

As hypothesized and consistent with the pilot study, severity/frequency of anxiety and depressive behaviors were associated with daily impairment, including children’s and parents’ distress, and children’s difficulties in psychosocial functioning and relationships. These findings are consistent with other studies that identify links between anxiety or depression and impairment in young children (Bufferd et al., 2011; Egger & Angold, 2006; Luby, Belden, et al., 2009), but the magnitude of associations between internalizing behavior and impairment in the present daily dairy study (particularly within each domain) are larger than previous work that utilized other methods (e.g., diagnostic interviews or subjective frequency ratings); these findings suggest that assessing the impact of children’s internalizing behavior on a daily basis may yield stronger associations between behavior and impairment than assessments that require longer time frames of retrospection. In addition, consistent with the pilot data, we found that associations between depression factor scores and daily child distress/impairment, parent-child relationship distress, and overall distress/impairment were approximately twice as large as associations between separation and social anxiety factor scores and these impairment variables. These data suggest that depressive behaviors (e.g., irritability, sadness, tearfulness, tantrums) may be more distressing and impairing for preschool-aged children and their families day-to-day compared to anxiety behaviors from the primary caregiver’s perspective. Overall, these data suggest that internalizing behaviors assessed on a daily basis are associated with distress and/or impairment in children’s and parent’s functioning, adding to the data on the frequency and severity of behaviors assessed in this study.

Limitations

The findings in this study should be interpreted with consideration of several limitations. First, the study relied on the use of single informant. In general, multiple informants could each provide useful and valid information about a child’s behavior. Given the focus on estimating precise frequency ratings for behaviors in the present study, relying on the primary caregivers’ reports was likely most informative (and feasible). Nevertheless, ratings from other caregivers/parents and childcare providers may be useful as well in informing data on frequency and severity of behaviors and associations with impairment. Second, some of the demographic characteristics of the sample may limit generalizability of the findings; for example, most parents in the sample were married or living together and about half of the parents graduated college. Therefore, the item parameters identified may not be as generalizable to populations with different demographic characteristics. Similarly, although approximately 45% of the children in the sample were identified as racial/ethnic minoritized individuals, there was insufficient statistical power to test potential differences in item parameters within specific race/ethnic groups. Although comparisons among children of different race/ethnicities would require operational definitions of these socially constructed categories (Helms et al., 2005; Kawachi et al., 2005; Sue, 1999), the literature could likely benefit from studies with larger samples of children of various racial/ethnic backgrounds as well as the inclusion of sociocultural variables that may impact children’s behavior and parents’ report of impairment and distress (e.g., discrimination).

Third, the present study focused on assessment of specific frequencies of behavior to begin to characterize the spectrum of normative to problematic internalizing behavior and quantify severity; however, knowledge of frequencies of behaviors without consideration of context is insufficient for predicting clinical risk. Additional assessment of children’s contexts (e.g., parenting behavior) is needed to further refine knowledge of the extent to which particular levels of behavior may be problematic. Other parameters of behavior such as duration, intensity, and setting, constellation of overlapping behaviors, the extent of dysfunction associated with behavior, and parental and/or societal construction of the meaning of children’s behaviors can influence the manner in which risk for or presence of psychopathology can be defined (Wakefield, 1992). Further, as the particular phrasing of items may influence parameters generated by IRT (and therefore the replicability of findings), different phrasing than used in the present study may yield somewhat different parameter estimates (Lane et al., 2016). Fourth, although our presentation of the frequency distributions provides clear context for the range of behaviors presented by young children, identification of the most appropriate number and locations of separations between categories is somewhat arbitrary. We aimed to optimize the translation of the frequencies of behaviors but testing alternative boundaries may be useful. Fifth, the time children spent in preschool/daycare could influence parental report of behavior each day. In the present sample, children were in preschool/daycare for an average of 22.5 hours per week. Although parents were told they should ask childcare providers how often behaviors occurred each day, and time spent in preschool/daycare was not associated with the behaviors assessed in this study, any time children spent out of parents’ care could impact the frequency of behaviors reported. Finally, diary items were selected from validated measures of child psychopathology and linked to DSM symptoms for separation and social anxiety and depression; other behaviors relevant to internalizing psychopathology may be important to assess.

Conclusions and Future Directions

The present study utilized a parent-report daily diary to assess the full spectrum of separation and social anxiety and depressive behaviors in a large, diverse community sample of preschool-aged children. The diary method reduced retrospective recall bias and informant and researcher subjectivity by eliciting open-ended information about frequencies of behavior on a daily basis. These data provide empirically derived parameters of the specific frequencies of anxiety and depressive behaviors that may be more severe. Further, we identified a continuum of severity of behaviors to further characterize the range of developmentally normative to clinically significant behavior. Such information is essential for refining screening and assessment practices during preschool age, a period in which 1) assessments are particularly challenging given that many developmentally normative behaviors overlap with symptoms of psychopathology (Bufferd et al., 2016); 2) accurate assessment is critical as earlier onset of internalizing difficulties may contribute to longer-term impairment and chronicity compared to later onset difficulties (Weissman et al., 1999); 3) intervention may be particularly effective due to neurodevelopmental and behavioral plasticity in early childhood (Hirshfeld-Becker & Biederman, 2002); and 4) there is a developmental window to prevent worsening anxiety and depressive symptoms prior to formal school entry using empirically-supported approaches (Luby, 2013; Luby et al., 2018). The data generated can provide child practitioners with empirically based norms or benchmarks to address parents’ concerns about the extent to which their preschool-aged children’s internalizing behaviors fall in the range of developmentally typical behavior versus those that may be more severe.

Longitudinal follow-up is an essential next step to evaluate whether the specific frequencies of behaviors identified as psychometrically severe in the present study predict future emotional and behavioral difficulties and impairment. Second, cross-sectional and longitudinal investigation of other parameters of young children’s’ daily behavior (e.g., duration, intensity, and setting of behavior; distress/impairment of each behavior for the child and family) must be investigated in addition to frequency to enhance the validity of the information obtained. Third, analysis of the specific day-to-day variability of the more common internalizing behaviors can further inform assessment efforts. In our pilot work, we have identified associations between daily affective (sadness) emotion dynamics, including instability and inertia, and symptoms and impairment assessed two years later (Chad-Friedman et al., in press). We also identified between- and within-person sadness and irritability each predicting the other the following day (Leppert et al., 2019). We will aim to replicate these investigations in this larger independent sample as well as examine patterns among daily anxiety patterns. Finally, it is also critical to examine whether known correlates of psychopathology identified across informants and methods (e.g., laboratory-observed behavior; physiology; parental psychopathology; familial stressors) correspond to the frequencies of behavior identified in the present study to further validate these findings and continue to refine screening and assessment efforts of internalizing behavior in preschool-aged children.