Dimensions of Depressive Symptoms in Young Children: Factor Analysis of the Preschool Feelings Checklist–Scale

Abstract

The current study is an investigation of the dimensionality of the Preschool Feelings Checklist–Scale (PFC-S), a caregiver-report questionnaire of early childhood depressive symptom severity. Caregivers of 450 young children, ages 3–8 years (M = 5.62, SD = 0.95; 49% Female; 7% Hispanic; 66% white), completed the PFC-S and questionnaires on child emotion regulation and expression and self-reported depressive symptomatology. Confirmatory factor analyses indicated that a one-factor structure did not adequately fit the current PFC-S data. Using exploratory factor analysis, a three-factor structure emerged as interpretable and structurally sound, yielding reliable factors related to Social and Behavioral Anhedonia, Emotional and Behavioral Dysregulation, and Excessive Guilt and Sadness. This factor structure showed configural and scalar invariance across preschool- and early middle childhood aged children as well as children assigned male and female sex at birth. Correlations between the three factors and constructs related to depression suggested preliminary construct validity. The current study provides initial evidence for a multidimensional structure of the PFC-S and improves our understanding of early childhood depressive symptoms.

Elevated depressive symptoms in early childhood¹ increase the risk for the development of not only depression, but also anxiety, attention-deficit/hyperactivity disorder, disruptive behavior, and general functional impairments (Luby, Heffelfinger, et al., 2004; Luby, Belden, et al., 2009; Luby, Si, et al., 2009; Luby et al., 2014; Gaffrey, Tillman, et al., 2018; Silver, Barch, et al., 2021; Silver, Olino, Carlson, et al., 2021; Silver, Olino, Luby, et al., 2021). Recent work indicates that treatments for depressive symptoms in preschoolers also improve anxious and externalizing symptoms, emotion regulation, executive functioning, and functional impairment (Luby et al., 2012; Luby et al., 2018; Luby et al., 2020). From late middle childhood (i.e., ages 10 and up) to adulthood, it has been suggested that depressive symptoms can be studied at different levels of specificity, ranging from a more general unitary level to more specific subdimensions that are present in several mental health disorders (Cervin et al., 2023; Nelson et al., 2017; Klein, 2023). The idea that the measurement of one mental health disorder results in the identification of subdimensions relevant across various disorders is the basis of novel conceptualizations of psychopathology (Kotov et al., 2017). Together, these lines of research highlight the heterogeneous nature of depressive symptoms and motivate the need for a deeper understanding of the structure of early emerging depressive symptoms, some of which may uniquely index depression and others that may indicate broader forms of psychopathology. Nevertheless, there are no published studies to date that systematically investigate this question, creating a critical gap in the literature. As a first step in filling this gap, the current study investigated the factor structure of one of the most frequently used research measures of early childhood depressive symptom severity, the Preschool Feelings Checklist–Scale (PFC-S; Luby et al., 1999; Luby, Heffelfinger, et al., 2004; Luby et al., 2012; Luby et al., 2018; Gaffrey, Barch, et al., 2018; Fowler & Gaffrey, 2022; Heaton et al., 2023).

The PFC-S is a caregiver-report questionnaire that is frequently used to obtain a dimensional measure of depressive symptoms in young children, including commonly co-occurring attentional and functional problems. Even though the PFC-S is named the Preschool Feelings Checklist–Scale, many previous studies have used the measure to assess depressive symptoms in male and female children as old as 8 years because continuity of depressive symptoms across this age range remains of particular interest in existing literature (Luby et al., 2012; Luby et al., 2018; Gaffrey, Tillman, et al., 2018; Gaffrey, Barch, et al., 2018; Fowler et al., 2021; Fowler & Gaffrey, 2022; Heaton et al., 2023). The PFC-S has several beneficial properties, which helps explain its regular use in studies of childhood depression. First, PFC-S items were constructed to capture manifestations of depressive symptoms using highly precise, developmentally sensitive descriptions appropriate for younger children (Klein et al., 2005; Luby, Mrakotsky, et al., 2003). For example, an item intended to assess the severity of a preoccupation with death is framed within the context of play (i.e., “Pretend plays about scary or sad things”). Second, each item is rated using a 5-point scale, aligning with dimensional conceptualizations of depression (Fried et al., 2022) and allowing caregivers to report subtle variations in symptom frequency and severity. Third, the PFC-S has strong internal consistency (Luby, Heffelfinger, et al., 2004; Silver, Olino, Luby, et al., 2021; Luby et al., 2012) and test-retest reliability (Luby et al., 2012). As a result, the PFC-S provides a unique and comprehensive measure of depressive symptoms in young children that is not available from measures covering a wider age range (e.g., Child Symptom Inventory-4; Gadow & Sprafkin, 2002), broader spectra of psychopathology (e.g., Child Behavior Checklist, Ivanova et al., 2010; McArthur Health and Behavior Questionnaire, Ablow et al., 1999), or providing a brief clinical screen (e.g., PROMIS Emotional Distress, Sherlock et al., 2022).

The PFC-S is typically scored by deriving the mean or sum of individual item scores. The same scoring process is used for children assigned male and female sex at birth and across the preschool and early middle childhood periods. This scoring approach has largely been justified by demonstrating high internal consistency (Cronbach’s α) across the set of PFC-S items. This approach to scoring reflects two major untested assumptions: first, it assumes that the entire set of associations among PFC-S items is accurately captured by a single latent factor (McNeish & Wolf, 2020), and second, it assumes that the measure is invariant across age and sex (Putnick & Bornstein, 2016). The first assumption is questioned by prior studies noting the heterogeneous nature of depression (Fried et al., 2022) and by recent evidence showing that PFC-S scores are correlated with psychopathology beyond depression alone (Heaton et al., 2023). In addition, solely relying on Cronbach’s α to provide evidence for a measure’s unidimensionality is an erroneous interpretation of the statistic (McNeish, 2018). Similarly, the second assumption (of age and sex invariance) is questioned by work suggesting that children’s depressive symptoms may (Bufferd et al., 2017; Angold et al., 1991) or may not (Luby, Heffelfinger, et al., 2003) differ by child age and sex assigned at birth. To better inform the way in which the PFC-S should be used, more direct tests of the measure’s dimensionality, reliability, and measurement invariance are now needed (Putnick & Bornstein, 2016; McNeish, 2018; Widaman & Revelle, 2023).

Importantly, while establishing and validating the internal factor structure, measurement invariance, and reliability of the PFC-S has implications for the pragmatic use of the measure (i.e., potential scoring and use across subgroups), it is also highly likely to inform our developmental understanding of depression. More specifically, characterizing the factor structure of the PFC-S has the potential to uncover measurable endophenotypes and transdiagnostic features of early childhood depressive symptoms. In turn, this could facilitate the discovery of more precise mechanisms that may be amenable to treatment in early life and related neurobiological correlates (Chu, Temkin, & Toffey, 2016; Hasler et al., 2004).

The current study had 3 primary goals: (1) investigate the factor structure and reliability of the PFC-S, (2) test invariance of the resulting factor structure across age groups and sex assigned at birth, and (3) examine the construct validity of the identified factor(s) by assessing how they correlate with established measures of child emotion expression and regulation and caregiver depressive symptomatology. The current study used a data-driven approach to identify the factor structure of the PFC-S. However, due to the heterogeneity of depressive symptoms in early childhood and the breadth of symptoms covered in the PFC-S, we a priori hypothesized that the PFC-S would be best represented by a reliable multidimensional factor structure. We specifically hypothesized that the PFC-S factor structure would be reflective of key features of depression (e.g., anhedonia) as well as others also considered to be more transdiagnostic in nature (e.g., behavioral dysregulation; Brown, 2007). Due to the breadth of depressive symptoms covered in the PFC-S, we also hypothesized that the resulting factor structure would demonstrate adequate invariance across the specified sample subgroups and that the resulting factor(s) would correlate with related, established measures in expected directions (e.g., positively correlated with negative affect).

Methods

Participants

Data used in the current study come from 3–8-year-old children and their caregivers who participated in one of three studies, conducted either at Duke University or Washington University at St. Louis. No child participated in more than one study. These studies investigated the behavioral, neural, and environmental correlates of early emerging internalizing symptomatology (Heaton, Camacho and Gaffrey, 2023; Bagdasarov et al., 2022; Fowler and Gaffrey, 2022; Fowler, Bogdan and Gaffrey, 2021; Gaffrey et al., 2021; Gaffrey, Barch, et al., 2018; Gaffrey, Tillman, et al., 2018; Gaffrey, Barch and Luby, 2016). Families were recruited from daycare centers, pediatricians’ offices, community events, parenting pages on Facebook, and university-managed recruitment databases. Child exclusionary criteria were reviewed over the phone and included neurological disorder, severe chronic illness, autism spectrum disorder, developmental delays, magnetic resonance imaging contraindications, use of psychiatric medication for mood or anxiety, or premature birth. Children were expected to be between 3 and 8 years old. Primary caregivers also completed the checklist version of the PFC to screen for the presence or absence of depressive symptoms in their children in the previous two weeks (Luby et al., 1999). Children at risk for developing depression (i.e., 3+ symptoms present) were oversampled, given the low base rate for these symptoms at this early age in the population. Prior to data collection, caregivers signed a written consent form, and children provided verbal assent. Families and children were compensated for participating. The studies were approved by the local Institutional Review Boards and conducted in accordance with the Declaration of Helsinki.

In total, 1005 children across all studies were screened. A total of 674 children met inclusion criteria and were enrolled. Behavioral assessments were completed by 467 children and their caregivers. Two participants were subsequently removed from the dataset due to an incidental finding (i.e., Chiari malformation) and a clerical error in data labeling. Fifteen out of the 465 remaining caregivers did not complete the PFC-S. This study included PFC-S data from the reporting caregivers of 450 children, ages 3–8 years (M = 5.62, SD = 0.95; 49% Female; 7% Hispanic; 66% white). See Tables 1 and 2 for complete demographics.

Table 1.

Demographic Characteristics of Child Participants

	Study 1		Study 2		Study 3		Full Sample
Child Characteristics	n	%	n	%	n	%	n	%
Sex assigned at birth
Male	55	60	52	47	124	50	231	51
Female	36	40	59	53	124	50	219	49
Race
White	65	71.4	65	58.6	168	67.8	298	66
Black	20	22	35	31.5	35	14.1	90	20
Asian	0	0	0	0	3	1.2	3	0.7
American Indian/Alaska Native	0	0	0	0	1	0.4	1	0.3
Other (unspecified)	1	1.1	0	0	9	3.6	10	2
Multi-Racial (unspecified)	5	5.5	11	9.9	32	12.9	48	11
Ethnicity
Hispanic/Latinx	5	5	3	3	25	10	3	7
Not Hispanic/Latinx	86	95	108	97	223	90	417	93
Preschool Feelings Checklist Symptom Cutoff
Less than 3 (low risk for depression)	61	67	72	65	195	79	328	73
3 or more (high risk for depression)	30	33	39	35	53	21	122	27
	M (SD)		M (SD)		M (SD)		M (SD)
Age in years	5.28 (0.87)		5.48 (0.87)		5.80 (0.97)		5.62 (0.95)
Preschool Feelings Checklist Score	2.43 (3.43)		2.26 (3.19)		1.26 (2.18)		1.74 (2.79)

Note. N = 450. The Preschool Feelings Checklist for which statistics are presented on this table refers to the screener used during study recruitment rather than the scale version.

Table 2.

Demographic Characteristics of Caregivers

	Study 1		Study 2		Study 3		Full Sample
Caregiver Characteristics	n	%	n	%	n	%	n	%
Relationship to Child
Father	4	4.4	0	0	30	12	34	7.6
Mother	85	93.4	111	100	218	88	414	92
Grandmother	1	1.1	0	0	0	0	1	0.2
Other (unspecified)	1	1.1	0	0	0	0	1	0.2
Caregiver Education (maximum)
Completed Grade School	-	-	0	0	1	0.4	1	0.2
Some High School	-	-	2	1.8	1	0.4	3	0.7
High School Diploma	-	-	5	4.5	11	4.4	16	3.6
Some College	-	-	36	32.4	54	21.8	90	20
Four-Year College Degree	-	-	30	27	77	31.1	107	23.8
Professional/Graduate	-	-	38	34.3	102	41.1	140	31
No Information	91	100	-	-	2	0.8	93	20.7
Family Income
< $30,000	-	-	32	29	-	-	32	7.1
$30 - $60,000	-	-	32	29	-	-	32	7.1
$60,000+	-	-	47	42	-	-	47	10.5
$0 - $20,000	-	-	-	-	14	5.6	14	3.1
$20 - $40,000	-	-	-	-	24	9.8	24	5.3
$40 - $60,000	-	-	-	-	23	9.3	23	5.1
$60 - $80,000	-	-	-	-	36	14.5	36	8
$80 - $100,00	-	-	-	-	36	14.5	36	8
> $100,000	-	-	-	-	108	43.5	108	24
No Information	91	100	-	-	-	-	91	20.2
Refused/Don’t Know	-	-	-	-	7	2.8	7	1.6

Note. N = 450. Caregiver education and family income was not uniformly collected across all studies.

Measures

Preschool Feelings Checklist–Scale (PFC-S)

The PFC-S contains 23 items rated by caregivers in reference to their child’s depressive symptoms (e.g., sadness, guilt, loneliness, irritability, tiredness) over the past two weeks, as well as attention problems and functional impairments (e.g., difficulty sleeping, social challenges) closely associated with depression. A 5-point scale is used (0 = Never, 1 = Rarely, 2 = Sometimes, 3 = Often, and 4 = Most of the time). Responses were treated as ordinal. Higher scores indicate increased depressive symptom severity.

Emotion Regulation Checklist (ERC)

The ERC (Shields & Cicchetti, 1997) is a caregiver-report measure assessing how children manage their emotional experiences in social settings. The ERC includes 24 items rated by caregivers on a 4-point scale (i.e., 1 = Never to 4 = Almost always). Two subscales can be calculated: Lability/Negativity (e.g., mood swings, anger proneness, overly exuberant) and Emotion Regulation (e.g., modulates excitement, transitions into activities well, responds positively to friendly overtures). The caregiver-reported subscales of the ERC show strong internal consistency in samples of young children (Elsayed et al., 2021).

Behavioral Inhibition/Activation Scale (BIS/BAS)

The BIS/BAS (Carver & White, 1994) is a 20-item questionnaire with a 7-point scale (1 = Extremely untrue to 7 = Extremely true). The scale assesses a caregiver’s perception of two motivational systems in their child, the BIS (i.e., sensitivity to potential punishment in the environment) and BAS (i.e., reward responsivity, fun-seeking behavior, and drive to pursue appetitive goals). Caregiver-reported BIS/BAS scores in a sample of young children have shown acceptable internal consistency (Blair et al., 2004).

Positive and Negative Affect Schedule – Child (PANAS)

The child PANAS (Laurent et al., 1999) is a 20-item questionnaire with a 5-point scale (i.e., 1 = Very slightly or not at all to 5 = Extremely). Caregivers considered the extent to which their child experienced positive (PA; e.g., excited, attentive, active) and negative (NA; e.g., afraid, ashamed, hostile) affect within the previous week. Caregiver reports of young children’s affect on the PANAS show strong internal consistency (Allan et al., 2013).

Beck Depression Inventory-II (BDI)

Caregivers reported on their own depressive symptomatology using the Beck Depression Inventory-II (BDI; Beck et al., 1961), a 21-item questionnaire with a 4-point scale (0 = No more than usual to 3 = All the time). Higher scores indicate greater caregiver depression symptom severity (e.g., loss of pleasure, guilt, worthlessness, sadness). The BDI in adults shows strong internal consistency (Beck et al., 1988).

Data Analysis

Mplus version 8.6 (Muthén and Muthén, 1998–2021) was used to conduct all factor analyses, test measurement invariance, and calculate factor scores. The following packages in R version 4.2.2 (RStudio Team, 2022) were used to organize and visualize data: tidyverse (Wickham et al., 2019), multiplex (Rivero Ostoic, 2020), VIM (Kowarik & Templ, 2016), see (Lüdecke, 2021), ggplot2 (Wickham, 2016), ggcorrplot (Kassambara & Patil, 2023), scales (Wickham et al., 2023), and apaTables (Stanley & Spence, 2018). Multivariate normality (MVN package; Korkmaz et al., 2014), internal consistency (psych package; Revelle, 2023), reliability (BifactorIndicesCalculator package; Dueber, 2022), and correlations (Hmisc package; Harrell, 2023) were also assessed in R.

Prior to conducting factor analyses on the PFC-S, we assessed item-level missingness (Figure S1), response frequency (Table S1), multivariate normality, and sampling adequacy (see Supplementary Material). Zero-order polychoric correlations were estimated across the PFC-S items to determine whether the factor analyses could be affected by item collinearity (Figure S2).

Factor Analyses & Reliability

Due to the ordinal, asymmetric, and multivariate non-normal nature of the PFC-S item-level data, polychoric correlations and robust diagonally weighted least squares estimation with mean and variance adjustments (WLSMV) were used in all factor analyses.

Confirmatory Factor Analysis (CFA).

A CFA was conducted to evaluate the suitability of a single-factor structure to represent the entire set of items on the PFC-S, as implied by the existing approach to scoring the measure. Before assessing model fit, a two-step iterative process was conducted to remove problematic items. First, items were flagged as problematic if their communality (h²; the proportion of item variance attributable to the latent construct) fell below 0.20 (Yong & Pearce, 2013). The item with the lowest communality was removed and the structure was recreated using the remaining items. Second, once all communalities were > 0.20, items with factor loadings (λ) below 0.30 were flagged as problematic (Rosellini & Brown, 2021). Although higher minimum factor loadings are typically recommended in scale development (i.e., λ > 0.35), a lower loading minimum may be warranted for clinical measures that include low base-rate items (Clark & Watson, 2019). Items failing to meet criteria were iteratively removed and the structure was recreated using the remaining items until all items met h² > 0.20 and λ > 0.30.

Subsequent model fit was assessed using suggested fit index cutoffs. We took a two-tiered approach to identify close and adequate fit (Hu & Bentler, 1999; Yuan et al, 2016). Close fit was determined by comparative fit (CFI) and Tucker-Lewis indices (TLI) ≥ .95, root mean square error of approximation (RMSEA) ≤ .06 with the upper 90% confidence interval (CI) ≤ .08, and standardized root mean square residual (SRMR) ≤ .08. Adequate fit was determined by CFI/TLI ≥ .90 and < .95, RMSEA > .06 and ≤ .08 with the upper 90% CI ≤ .10, and SRMR > .08 and ≤ .10. It was decided a priori that inadequate CFA fit would be followed by EFA.

Exploratory Factor Analysis (EFA).

The set of PFC-S factor structures to be explored was dictated by a parallel analysis (PA; Horn 1965). Models with the number of factors dictated by the PA and those with ±1 factor were investigated further (Lim & Jhang, 2019). The PA was conducted on principal component eigenvalues, which tends to provide better factor retention estimates than PA on factor eigenvalues when ordinal data and polychoric correlations are used (Garrido et al., 2013). The models suggested by the PA procedure were then submitted to a set of principal axis EFAs using an oblique Crawford-Ferguson Varimax rotation (Sass & Schmitt, 2010). A sensitivity analysis was conducted on the main factor model of interest using a Direct Oblimin rotation (Table S4). Prior to interpreting the factors resulting from the EFAs, the same two-step procedure used in the CFA was followed to ensure the inclusion of only salient items (i.e., h² > 0.20 and λ > 0.30). In addition, to ensure adequate factor identification, at least three “cardinal” items per factor were expected (i.e., items associated with only one factor with λ > 0.30; Bollen, 1989), and cross-loadings were allowed (i.e., items associated with two or more factors with λ’s > 0.30). The interpretability of cross-loadings influenced the justification of a model.

If the suitability of an EFA model was supported, an exploratory structural equation model (ESEM) with a Geomin rotation and an epsilon value of 0.5 (Marsh et al., 2014) was specified to recreate the original EFA model and allow for the assessment of factor reliability measures and measurement invariance.

Reliability.

Internal consistency for each subscale was measured using the average inter-item correlation (AIC; Cortina, 1993; Clark & Watson, 2019). Polychoric correlations were used given the ordinal nature of response options across subscales. Internal consistency was considered strong when .30 < AIC < .50 (Briggs & Cheek, 1986; Clark & Watson, 2019). Cronbach’s α was also calculated for each subscale (Table S5).

Additional reliability statistics were reviewed for the resulting PFC-S factor(s). We calculated the proportion of explained common variance from the entire set of items attributable to each factor (ECV; Rodriguez et al., 2016) and McDonald’s omega (ω; McNeish, 2018). These statistics were used descriptively without interpretational cutoffs (Bonifay et al., 2015). Construct reliability was estimated using statistic H for which values ≥ .70 provide evidence of a replicable factor (Rodriguez et al., 2016). Factor determinacy (FD) was calculated using values ≥ .90 as indicative of dependable factor score estimates (Grice, 2001).

Measurement Invariance

To better understand if the PFC-S demonstrates psychometric equivalence across preschool-aged children (3–5 years old, n = 294) versus children in the early middle childhood period (6–8 years old, n = 156) and, separately, across children assigned male (n = 231) versus female (n = 219) sex at birth, we assessed measurement invariance following a three-step process with adaptations for ESEMs (Widaman & Reise, 1997; Muthén and Muthén, 1998–2021). Model convergence was ensured by matching item response coverage across groups using minimal winsorization.

First, a configural invariance model was specified using WLSMV estimation, theta parameterization (Millsap & Yun-Tein, 2004), factor variance(s) fixed to one, item residual variances fixed to one, freely estimated item thresholds and factor loadings, and latent mean(s) fixed to zero (Thurstone, 1947). Adequate or close model fit of this model, as specified above, would suggest that the PFC-S demonstrates an equivalent factor structure across groups (Putnick & Bornstein, 2016). A metric model alone should not be estimated for ESEMs (Muthén and Muthén, 1998–2021). Therefore, a scalar model was specified in the second step. Factor loadings and thresholds were constrained to be equal across groups, item residual variances were fixed at one in a group and freed in the other, factor mean(s) were fixed at zero in a group and freed in the other, and factor variance(s) were fixed at one in a group and freed in the other. Factor loadings and item threshold constraints test the extent to which PFC-S items similarly contribute to the factors and factor means can be compared across groups, respectively (Putnick & Bornstein, 2016). The third and final model tests residual invariance, which involves updating the scalar model to constrain item residual variances equal (i.e., to one) across groups. This additional constraint tests the extent to which item specific and error variances are similar across groups (Putnick & Bornstein, 2016).

If the PFC-S demonstrated configural invariance, two sets of metrics were to be used together to test scalar invariance: a χ² difference test (χ²_diff) with an adjustment for nested model comparisons and WLSMV estimation (Satorra, 2000) and change between models in CFI, TLI, and RMSEA (Chen, 2007). A significant χ²_diff result would suggest that scalar invariance was not met. Rather than rely solely on a χ²-based test which tends to be overly sensitive in large sample sizes, lack of scalar invariance also had to be indicted by worsened model fit according to a decrease in CFI of .0004 and in TLI of .0006 and an increase in RMSEA of .0052. These cutoff values are more appropriate to test scalar invariance in ESEM with ordinal indicators (Jin, 2020) than traditional cutoffs (Chen, 2007). Residual invariance was tested only if the χ²_diff test indicated scalar invariance, as changes in alternative fit indices appropriate for use in ESEM with ordinal indicators have not been proposed at this level of invariance. Failure to reach invariance at one level precluded tests for stricter levels of invariance. Partial invariance was not investigated to limit researcher decision degrees of freedom in this largely exploratory study.

Construct Validation

Validation of the PFC-S factor(s) was based on Pearson’s r correlations between PFC-S factor score(s) and unit-weighted mean composite scores (i.e., mean of item scores) of related constructs (i.e., child emotion regulation, inhibition/approach behavior, positive/negative affect; caregiver depressive symptomatology). A correlation matrix using unit-weighted mean composite scores of the PFC-S factor(s) was also analyzed (Table S6). Correlations significantly different from zero that were ≤ .19 were considered small, between .20 and .29 were considered moderate, and .30+ were considered large (Hemphill, 2003; Gignac & Szodorai, 2016; Funder & Ozer, 2019).

Results

Factor Analyses & Reliability

Confirmatory Factor Analysis (CFA)

See Table 3. While all λ’s of the one-factor model were above 0.30, two CFA iterations indicated that item 10 (“Keeps to him/herself”) and item 16 (“Isn’t upset by things that other children his/her age find upsetting”) had h² < 0.20. These items were removed in this order and a final one-factor model was estimated. Although one index was at the cusp of indicating close fit (SRMR = 0.08), the one-factor model did not adequately fit the data overall (RMSEA = 0.10 [90% CI = 0.09, 0.10]; TLI = 0.87; CFI = 0.86).

Table 3.

Results from a One-Factor Confirmatory Factor Analysis of the Preschool Feelings Checklist–Scale (PFC-S)

Item	F1	h 2
1. Interested in playing with other kidsa	.50	.25
2. Appears sad or says s/he feels sad	.61	.37
3. Engages in activities that make him/her happya	.63	.40
4. Has a lot of trouble following simple directions	.68	.47
5. Seems less excited about play/activities than other kidsa	.61	.37
6. Has difficulty sleeping at night	.47	.22
7. Whines/cries	.60	.36
8. Can’t pay attention to games or tasks	.65	.44
9. Enjoys activities and playa	.65	.42
10. Keeps to him/herself	Removed
11. Pretend plays about scary/sad things	.58	.34
12. Blames himself for things	.78	.61
13. Seems to lack confidence	.63	.40
14. Is having problem at preschool/daycare	.67	.45
15. Isn’t excited about things that other children his/her age find exciting	.59	.34
16. Isn’t upset by things that other children his/her age find upsetting	Removed
17. Seems to be very tired and has low energy	.63	.39
18. Seems to feel overly guilty	.84	.71
19. Seems lonely	.76	.57
20. Has unusually increased or decreased appetite	.65	.42
21. Used to behave his/her age but now acts younger	.62	.39
22. Has trouble getting along with other kids	.73	.53
23. Seems more irritable/grouchy than other kids his/her age	.74	.55

Note. N = 450. Estimation method was Weighted Least Squares, mean and variance adjusted. h² = communality. Factor loadings ≥ .30 bolded for emphasis. F1 = Factor.

^aThese items were reverse coded prior to analysis.

Exploratory Factor Analysis (EFA)

The PA suggested that a three-factor solution be extracted (Figure S3). Therefore, the final set of models analyzed included models with 2–4 factors. While the factor loading patterns of the two- and four-factor models were uninterpretable, the three-factor solution was interpretable and evidenced close fit to the data across all indices (RMSEA = 0.06 [90% CI = 0.05, 0.07]; TLI = 0.95; CFI = 0.96; SRMR = .05). The three-factor solution is described below. See the Supplementary Material for details on the two- and four-factor models (Table S2).

In the three-factor solution, all PFC-S items were associated with h² > 0.20. See Table 4. Item 16 was removed from this model because it was associated with λ’s < 0.30. The resulting inter-factor correlations suggest related but unique constructs. Factor 1 was indicated by five cardinal items largely related to diminished interest and enjoyment in activities and peer engagement. Factor 2 was indicated by seven items related to broad emotional and behavioral dysregulation. Factor 3 was indicated by seven cardinal items largely related to sadness and excessive guilt. Three items had cross-loadings associated with two of the three factors. Factors 1 and 3 cross-loaded onto item 17 (“Seems to be very tired and has low energy”), indicating shared variance with items related to anhedonic and excessive guilt presentations. Factors 2 and 3 cross-loaded onto item 21 (“Used to behave his/her age but now acts younger”), indicating shared variance with items related to broad dysregulation and expressions of excessive guilt. Factors 1 and 2 cross-loaded onto item 22 (“Has trouble getting along with other kids”), indicating shared variance with items related to anhedonia and broad dysregulation. These cross-loadings were largely consistent with broader factor interpretations. Based on these factor loading patterns, factors 1–3 of this model were respectively and tentatively labeled Social and Behavioral Anhedonia (SBA), Emotional and Behavioral Dysregulation (EBD), and Excessive Guilt and Sadness (EGS).

Table 4.

Three-Factor Principal Axis Factor Analysis of the Preschool Feelings Checklist–Scale (PFC-S)

Item	F1	F2	F3	h 2
1. Interested in playing with other kidsa	.76	−.01	−.02	.56
2. Appears sad or says s/he feels sad	.14	.24	.39	.39
3. Engages in activities that make him/her happya	.89	−.01	−.06	.75
4. Has a lot of trouble following simple directions	−.03	.92	−.09	.75
5. Seems less excited about play/activities than other kids	.35	.29	.18	.41
6. Has difficulty sleeping at night	.05	.32	.22	.24
7. Whines/cries	.08	.45	.22	.39
8. Can’t pay attention to games or tasks	−.02	.84	−.05	.65
9. Enjoys activities and playa	.83	.01	−.03	.68
10. Keeps to him/herself	.39	−.04	.20	.24
11. Pretend plays about scary/sad things	.10	.39	.25	.36
12. Blames himself for things	−.07	.02	.91	.80
13. Seems to lack confidence	.02	.05	.69	.52
14. Is having problem at preschool/daycare	.14	.50	.21	.49
15. Isn’t excited about things that other children his/her age find exciting	.28	.16	.33	.37
16. Isn’t upset by things that other children his/her age find upsetting	Removed
17. Seems to be very tired and has low energy	.33	.09	.40	.44
18. Seems to feel overly guilty	.05	−.01	.90	.83
19. Seems lonely	.21	.19	.55	.61
20. Has unusually increased or decreased appetite	.20	.16	.47	.45
21. Used to behave his/her age but now acts younger	.12	.37	.30	.40
22. Has trouble getting along with other kids	.36	.42	.18	.59
23. Seems more irritable/grouchy than other kids his/her age	.29	.43	.25	.58
	Inter-Factor Correlations
	F1	F2	F3
F1	1
F2	.41*	1
F3	.39*	.46*	1

Note. N = 450. Rotation method was oblique Crawford-Ferguson Varimax. Estimation method was Weighted Least Squares, mean and variance adjusted. h² = communality. Factor loadings ≥ .30 bolded for emphasis. F1/F2/F3 = Factors.

^*indicates p < .05.

^aThese items were reverse coded prior to analysis.

Reliability

Overall, all measures, including the three PFC-S factors, showed strong internal consistency (AIC). See Table 5. Although the AIC for the BIS subscale fell below the .30 expected cutoff, this may be acceptable given that the subscale taps into a slightly broader construct compared to the other measures in the study. Moreover, the three PFC-S factors evidenced strong reliability. The majority of variance among the entire set of PFC-S items was attributable to the three factors (ECV). Each factor demonstrated strong composite score reliability (ω). The factors showed evidence of strong FD and replicability (H).

Table 5.

Psychometric Properties for Scales of Interest

Scale	n	M	SD	Range	AIC	ECV	ω	H	FD
Child PFC-S
F1 (SBA)	450	0.50	0.41	0 – 2.43	.42	0.83	0.81	0.89	0.94
F2 (EBD)	449	0.82	0.57	0 – 3.33	.44	0.80	0.82	0.89	0.95
F3 (EGS)	449	0.52	0.49	0 – 2.78	.48	0.86	0.84	0.91	0.95
Child ERC
Lability/Negativity	447	1.75	0.42	1.13 – 3.73	.46	-	-	-	-
Emotion Regulation	447	3.50	0.41	2.13 – 4	.40	-	-	-	-
Child BIS/BAS
Total BIS	363	4.23	0.92	1.29 – 7	.27	-	-	-	-
Total BAS	363	5.14	0.71	2.5 – 6.77	.32	-	-	-	-
Drive	363	4.56	1.15	1.25 – 7	.57	-	-	-	-
Fun-Seeking	363	4.69	1.03	1 – 7	.36	-	-	-	-
Reward	363	5.97	0.59	2.6 – 7	.46	-	-	-	-
Child PANAS
Negative Affect	365	1.52	0.49	1 – 3.7	.48	-	-	-	-
Positive Affect	365	3.54	0.75	1 – 5	.54	-	-	-	-
Caregiver BDI	447	0.52	0.49	0 – 2.78	.49	-	-	-	-

Note. All scales were calculated as the mean of the items pertaining to each (minimum 75% item-level completion), making the mean and standard deviation estimates interpretable relative to the scale’s range of response options. M = mean, SD = standard deviation, AIC = average inter-item correlation, ECV = proportion of explained common variance from the entire set of items attributable to each factor, ω = McDonald’s omega, H = factor replicability, FD = factor determinacy, F1/F2/F3 = Factors, SBA = Social and Behavioral Anhedonia, EBD = Emotional and Behavioral Dysregulation, EGS = Excessive Guilt and Sadness, PFC-S = Preschool Feelings Checklist–Scale, ERC = Emotion Regulation Checklist, BIS/BAS = Behavior Inhibition/Activation Scale, PANAS = Positive and Negative Affect Schedule, BDI = Beck’s Depression Inventory – II.

Measurement Invariance

The three-factor model of the PFC-S demonstrated close configural invariance across preschool-age children and those in early middle childhood (RMSEA = 0.06 [90% CI = 0.05, 0.06]; TLI = 0.95; CFI = 0.97; SRMR = 0.05) and children identified as male versus female (RMSEA = 0.06 [90% CI = 0.05, 0.07]; TLI = 0.95; CFI = 0.96; SRMR = 0.06). Whereas the χ²_diff test suggested scalar invariance was not met, changes in alternative model fit indices² suggested scalar invariance was met across preschool-age children and those in early middle childhood (χ²_diff = 137.98, df = 105, p = .017; ΔCFI = +.005; ΔTLI = +.017; ΔRMSEA = −.012) and children identified as male versus female (χ²_diff = 150.24, df = 108, p = .005; ΔCFI = +.005; ΔTLI = +.018; ΔRMSEA = −.011). Due to the significant results of the χ²_diff test, residual invariance could not be tested. Provided in the Supplementary Material (Tables S6 and S7) are estimated modification indices for the factor loadings and item thresholds of the three-factor model that were identified as potentially non-invariant across the subgroups of interest. These are provided to help inform future factor model specifications of the PFC-S. While not the focus of the present assessment, modification indices related to item residual covariances are also provided (Table S9), as these may help with the identification of potential method variance and/or redundancy among items in future investigations of the PFC-S.

Construct Validation

A unique and negative correlation was found between children’s age and EBD. Large positive and negative correlations were found between all three PFC-S factors and the Lability/Negativity and Emotion Regulation subscales of the ERC, respectively. The BIS subscale positively correlated with the three factors, sharing moderate correlations with SBA and EBD and a large correlation with EGS. EBD was uniquely, positively, and moderately correlated with the BAS and BAS Fun-Seeking subscales. The BAS Drive subscale shared a large correlation with EBD and small correlations with SBA and EGS. Interestingly, the BAS Reward Responsiveness subscale shared a small yet uniquely significant negative correlation with SBA. The PANAS NA subscale demonstrated large positive correlations with all three factors, while the PA subscale was moderately correlated with SBA and EBD and shared a small correlation with EGS. Finally, large correlations were found for all three factors with caregiver BDI scores. See Table 6. Using unit-weighted composite mean scores of the PFC-S factors did not lead to any changes in the interpretations of these correlations (Table S6).

Table 6.

Correlation Matrix of Preschool Feelings Checklist – Scale (PFC-S) Factor Scores and Related Constructs

PFC-S Factor	Age	ERC NL	ERC ER	BIS	BAS	BAS Drive	BAS Fun	BAS Reward	PANAS NA	PANAS PA	Caregiver BDI
F1 (SBA)	−.02	.48**	−.64**	.23**	.05	.17**	.01	−.11*	.33**	−.29**	.31**
F2 (EBD)	−.16**	.68**	−.47**	.24**	.29**	.38**	.24**	.00	.43**	−.20**	.37**
F3 (EGS)	.03	.42**	−.38**	.45**	.05	.13*	−.01	−.02	.34**	−.16**	.31**

Note. Pearson’s r correlations calculated using estimated factor scores for the PFC-S factors and unit-weighted composite mean scores for all other variables (minimum 75% item-level completion). SBA = Social and Behavioral Anhedonia, EBD = Emotional and Behavioral Dysregulation, EGS = Excessive Guilt and Sadness, PFC-S = Preschool Feelings Checklist–Scale, ERC = Emotion Regulation Checklist, NL = Negativity/Lability, ER = Emotion Regulation, BIS/BAS = Behavior Inhibition/Activation Scale, PANAS = Positive and Negative Affect Schedule, NA = Negative Affect, PA = Positive Affect, BDI = Beck’s Depression Inventory – II, F1/F2/F3 = Factors.

^*indicates p < .05.

^**indicates p < .01.

Discussion

The current study investigated the factor structure, measurement invariance, and construct validity of the PFC-S, a frequently used measure of depressive symptom levels in young children. Factor analytic results indicated that the PFC-S was best represented by a three-factor solution, distinguishing reliable and replicable dimensions of SBA, EBD, and EGS. In addition, measurement invariance analyses indicated that the three-factor structure demonstrated configural and scalar invariance across age groups (i.e., 3–5- or 6–8-year-olds) and assigned sex (i.e., male or female). A construct validity analysis showed that identified factors correlated in expected ways with other relevant measures of child emotion expression and regulation and caregiver depressive symptomatology. These findings suggest that the PFC-S has the potential to identify and provide information on multiple distinct yet related features of depression from preschool through the early middle childhood period in existing as well as future datasets.

Previous use of the PFC-S to measure depressive symptom levels in young children has relied upon an assumed unitary factor structure. Although previous work notes that the entire set of PFC-S items together show high levels of internal consistency, internal consistency cannot fully explain the dimensionality of a measure (McNeish, 2018). In fact, adult depression measures often show internally consistent total scores and an underlying multidimensional factor structure (see commentary in Fried et al., 2022). As a result, the current study conducted a confirmatory one-factor analysis directly testing this assumption. As anticipated, it failed to adequately fit PFC-S study data. One proposed explanation for this phenomenon is that measures capturing both depth and breadth of depressive symptom presentations, such as the PFC-S, are best represented by multidimensional structures due to the inherently multifaceted nature of depression (Fried et al., 2022). Matching prior findings of depressive symptom measures in older children (e.g., Child Depression Inventory; Stumper et al., 2019; Craighead et al., 1998; Cole et al., 2000), the current findings indicate that depressive symptoms in early childhood are also multifaceted in nature and require measurement approaches that can accurately assess them.

In line with this, an exploratory three-factor model of the PFC-S emerged as the most interpretable model with close fit to the data in this study. Three factors emerged, interpreted as reflective of SBA, EBD, and EGS. While these factors may be interpreted in various ways, we attempt to provide interpretations that are both parsimonious and reflective of the variance primarily shared across the cardinal items of the factor loading matrix resulting from the EFA of the PFC-S. The SBA factor is primarily reflective of diminished interest in typically enjoyed activities and play, a hallmark of an anhedonic presentation in early childhood (Luby, Mrakotsky, et al., 2003), and social withdrawal related to this diminished positive response style (Prabhakar et al., 2022). The EBD factor is reflective of a broad range of dysregulated emotion and behavior, spanning difficulties in attention and executive functioning, irritability, sleep problems, crying, and scary or sad themes in play. While these challenges are commonly present in early childhood depression, they are also present in other forms of youth psychopathology (Whalen et al., 2017). The EGS factor is primarily reflective of excessive guilt that manifests in young children as prolonged self-blame, low confidence, and rumination; negative affect, loneliness, and appetite disruptions may be related manifestations (Tilghman-Osborne et al., 2012; Caprara et al., 1992; Muris & Meesters, 2014; Huang & Dong, 2014). Children with higher depressive symptomatology are prone to guilt-related neural alterations, sadness, and negative self-evaluation (Donohue et al., 2020; Zahn-Waxler & Kochanski, 1990; Malti et al., 2016). The emergence of these distinct yet related constructs from the PFC-S reinforces the view (Brown, 2007) that depression is composed of both transdiagnostic mechanisms (i.e., emotion dysregulation; Beauchaine & Cicchetti, 2019) and core features more specifically, but not solely, linked to depression (i.e., anhedonia and excessive guilt; Luby, Mrakotsky, et al., 2003). As a result, the current findings indicate that the PFC-S has the potential to provide researchers and clinicians a greater level of specificity in the evaluation of developmentally appropriate depressive endophenotypes (Whalen et al., 2017).

The three identified factors demonstrated strong internal consistency, reliability, replicability, and factor determinacy. Importantly, the correlation pattern among the three factors of the PFC-S indicate relatedness but not multicollinearity. All three factors demonstrated convergent construct validity by correlating in expected ways with related constructs, including emotion regulation, behavioral inhibition, positive and negative affect, and caregiver depressive symptoms. While the correlations between each factor and these measures were largely similar, these results provide important support for their associations with well-established mechanisms, correlates, and risk factors of depressive symptomatology. That the correlations were largely similar may be due to the external measures assessing broad constructs that are commonly altered in depression. It is also possible that these correlations are behaving similarly because the factor scores derived from the three-factor model are explaining item variance from an ESEM in which all factors were allowed to load to some extent on all PFC-S items. That the three factors show larger but not multicollinear correlations with each other may also be evidence for this. Together, these results situate the identified PFC-S factors among related constructs of wide research and clinical interest, highlighting the utility of these factors in the assessment of meaningful individual differences on the constructs.

However, each factor also uniquely correlated with certain constructs which indicated important preliminary discriminant validity. Interestingly, we found a unique and negative correlation between the BAS Reward Responsiveness subscale and SBA. This finding suggests that this factor may provide a unique measure of altered positive emotion (Belden et al., 2016), a key symptom of depression across multiple age groups, including preschoolers (Luby, Mrakotsky, et al., 2004), which may be closely aligned with altered neural reward processing and social problems in young children (Gaffrey et al., 2013; Gaffrey, Tillman, et al., 2018; Prabhakar et al., 2022). We also found a unique and negative correlation between child age and EBD, which aligns well with prior work showing that children’s regulatory capacities improve with age (Ratcliff et al., 2021). Further, unique positive correlations were found for both the BAS and the BAS Fun-Seeking subscales with EBD. It is possible that elevated EBD scores reflect difficulties with impulse control in the presence of desirable objects and/or events (Beauchaine, 2015); it is also possible that they reflect delayed and/or altered patterns of development in this and/or closely related skills (e.g., executive function more broadly). Unfortunately, the current study cannot make this type of functional distinction, and future research in this area is now needed.

The current study did not include an external validator specific to the experience of guilt, as the analyses on which the present work is based were started after all data collection had finished. Consequently, the EGS factor did not show unique correlations with any of the validation measures, as the majority of the validation measures indexed broader constructs related to depression or were more specific to reward processes largely unrelated to guilt. Importantly, the EGS factor was correlated in expected ways with positive and negative affect, emotion regulation, behavioral inhibition, and parental depressive symptom severity. While face validity for the EGS factor is clear, further research is needed to provide greater specificity to its convergent validity with other measures of guilt. Study results suggest that the PFC-S has the potential to catalyze research on behavioral and neurobiological correlates of depression and further establish a strong foundation for the measurement of distinct disorder-related dimensions.

Finally, the three-factor structure of the PFC-S showed configural and scalar invariance across subgroups of 3–5- and 6–8-year-olds as well as among both male and female children. These results are critically important given the inclusion of these groups in prior work using the PFC-S (Luby et al., 2012; Luby et al., 2018; Gaffrey, Barch, et al., 2018; Fowler et al., 2021; Fowler & Gaffrey, 2022; Heaton et al., 2023). Evidence of configural invariance suggests that the overarching PFC-S three-factor structure broadly applies to 3 to 5-year-olds and 6 to 8-year-olds and to children assigned male and female sex. Evidence of metric and scalar invariance (estimated in one step) suggests that the relations between items and factors are quite similar across subgroups and differences in item-level scores across subgroups are fully accounted for by subgroup differences in the latent factors. Interestingly, when viewed on their own, the results of the χ²_diff tests suggested lack of scalar invariance, leading to the inability to test residual invariance of the three-factor PFC-S model. However, it is rare that scalar invariance is met when using direct model comparison tests (Rutkowski & Svetina, 2013). Furthermore, supplemental data on the modification indices resulting from the scalar models suggest that very few, if any, factor loadings and item thresholds potentially lacking invariance contributed to model misfit (Tables S7 and S8). It is possible that scalar invariance was not met according to the χ²_diff tests because other areas of the model may have substantially worsened model fit (i.e., item residual covariances; Table S9), despite not being the focus of tests of scalar invariance. These supplemental data are provided to help contextualize these results and provide a useful point of departure for future work to clarify the utility of the PFC-S across the identified subgroups.

Recommendations

While the current results suggest that the PFC-S’s factor structure is multidimensional and reflective of transdiagnostic constructs that are commonly seen in, but not limited to, early childhood depression, the factor structure of the PFC-S cannot be definitively established by the exploratory methods in this initial investigation. As such, we provide recommendations for future research on the psychometrics of the PFC-S. First, we recommend that the factor structure of the PFC-S continue to be investigated in existing and new datasets. Future studies should use confirmatory approaches to compare the one- versus three-factor models investigated here. In the case that neither of these models provide adequate and parsimonious accounts of the PFC-S data, the use of EFA may be warranted. Second, we suggest that future work consider the presence of method variance and/or item redundancy that may affect the factor structure of the PFC-S. To this end, researchers may find it beneficial to consider potential method variance across items with similar wording and to review the correlations among responses to the PFC-S items. Third, if a multidimensional model of the PFC-S (e.g., the three-factor model presented here) shows adequate fit and interpretability, we suggest that a bifactor model be tested. As such, it is possible that the PFC-S may still have the potential to reflect a single general factor of depressive symptomatology alongside the three specific dimensions found here (i.e., bifactor model). Our findings do not eliminate this possibility, and the close fit of the one-factor model according to the SRMR indicates this may be an important next step to investigate. Fourth, future work should continue to assess the measurement invariance of prevailing factor models of the PFC-S across groups of children who differ according to characteristics that may be relevant to the study of depressive symptoms, such as age and sex assigned at birth. Doing so will help clarify the optimal use of the PFC-S across the identified subgroups. Relatedly, future work should aim to test the longitudinal invariance of the PFC-S factor structure to provide a stronger foundation from which to understand changes in early childhood depressive symptomatology over time. Fifth, the evaluation of the PFC-S factors’ convergent and divergent validity would further benefit from future research that uses more specific questionnaire and behavioral measures to test the convergent and divergent validity of the constructs they are proposed to reflect. The current results serve as important foundational steps for the identification of specific factors of the PFC-S, and future confirmatory studies are now needed to replicate and expand on them.

We also provide practical recommendations here for the use of the PFC-S to measure dimensions of depressive symptoms in early childhood samples. First, we urge researchers to estimate and assess one- and three-factor structures of the PFC-S in their own data, rather than assume either of them to be adequately reflective of their own data. Second, if interested in how the factors are associated with other constructs, researchers are urged to test their hypotheses within a structural equation modeling framework that incorporates a factor model of the PFC-S. This has the added benefit of minimizing the effect of measurement error on the results. Third, if analyzing the data within a structural equation model is not possible, we suggest that researchers use the factor scores resulting from a factor analysis of the PFC-S to test associations with external constructs. Fourth, if unit-weighted composite scores (i.e., sum or mean of scores across items onto which a factor loaded) of the three PFC-S factors are used instead, researchers are urged to note the limitation that residual invariance has not yet been established for this measure. This means that the calculated composite scores may differ across subgroups as a function of subgroup differences in the error variance of the items, which could cloud the interpretability of findings. While the measurement invariance assessment results presented here may preliminarily discourage the use of composite scores, the three factors showed strong composite score reliability (ω). Moreover, calculated PFC-S mean composite scores showed correlation patterns with the external validation measures (Table S6) that were similar to the patterns shown when PFC-S factor scores were used (Table 6). These results indicate that the PFC-S continues to show promise as a measure of individual differences among specific depressive symptom dimensions during early childhood. More research is needed to ensure the most optimal use of the PFC-S.

Limitations

The current results should be considered alongside their limitations. First, biases have been documented in caregiver reports of children’s depressive symptomatology (Najman et al., 2000). However, recent work suggests that the influence of caregiver symptomatology may not lead to substantial reporting bias of child characteristics (Olino et al., 2020). Second, relative to national demographics of the United States, our sample underrepresented ethnically minoritized groups of children and caregivers with less than a four-year college degree (U.S. Census Bureau, 2021). Non-maternal caregivers were also underrepresented as reporters of child characteristics in our sample. Third, despite having about one-fourth of the children in the sample flagged as being at-risk of developing depression, endorsement of children’s depressive symptom severity on each item was right-skewed. However, the level at which caregivers endorsed their children’s depressive symptoms levels coincides with previous reports of PFC-S total score distributions in prior studies (Luby et al., 2012; Luby et al., 2018). Finally, it is possible that method variance may be influencing factor identification in the EFA. Future work should consider modeling method variance to test the stability of the factors.

Conclusion

Ultimately, findings from the current study suggest that caregiver responses on the PFC-S are influenced by latent factors reflective of social and behavioral anhedonia, excessive guilt and sadness, and emotional and behavioral dysregulation dimensions that show configural and scalar invariance among young children. Although the current results were produced using exploratory approaches and need to be confirmed in future work, the identification of multiple PFC-S factors provides further evidence for the multidimensional nature of early childhood depressive symptoms and offers new opportunities for elucidating neurobiological and environmental correlates of depression-related symptomatology during this critical developmental period.