A Brief Early Childhood Screening Tool for Psychopathology Risk in Primary Care: The Moderating Role of Poverty

Abstract

Objectives

To evaluate the Preschool Feeling Checklist (PFC) utility for predicting later mental disorders and functioning for children and assess whether the PFC’s predictive utility differs as a function of childhood poverty.

Study design

We analyzed data from a prospective longitudinal study of preschoolers in St Louis. Preschoolers (N = 287) were recruited from primary care sites and were assessed annually for 10–15 years. The PFC screened for depressive symptoms. Later age-appropriate psychiatric diagnostic interviews were used to derive Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, diagnoses. Regression and moderation analyses, and multilevel modeling were used to test the association between the PFC and later outcomes, and whether this relationship was moderated by income-to-needs.

Results

The PFC predicted major depressive disorder (OR 1.13, P < .001), attention deficit hyperactivity disorder (OR 1.16, P < .001), and mania (OR 1.18, P < .05) in adolescence and early adulthood. Income-to-needs was a moderator in the predictive pathway between the PFC and later major depressive disorder (OR 1.10, P < .05) and mania (OR 1.19, P < .001) with the measure less predictive for children living in poverty. The PFC predicted worse functioning by the final assessment (b = 1.71, SE = 0.51, P = .001).

Conclusions

The PFC served as an indicator of risk for later attention deficit hyperactivity disorder and impairment in all children. It has predictive utility for later mood disorders only in children living above the poverty line. Predicting depression in children living below the poverty line may require consideration of risk factors not covered by the PFC.

Evidence supports the validity of a depressive syndrome in preschool-aged children.^1–6 Given that preschool depression negatively affects development, often persists into adolescence,^7–10 and may be more difficult to treat later in development,¹¹ early recognition is important. The pragmatic clinical importance of early intervention is now further underscored by the availability of an empirically tested and manualized and relatively short parent–child therapy (Parent-Child Interaction Therapy Emotion Development) for which large effect sizes have been demonstrated.¹² However, pediatricians and primary care practitioners often fail to recognize or diagnose early psychiatric disorders such as depression.¹³ A brief and feasible screening tool to identify young children in need of clinical evaluation is necessary. However, it is important to acknowledge that even a screening tool with strong psychometric properties that will allow identification of those who should be targeted for early intervention using empirically tested therapies cannot impact the public health without the ready availability of these therapies in communities. Ultimately, such a screening tool will need to be backed by a healthcare system that can reliably deliver effective mental health services to individuals who are at risk on the screener.

The Preschool Feelings Checklist (PFC)¹⁴ is a 16-item parent-report checklist originally designed to identify preschoolers with symptoms of depression that has been increasingly used for this purpose.^3,15–17 The first validation study of the PFC was conducted in a sample of 174 preschoolers enriched for depression.¹⁴ In this sample, the PFC exhibited high internal consistency and strong correlations with the Internalizing Problems scale on the Child Behavioral Checklist.¹⁸ Moreover, the PFC demonstrated good sensitivity (0.92) and specificity (0.84) for identifying cases of preschool depression at a score of ≥3.

Silver et al conducted a systematic investigation of the PFC’s psychometric properties in a longitudinal (ie, aged 3–15 years old) community sample (N = 490) and provided strong support for the PFC’s validity.¹⁹ The PFC was significantly associated with concurrent and future depression in later childhood and adolescence. However, it also showed strong concurrent and predictive associations with a range of other mental disorders and global functioning, suggesting that it indexes broad transdiagnostic liability for later psychopathology and impairment. In addition, the PFC outperformed the longer, proprietary Child Behavioral Checklist in predicting diagnoses and functioning.¹⁹

The number of cases of depression in Silver et al was small, underscoring the need to examine predictive validity in clinically enriched samples.¹⁹ Second, as the PFC was designed to quickly identify children in need of mental health referrals, it is important to determine how the PFC functions as a screening measure more broadly for children from diverse demographic backgrounds. It remains unclear whether the PFC’s predictive utility, or in other words the PFC’s ability to predict later psychopathology, differs as a function of childhood poverty, a well-established robust risk factor for later psychopathology.²⁰

Youth with lower socioeconomic status (SES) exhibit greater levels of psychopathology than their peers from greater-SES families, with stronger associations to behavior problems than depression or anxiety.²¹ Moreover, measures of depression are subject to psychometric biases associated with SES,²² and, as such, measurements of depression should include consideration of possible SES disparities. Indicators of low SES have been shown to moderate the effects between several predictors (eg, stressful life events and discrimination) and later depression.^23,24

The current study extends the original validation of the PFC in a 15-year longitudinal study sample, the Preschool Depression Study, that was enriched for early childhood depression.¹⁶ The present paper examines the predictive utility of the PFC for later depression and other mental disorders, as well as global functioning. We explore whether the PFC’s performance and predictive utility differs for children living in poverty.

Methods

Preschoolers between the ages of 3.00 and 5.11 years were recruited from primary care and daycare sites in the St Louis metropolitan area and screened with the PFC.^14,25 The present study sample is different from the sample used in the original validation study.¹⁰ Children with symptoms of depression were oversampled, and children with symptoms of other psychiatric disorders and healthy children were included as comparison groups. The Preschool Depression Study was originally designed to investigate the question of whether clinical depression could be identified in preschool-aged children and whether it showed discriminant validity from other preschool disorders. For this reason, those with high scores on the PFC were oversampled. Although those with high scores on the PFC were oversampled, there exist some subjects with a score of 1 or 2 who were included in the present sample. This small number of subjects slipped through the original algorithm for recruitment that sought to include participants with a score of 0 or 3 or greater. However, the algorithm for recruitment was generally followed with these exceptions. The longitudinal arm of the study was pursued to determine both the developmental course and outcomes of preschool depression and to determine whether episodes of preschool depression had enduring impact on other aspects of development. Details of the study recruitment methods and participant flow have been described previously.¹⁰

Children and their primary caregivers participated in up to 10 comprehensive annual assessments from ages 3.00–21.70 years (Figure 1). After receiving a complete description of the study, parents provided written informed consent and children provided assent/consent based on age as appropriate. All study procedures were approved in advance by the Washington University School of Medicine institutional review board.

Figure 1.

Preschool Depression Study flowchart.

Measures

Early Childhood Psychopathology.

The PFC²⁵ was used to assess preschool depressive symptoms (measure available free of charge at eedp.wustl.edu). The 16 dichotomous items were summed to give a total score. In previous studies, the point of maximal sensitivity (ability to correctly identify the depressed children) and specificity (not falsely identifying those with no disorder) was found at a PFC score of 3 or greater, which was associated with values of 0.92 and 0.84, respectively.¹⁴ Standard psychometric results (ie, sensitivity, specificity, positive predictive value, negative predictive value) for the prediction of major depressive disorder (MDD) at any point during the follow-up using the PFC in the present study are available in Table I. This measure has demonstrated good internal consistency, with a Cronbach alpha ranging from 0.76¹⁴ to 0.85.¹⁹

Table I.

Psychometric results for the prediction of MDD at any point during the follow-up using the PFC

Cutoffs	Sensitivity	Specificity	PPV	NPV
≥1	0.9540	0.2000	0.6148	0.7647
≥2	0.9253	0.2538	0.6240	0.7174
≥3	0.9023	0.3154	0.6382	0.7069
≥4	0.7529	0.4692	0.6550	0.5865
≥5	0.6264	0.6769	0.7219	0.5752
≥6	0.4598	0.8077	0.7619	0.5276
≥7	0.3333	0.8462	0.7436	0.4867
≥8	0.2414	0.8692	0.7119	0.4612
≥9	0.1322	0.9000	0.6389	0.4366
≥10	0.0977	0.9538	0.7391	0.4413
≥11	0.0517	0.9846	0.8182	0.4369
≥12	0.0287	0.9846	0.7143	0.4310
≥13	0.0230	1.0000	1.0000	0.4333
≥14	0.0115	1.0000	1.0000	0.4305
≥15	0.0057	1.0000	1.0000	0.4290

NPV, negative predictive value; PPV, positive predictive value.

During annual assessment waves when children were 3.00–7.90 years, parents were interviewed about their child using the Preschool Age Psychiatric Assessment (PAPA), an age-appropriate and reliable psychiatric diagnostic interview.²⁶ The PAPA was used to derive Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition -5 diagnoses (Appendix; available at www.jpeds.com). Further details on training and administration of the PAPA in the study sample can be found elsewhere.¹⁰

School-Age and Adolescent Psychopathology.

During assessment waves when children were age 8.00–8.90 years, the Child and Adolescent Psychiatric Assessment²⁷ was administered to parents only, and when children were age 9.0 years and older, the Child and Adolescent Psychiatric Assessment was administered separately to parents and children. The Kiddie Schedule for Affective Disorders and Schizophrenia²⁸ was administered separately to parents and children at the last 2 assessment waves when children ranged in age from 12.30 to 21.60 years. For the present study, we aggregated diagnoses across all assessment waves to create a single variable for each diagnosis (Appendix).

Income-to-Needs Ratio.

The income-to-needs ratio was used to index economic disadvantage. It was operationalized as the total family income divided by the federal poverty level based on family size and geographic region in the year of data collection.²⁹ The value was calculated for each child’s baseline session based on caregiver report.

Functioning.

The Preschool and Early Childhood Functional Assessment Scale/Child and Adolescent Functional Assessment Scale (CAFAS)^30,31 was used as an independent measure of youth’s functional impairment (Appendix). The Preschool and Early Childhood Functional Assessment Scale (for ages 3.00–7.90 years) and CAFAS (age ≥8.00 years) are interviewer-rated measures that assess the psychosocial functioning and impairment.³²

Data Analyses.

Multilevel models were conducted using SAS (SAS Institute), version 9.4,³³ and all other statistical analyses were performed using SPSS 25 (International Business Machines).³⁴ Moderation analyses were performed using PROCESS.³⁵ First, to examine unique associations between the PFC and subsequent diagnoses, we conducted logistic regressions, with PFC total score, age, and income-to-needs ratio at baseline (ages 3.00–5.11 years) as the independent variables and specific diagnoses at follow-up (ages 3.11–21.70 years) as the dependent variable. For logistic regression analyses, 3 models were run. Step 1 excluded youth with the corresponding diagnosis at baseline to assess whether the PFC was useful in predicting new onsets of the disorder. Step 2 was identical to Step 1 but also controlled for age and income-to-needs ratio at baseline to assess whether the PFC was useful in predicting new onsets of the disorder in the context of covariates. All models were also run with race as a covariate. In each model, results remained the same after controlling for race. However, these findings are not presented in the manuscript, as race and class were closely related in the present data and may lead to misleading conclusions attributed to race. Step 3 was identical to Step 2 but tested the moderation analyses.

Next, we compared the associations of the PFC and global functioning using multilevel modeling, respectively. This analysis also tested whether the associations were moderated by income-to-needs ratio at baseline (ages 3.00–5.11 years). In multilevel models testing the associations between the PFC and functioning across time, time was defined as assessment wave, centered at the final assessment at which the CAFAS was administered (ages 10.10–15.80 years), so the intercept reflects the level of the dependent variable at the final assessment.

Results

This sample comprised 287 preschoolers with a mean age of 4.45 (0.80) years at baseline (ages 3.00–5.11 years). The sample was 50% male (N = 145), with an average income-to-needs ratio of 2.02 (1.87). By definition, 1.0 is the poverty line, and numbers above that are multiples of income to needs (eg, 3.0 is income of 3 times the poverty line). The sample was 52.6% White (N = 151), 34.5% Black (N = 99), and 12.9% “other” (N = 37).

The mean score of the PFC at screening was 4.72 (SD 3.04). The PFC was correlated with children’s age at baseline, r(287) = 0.13, P = .026, and income-to-needs ratio, r(264) = −0.36, P < .001, such that older and lower income children had greater PFC scores. PFC scores were significantly lower in White (3.97 ± 2.95) than Black (5.61 ± 2.84, t = 4.30, P < .001) and “other” children (5.46 ± 3.21, t = 2.76, P < .001). PFC scores were not associated with sex, t = 0.93, P = .355. Race and income-to-needs are highly correlated in this sample (r[264] = −0.39, P < .001), so the present study focuses on income-to-needs instead of race to capture the most interpretable variance between children.

Predictive Utility: PFC and MDD

First, a logistic regression was run to determine whether PFC scores were associated with MDD at any time during follow-up (ages 3.11–21.70 years). In Step 1, the PFC significantly predicted a diagnosis of MDD (Table II). In Step 2, PFC scores remained significantly associated with a subsequent MDD diagnosis (Table II).

Table II.

Logistic regression analyses using the PFC to predict subsequent diagnoses at any point during follow-up (T2-T10)

	MDD		ADHD		ODD		CD		GAD		PTSD		SAD		Mania
Variables in model	OR	95% CI	OR	95% CI	OR	95% CI	OR	95% CI	OR	95% CI	OR	95% CI	OR	95% CI	OR	95% CI
Model 1*
PFC	1.13†	1.03–1.25	1.22†	1.09–1.35	1.12‡	1.00–1.25	1.10	0.97–1.23	1.08§	0.99–1.17	1.12	0.95–1.32	1.01	0.89–1.14	1.23†	1.09–1.38
Model 2¶
PFC	1.11‡	1.00–1.24	1.16†	1.04–1.30	1.09	0.96–1.24	1.01	0.88–1.16	1.09§	0.9–1.20	1.09	0.91–1.31	1.00	0.88–1.14	1.18‡	1.03–1.34
Age at T1	1.04	0.71–1.52	1.20	0.80–1.79	0.81	0.51–1.28	1.14	0.70–1.84	1.12	0.81–1.56	0.97	0.51–1.87	1.02	0.64–1.63	1.66‡	1.03–2.67
Income to needs at T1	0.84	0.63–1.11	0.97	0.73–1.29	0.96	0.69–1.34	0.60†	0.43–0.84	1.13	0.89–1.43	0.83	0.52–1.32	0.99	0.71–1.39	0.97	0.69–1.35
Model 3**
PFC	0.90‡	0.71–1.14	1.19	0.94–1.50	1.21	0.93–1.56	0.93	0.72–1.21	1.18§	0.97–1.43	1.12	0.81–1.55	1.02	0.78–1.33	0.83	0.63–1.11
Age at T1	1.02	0.69–1.49	1.20	0.80–1.81	0.83	0.52–1.31	1.12	0.69–1.81	1.14	0.82–1.59	0.98	0.51–1.89	1.02	0.64–1.63	1.58§	0.97–2.57
Income-to-Needs at T1	0.53‡	0.30–0.92	1.02	0.57–1.82	1.23	0.64–2.36	0.49‡	0.25–0.96	1.35	0.84–2.16	0.90	0.35–2.29	1.03	0.55–1.95	0.35‡	0.16–0.78
PFC× income-to-needs	1.10‡	0.99–1.22	0.99	0.90–1.08	0.95	0.85–1.06	1.04	0.92–1.17	0.96	0.89–1.04	0.98	0.85–1.14	0.99	0.89–1.10	1.19†	1.05–1.35

CD, conduct disorder; GAD, generalized anxiety disorder; PTSD, post-traumatic stress disorder; SAD, social anxiety disorder.

^*Model 1 excludes those with the corresponding disorder at T1.

^†<.01.

^‡<.05.

^§<.10.

^¶Model 2 excludes those with the corresponding disorder at T1 and adds age and income-to-needs ratio at T1 as covariates.

^**Model 3 excludes those with the corresponding disorder at T1, controls for age and income-to-needs ratio at T1, and presents the moderation analyses.

We also tested whether the association between PFC and depression-related outcomes were moderated by income-to-needs. In Step 3, the interaction term for PFC by baseline (ages 3.00–5.11) income-to-needs was significant (Table II). The relationship between PFC scores and MDD diagnoses at follow-up was stronger among youth in greater-SES families (Figure 2, A). Conditional effects of the PFC at values of the moderator (ie, income to needs) revealed that the PFC significantly predicted later MDD at the mean, and at +1 SD above the mean, of income to needs (Table III).

Figure 2.

A, Relation between predicted probability of major depressive disorder and preschool feelings checklist total as a function of income-to-needs ratio. B, Relation between predicted probability of mania and preschool feelings checklist total as a function of income-to-needs ratio.

Table III.

Conditional effects of the PFC at values (−1 SD, mean, and +1 SD) of income-to-needs

Values of income-to-needs	Effect	Se	Z	P value	LL CI	UL CI
MDD
−1 SD of mean	.00	.07	.05	.95	−.14	.15
Mean	.11	.05	1.98	.04†	.00	.22
+1 SD of mean	.22	.08	2.66	.00‡	.05	.38
Mania
−1 SD of mean	−.01	.09	−.16	.86	−.20	.17
Mean	.19	.07	2.56	.01†	.04	.33
+1 SD of mean	.40	.11	3.46	.00‡	.17	.62

LL, lower limit; UL, upper limit.

^*<.10.

^†<.05.

^‡<.01.

Predictive Utility: PFC and Other Disorders

A series of logistic regressions were run to determine which diagnoses the PFC was associated with at any time during follow-up (ages 3.11–21.70 years). In Step 1, the PFC significantly predicted attention deficit hyperactivity disorder (ADHD), oppositional defiant disorder (ODD), and mania (Table II). In Step 2, the PFC significantly predicted a diagnosis of ADHD and mania (Table II).

Finally, we tested whether the association between PFC and subsequent diagnoses were moderated by income-to-needs. In Step 3, the interaction term for PFC by income-to-needs was significant only for mania (Table II). The concordance between PFC scores and mania diagnoses at follow-up was greater among subjects in middle- and greater-SES families (Figure 2, B). Conditional effects of the PFC at values of the moderator (ie, income-to-needs) revealed that the PFC significantly predicted later mania at the mean, and +1 SD above the mean, of income to needs (Table III).

Predictive Utility: PFC and Functioning

In multilevel models, when we controlled for income-to-needs and age at baseline, the PFC predicted the intercept (b = 1.71, SE = 0.51, t = 3.32 P = .001), but not the slope (b = −0.18, SE = 0.10, t = −1.71, P = .088), of functioning over time. As such, children with greater PFC scores at baseline had worse functioning by ages 10.10–15.80 years than children with lower PFC scores at ages 3.0–5.11 years. Income-to-needs did not serve as a moderator in this model (b = 0.24, SE = 0.32, t = 0.73, P = .464).

Discussion

The PFC is a free and feasible 2- to 5-minute screening tool that can be used to identify young children from the general population in need of a clinical mental health evaluation. Our findings demonstrate the strong predictive utility of the PFC for depression, mania, ADHD, and functional impairment. The PFC performed differently for children from more vs less economically advantaged backgrounds in the area of mood disorders. We tested the role of income to needs as a moderator in the predictive pathway between the PFC and all psychiatric outcomes (ie, MDD, ADHD, ODD, conduct disorder, generalized anxiety disorder, post-traumatic stress disorder, social anxiety disorder, mania, and functional impairment). Although the PFC significantly predicted later ADHD and impairment, income to needs did not act as a moderator and thus can be a useful tool to predict these outcomes in all populations. However, significant moderation was only found for the PFC and later mood disorders (MDD and mania), suggesting that the PFC is a strong predictor of these later mood disorders only in children living above the poverty line.

When we excluded participants with MDD at baseline and controlledfor age and income-to-needs, the PFC stillpredicted later childhood mood disorders (depression and mania), suggesting it is useful to predict new onsets of later mood disorders. These findings have implications for the use of the PFC in primary care settings for identifying children at high risk for later mood disorders, only for those living above the poverty line. This is particularly noteworthy, as 82% of children younger than age 18 years live above the poverty line, suggesting that the PFC is useful to predict mood disorders in at least 82% of the child population in the US.³⁶

Moreover, findings suggest that symptoms of depression arising in the context of poverty may be driven by a broader set of risk factors. Multiple pathways to depression among children from lower-income families (eg, greater PFC scores may capture/reflect one pathway, but other risk pathways appear salient for children living in poverty with low PFC scores) were found. As such, income-to-needs should be considered when interpreting the predictive utility of PFC scores and risk for mood disorders. Importantly, these findings do not suggest that children from low-income families with greater PFC scores are less likely to become depressed, rather, they indicate that among children facing poverty, those with low PFC scores may also be at risk for later depression.

Although recent literature examining the predictive utility of the PFC for concurrent depression found no differences in the utility of the screening tool across families from different SES and racial/ethnic backgrounds,¹⁹ it is well known that variation in SES is a strong predictor of child psychopathology.²¹ The findings of past research and the present study suggest the PFC should be administered to all, with a particular note of caution when interpreting results with regard to risk for later mood disorders for those from low-income families. Future research should focus on improving the measurement of preschool depression among low-income young children, and specifying prevention approaches that account for the child’s resources (ie, universal prevention for low-income families and indicated prevention for middle/high income families) should be considered.

The PFC also predicted later externalizing outcomes, specifically ADHD and ODD. These results are similar to previous findings demonstrating the PFC as a useful tool in indexing and predicting later psychopathology more broadly¹⁹ and indicates that the PFC may be useful to predict new onsets of ADHD and ODD. However, in the present study, when we controlled for risk factors such as age and income-to-needs, the PFC only predicted future ADHD, indicating that young children with greater PFC scores may be at increased risk for ADHD regardless of poverty status. Findings suggest that the PFC predicted diagnoses even when clinical diagnostic criteria were not met in the preschool period, supporting the PFC as a useful tool for identifying clinically important subthreshold states more broadly including in children living in poverty.

Data also demonstrate the predictive utility of the PFC for later functional impairment and suggest that children with greater PFC scores are at increased risk for having poorer functional impairment later in development. This finding arose across the income spectrum suggesting the PFC may be a useful tool to identify children in need of further mental health evaluation.

Because our sample was enriched for early childhood depression and other disorders, it had greater power to detect effects. However, the study had several limitations. First, the number of children in this sample living below the poverty line was relatively small (n = 75) and therefore, our power to detect significant moderation effects may have been limited. Second, the small sample in the current study did not allow for the internal validity of the predictive logistic regression model to be tested. Future research on the PFC should demonstrate that PFC scores calibrate the outcomes of interest. Studies with larger samples of children living in poverty would be fruitful for further clarifying the longitudinal predictive utility of the PFC for children living in poverty.

The relationship between the PFC and later mood outcomes were significantly moderated by income-to-needs, demonstrating that the PFC strongly predicts later depression and mania only in children living above the poverty line. For children living below the poverty line, risk factors may have resulted in a multitude of outcomes across development. This finding underscores the detrimental power of poverty and related psychosocial adversity in the risk trajectories for mood disorders in childhood. Its predictive utility for mood disorders should be applied to children living above but not below the poverty line. Given that the PFC predicts later ADHD and impairment across all children regardless of poverty status, it may also be a good indicator of when further mental health evaluation is needed. The addition of this screening component in primary care settings has the potential to greatly facilitate early identification of risk for psychopathology and refer for needed clinical assessment.

Glossary

ADHD

Attention deficit hyperactivity disorder

CAFAS

Child and Adolescent Functional Assessment Scale

MDD

Major depressive disorder

ODD

Oppositional Defiant Disorder

PAPA

Preschool Age Psychiatric Assessment

PFC

Preschool Feelings Checklist

SES

Socioeconomic status