Skip to main content
PMC home page
Journal of Pediatric Psychology logoLink to Journal of Pediatric Psychology
. 2022 Apr 18;47(5):547–558. doi: 10.1093/jpepsy/jsac029

Measuring PROMIS® Emotional Distress in Early Childhood

Phillip Sherlock 1,✉, Courtney K Blackwell 1, Michael A Kallen 1, Jin-Shei Lai 1, David Cella 1, Sheila Krogh-Jespersen 1, Joan L Luby 2, Kristin A Buss 3, James Burns 1, Lauren S Wakschlag 1
PMCID: PMC9113325  PMID: 35552432

Abstract

Objective

Create and validate developmentally sensitive parent-report measures of emotional distress for children ages 1–5 years that conceptually align with the Patient-Reported Outcome Measurement Information System (PROMIS®) pediatric measures.

Methods

Initial items were generated based on expert and parent input regarding core components of emotional distress in early childhood and review of theoretical and empirical work in this domain. Items were psychometrically tested using data from two waves of panel surveys. Item response theory (IRT) was applied to develop item calibration parameters (Wave 1), and scores were centered on a general U.S. population sample (Wave 2). Final PROMIS early childhood (EC) instruments were compared with existing measures of related constructs to establish construct validity.

Results

Experts and parents confirmed the content validity of the existing PROMIS Pediatric emotional distress domains (i.e., anger, anxiety, and depressive symptoms) as developmentally salient for young children. Existing items were adapted and expanded for early childhood by employing best practices from developmental measurement science. Item banks as well as 4- and 8-item short forms, free from differential item functioning across sex and age, were constructed for the three domains based on rigorous IRT analyses. Correlations with subscales from previously validated measures provided further evidence of construct validity.

Conclusions

The PROMIS EC Anger/Irritability, Anxiety, and Depressive Symptoms measures demonstrated good reliability and initial evidence of validity for use in early childhood. This is an important contribution to advancing brief, efficient measurement of emotional distress in young children, closing a developmental gap in PROMIS pediatric emotional distress assessment.

Keywords: anger, anxiety, depression, depressive symptoms, early childhood, emotional distress, irritability, IRT, mental health, PROMIS

Introduction

From early ages, children’s emotional distress can significantly affect social-emotional trajectories (Alink et al., 2006; Carbonneau et al., 2016; Hay et al., 2014). The Patient-Reported Outcomes Measurement Information System (PROMIS®) pediatric initiative validated brief and robust emotional distress measures to capture anxiety, anger, and depressive symptoms for 5–17 year olds. Such measures parallel PROMIS adult instruments, allowing for continuity of assessment from pediatric to adult health (Varni et al., 2014), and have been validated across chronic health conditions (DeWalt et al., 2015; Irwin et al., 2012). Recent advancements in measurement science suggest such constructs can be reliably and validly measured in younger children, with associations to impairment in functioning and predictive utility across the lifespan (Briggs-Gowan et al., 2004; Buss et al., 2013; Egger & Angold, 2006; Luby, 2010; Wakschlag et al., 2018). However, a lack of pragmatic developmentally sensitive measures feasible for clinical use remains a gap in the field. To bridge this divide, we used PROMIS methodology (Cella et al., 2007; PROMIS Cooperative Group, 2013), and drew on the developmental specification framework to ensure a developmentally meaningful process to: (a) Create lifespan coherent (i.e., varied developmental manifestations of a consistent phenomenology over time as described in Wakschlag et al., 2010; Carter et al., 2013; Blackwell et al., 2020) PROMIS early childhood (EC) parent report measures of emotional distress that conceptually align with PROMIS Pediatric instruments and are calibrated and normed to the general US population; and (b) Conduct preliminary validation of these instruments.

Methods

Cella et al. and Lai et al. (this issue) provide detailed qualitative and quantitative methods, respectively, used to develop the PROMIS EC instruments. Here, we focus on elements of the process unique to developing the emotional distress measures. Data are available upon request.

Concept Specification

We hosted a half-day meeting with 15 experts in developmental psychology and psychopathology and measurement sciences, followed by 10 semistructured interviews with parents of 1–5 year olds. These were designed to review existing PROMIS Pediatric emotional distress domain frameworks, discuss their relevance for early childhood, and determine gaps and modifications for younger children. The three authors with specialized expertise in early childhood irritability (L.W.), depression (J.L.), and anxiety (K.B.) provided input throughout the concept specification and item writing processes. See Cella et al. (this issue and associated Supplementary Materials) for expert and participant details, interview guides, and the qualitative data analysis procedures.

Draft Item Pool Development

As described in Cella et al. (this issue), the existing PROMIS Pediatric instruments for 5–17 year olds guided our measurement framework. To ensure developmental sensitivity, it was also essential to not constrain the PROMIS EC measures to emotional distress features defined solely by those in older children. The developmental specification framework generated and validated by the senior author (L.W.) was specifically designed to combine lifespan coherence with characterization of developmentally specific features (Wakschlag et al., 2010). It served as the basis for the Multidimensional Assessment Profiles (MAPS) survey instruments, which have been validated on over 5,000 young children from community samples (Krogh-Jespersen et al., 2021; Wakschlag et al., 2018). For the PROMIS EC emotional distress measures, we drew on the MAP-Disruptive Behavior (MAP-DB) Temper Loss (irritability) scale (Krogh-Jespersen et al., 2021; Wakschlag et al., 2012, 2014) and the newly developed internalizing scales capturing anxious and depressive behaviors MAPS Internalizing (MAPS-INT; Wakschlag et al., 2021), collectively known as the MAPS-INT scales. The MAPS surveys were specifically derived to identify those features of dysregulation and context that are most informative for differentiating atypical patterns in early childhood and are consistent with other assessment methods created to characterize developmental phenotypes of early emotional distress (Buss et al., 2013; Luby, 2010; Petitclerc et al., 2015). The MAPS scales were originally designed to operationalize the NIMH Research Domain Criteria construct of the full normal: abnormal spectrum (Casey et al., 2014). As such, its scales are not consistent with the pragmatic PROMIS framework. However, based on clinical and predictive validity of this developmental approach (Damme et al., 2020; Wakschlag et al., 2015; Wiggins et al., 2021), we drew on the wording of MAPS items to developmentally enrich those modified from the PROMIS Pediatric item banks. Supplementary Materials A–C show how each existing item from the PROMIS Pediatric parent proxy measures was adapted for use in the PROMIS EC emotional distress measures.

Per PROMIS standards (PROMIS Cooperative Group, 2013), all draft items underwent translatability review (Devine et al., 2018) followed by cognitive interviews with at least five parents per item (see Cella et al., this issue, for participant recruitment information, including Table 3 for participant demographics) and Lexile reading-level analysis for each item. Across the three domains, only items at or below the sixth grade reading level were retained.

Table III.

Correlations for PROMIS EC Emotional Distress 8-Item Short Forms

Pearson correlation coefficients (Prob > |r| under H0: Rho = 0) PROMIS Anger/Irritability PROMIS Anxiety PROMIS Depression
SF-8 SF-8 SF-8
MAPS-INT
 General anxiety (N = 188) .37 (<.001) .81 (<.001)a .64 (<.001)
 Separation anxiety (N = 188) .41 (<.001) .51 (<.001)a .37 (<.001)
 Depression (N = 188) .35 (<.001) .55 (<.001) .71 (<.001)a
MAP-DB
 Temper loss (N = 188) .71 (<.001)a .44 (<.001) .43 (<.001)
CBCL
 Anxious/depressed T-score (N = 183) .25 (<.001) .40 (<.0001) a .30 (<.001)a
 Somatic complaints T-score (N = 183) .18 (.017) .27 (<.001) .23 (<.001)
 Withdrawn T-score (N = 183) .09 (.211) .25 (<.001) .26 (.<001)
 Internalizing T-score (N = 183) .37 (<.001) .51 (<.001) a .42 (<.001)a
 Externalizing T-score (N = 183) .55 (<.001)a .42 (<.001) .35 (<.001)
SDQ
 Emotional problems (N = 188) .12 (.094) .45 (<.001)a .28 (<.001)a
 Prosocial behavior (N = 188) −.18 (.013) −.16 (.024) −.29 (<.001)
ITSEA
 Inhibition to novelty (N = 188) .22 (.003) .43 (<.001) .35 (<.001)
 Separation distress (N = 174) .40 (<.001) .41 (<.001)a .28 (<.001)
 Anxiety (N = 188) .15 (.044) .42 (<.001)a .21 (.004)
 Depression/withdrawal (N = 188) .05 (.512) .19 (.009) .23 (.001)a
BITSEA
 Competence (N = 188) .12 (.104) .12 (.109) .02 (.817)
Open in a new tab

Note. Bold suggests convergent validity; only correlations >.40 were deemed adequate evidence of convergent validity. PROMIS EC = Patient-Reported Outcome Measurement Information System Early childhood; CBCL = Child Behavior Checklist; ITSEA = Infant Toddler Social Emotional Assessment; BITSEA = Brief-ITSEA.

a

A priori hypothesis of convergent validity.

Item Bank Development and Psychometric Evaluation

Two waves of panel surveys were conducted. We used data collected from the Wave 1 testing Form B (N = 700) to determine optimal items to be included in the final item banks, of which a subset was administered for Wave 2 testing (N = 1,057) to finalize item parameters and establish reference values. All items used the same item context (“In the past 7 days…”) and a 5-point Likert scale from 1 (Never) to 5 (Always). See Lai et al. (this issue) for sample characteristics, data collection procedures, and analytic approaches. Using Wave 1 data, we conducted exploratory and confirmatory factor analyses (EFA; CFA) to confirm unidimensionality of items within Anxiety, Anger/Irritability, and Depressive Symptoms item pools. We applied the graded response model (GRM; Samejima, 1997) to estimate item parameters and examined differential item functioning (DIF) between child age (1–2 vs. 3–5 years) and sex (female vs. male). We created the 8-item short-forms used in the Wave 2 testing by selecting a representative set of clinically relevant items while optimizing reliability.

Using the combined data from Waves 1 and 2, we estimated item parameters using multigroup GRM analyses, centering on the Wave 2 sample because it was a probability-based sample weighted to the U.S. Census Bureau’s March 2018 Current Population Survey. We estimated item parameters that were administered only in Wave 1 using a “fixed-parameter calibration” approach so that all items were on the same measurement continuum of their respective scales. We also simulated computer adaptive testing (CAT) administration and evaluated the mean and median number of items administered and Pearson correlations between CAT scores and full bank scores.

Across-Domain Associations and Known-Groups Validity

We examined associations between the emotional distress measures and other PROMIS EC domains using both Pearson r and Spearman rho correlations, as appropriate per measure score distribution. We used standard criteria to assess strength of associations: (r = 0, no correlation; r = below ± .10, low; r = ± .30, moderate; r ≥ ±.50, large; r = 1, perfect correlation; Cohen, 1988). We assessed known-group differences as follows: (a) Better versus worse general health (PROMIS EC Global Health T-scores [< 45 vs. ≥ 45]); (b) Parent-reported emotional/behavioral/developmental (EBD) condition (e.g., anxiety, Attention-Deficit/Hyperactivity Disorder, Intellectual Disability) versus not; and (c) Parent-reported physical or EBD condition versus not. We estimated mean group differences using one-way analyses of variance (ANOVAs) for each of the scales with two way-hypothesis tests. We estimated effect size with η 2 using the following criteria: small = 0.02–0.06 (exclusive), medium = 0.06 (inclusive)—0.14 (exclusive); and large ≥ 0.14 (Cohen, 1988).

Convergent Validity

The convergent validity analyses used a separate sample of participants from the When 2 Worry (W2W) study, which is comprised of diverse children recruited around 12 months of age and followed through preschool age (see Krogh-Jespersen et al., 2021; for detailed sample description). See Supplementary Material D for more information related to the W2W sample. A subset (N = 188) of W2W participants completed both the PROMIS EC emotional distress measures and previously validated parent-report questionnaires: MAPS-INT (Wakschlag et al., 2021) and MAP-DB (Krogh-Jespersen et al., 2021; Wakschlag et al., 2014); Child Behavior Checklist—Preschool version (CBCL; Achenbach & Rescorla, 2000); Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997); Infant Toddler Social Emotional Assessment (ITSEA; Carter et al., 2003); and the Brief-ITSEA (BITSEA; Briggs-Gowan & Carter, 2006). We estimated correlations between the PROMIS EC measures and subscales from these legacy measures. See Supplementary Material E for comparator reliabilities. Correlations related to specific hypotheses are discussed in the results section of this study. Correlations >.40 were regarded as evidence of convergent validity (Cohen, 1988). Other scales included as comparators were exploratory in nature and did not have prespecified hypotheses.

Results

Concept Specification

Experts agreed that the existing PROMIS emotional distress domain frameworks worked for younger children, with some modifications (detailed below). They also emphasized the unique importance of context given that many behaviors considered symptoms at older ages are normative in early childhood. Relevant contexts drawn from the MAP-DB included familiar and unfamiliar settings and varied interactional partners. Dysregulation (e.g., tantrum till exhausted) was also noted as a distinguishing feature. Additionally, experts discussed whether the traditional PROMIS 7-day recall period would be sufficient to avoid characterizing transient perturbations as a stable pattern, given the rapid pace of developmental change in early childhood. Experts agree to evaluate the recall period during cognitive interviews.

Anger/Irritability

Experts first decided the Anger domain should be renamed “Anger/Irritability” as the term “irritability” is more developmentally appropriate and of high salience for transdiagnostic prediction of mental health risk (Wakschlag et al., 2018). Experts used the extensively validated MAP-DB Temper Loss scale as a jumping off point to confirm and refine the three PROMIS Pediatric domain facets—angry mood (irritability, frustration), negative social cognitions (interpersonal sensitivity, envy, disagreeableness), and efforts to control anger. They confirmed the relevance of angry mood and efforts to control anger but did not view negative social cognition as appropriate for 1–5 year olds. Parents corroborated expert perspectives, with all 10 describing angry mood using such terms as “fussy,” “cranky,” “pouty,” “whiny,” “irritable,” and “temperamental.” Six parents, all of whom had children 19 months and older, also described their children’s effort to control anger. Some explained how their children “easily shift” out of their angry mood, whereas others described the opposite, saying their children are “quick to anger.” Parents did not discuss negative social cognition in the context of anger/irritability.

Additionally, experts suggested adding temper tantrums as a facet because they are the most discrete expression of anger and frustration in young children and occur in the majority of young children (Wakschlag et al., 2012). All 10 parents confirmed this, reporting temper tantrums as common behavioral manifestations of children’s anger/irritability. They described tantrums on a spectrum from mild (e.g., crying, crossing/folding arms) to moderate (e.g., yelling, stomping) to extreme (e.g., biting/harming self or others, banging head on the wall, hitting people, or destroying things).

Anxiety

Experts confirmed two of the four PROMIS Pediatric facets were relevant for early childhood—fear (fearfulness, panic) and anxious misery (worry, dread). They were unsure as to whether the hyperarousal (tension, nervousness, restlessness) and somatic symptoms (racing heart, dizziness) were appropriate or could be reliably captured via parent report for younger children. Experts also suggested adding a social/separation anxiety facet given its prevalence in early childhood and clear behavioral manifestations (e.g., inconsolable when separating from caregivers). They also noted how words that experts deem representative of anxiety may elicit different responses from parents. For example, one expert noted that if a parent saw the word “fearful,” they may respond that their child does not exhibit this behavior, but if the same parent saw the word “clingy,” they would endorse the item, regardless of clingy being a behavioral expression of fear. This suggested testing varied terms to determine those most discriminating.

Parents corroborated experts’ perspectives, with most discussing fear (N = 8)—using such descriptors as crying, having scared or panicked looks, or tone of voice as indicators of their children being scared—and anxious misery (N = 9)—using words such as “worried.” The majority of parents (N = 9) also discussed social/separation anxiety, particularly how their children were clingy and cried, especially with new people and in unfamiliar situations. Although hyperarousal was the least mentioned facet, most parents (N = 7) still discussed it, describing behaviors such as “jittery or fidgety,” “constantly on edge,” “not being able to sit still,” “heart racing,” and difficulty breathing (e.g., “lose her breath,” “try and catch her breath”).

Depressive Symptoms

Of the four PROMIS Pediatric facets, experts agreed negative mood was the most relevant for early childhood and thought negative views of self was important to capture for 3–5 year olds, though less relevant for the youngest children as self-concept begins to develop around age three. Similar to the Anxiety domain, they suggested negative social cognition was not appropriate. Experts also believed the decreased positive affect and engagement facet insufficiently captured the concept of anhedonia for early childhood and suggested additional emphasis on lack of engagement in playing and loss of pleasure. They also underscored the distinction between sadness, the focal expression for this domain, and not experiencing joy.

All parents (N = 10) described depressive symptoms in terms of negative mood, using phrases such as “always sad,” “withdrawn,” and “not talkative.” Most parents (N = 8) also described anhedonia as a component of depressive symptoms, describing behaviors such as refusing to eat and sleep, a lack of interest in normally fun activities such as playing with friends, and general malaise. Parents had a more difficult time describing negative views of self. Instead, some parents (N = 4), all of whom had children at least 25 months of age, described their children having positive self-views (e.g., “confident”), whereas only one parent noted her son was “not confident mentally.” Surprisingly, most parents (N = 7) discussed negative social cognition, using phrases such as “keeps to himself,” “isolation,” “lack of connectedness,” and “lonely.”

Draft Item Pool Development

Based on expert and parent input, as well as review of the existing PROMIS emotional distress domain frameworks, we identified four facets of Anxiety: fear (fearfulness, panic), anxious misery (worry/dread), hyperarousal (tension, nervousness), social/separation anxiety (distress when separating from caregivers); two facets of Anger/Irritability: angry/irritable mood (e.g., grouchiness), angry/irritable behavior (e.g., tantrums); and three facets of Depressive Symptoms: sad/withdrawn, negative views of self (self-criticism, low self-esteem), anhedonia (loss of interest, inability to engage in play, lack of enjoyment). Although parents reported on negative social cognition during interviews, experts questioned whether this facet was age-appropriate and did not feel that parents could reliably respond via survey (vs. interview) because the facet is less observable than the others. Furthermore, experts regarded negative social cognition as an improbable, emerging facet in early childhood—though some young children are capable of these behaviors, they are uninformative for clinical identification due to low prevalence as they are outside the developmental capacity of many children at this young age (Wakschlag et al., 2010, 2012, 2018).

We reviewed all items from PROMIS Pediatric measures for age appropriateness, with the goal of retaining items as is or with minor modification whenever possible. We also incorporated language from parent concept elicitation interviews. We drafted 53 items, of which the 49 new items underwent cognitive interviews (see Cella et al., this issue Table 3 for participant information) as four were existing items that were already tested (Irwin et al., 2010, 2012). Specifically, we drafted 20 Anger/Irritability items, including one verbatim PROMIS Pediatric item, and all new items were drawn from the MAP-DB Temper Loss (irritability) scale. Following cognitive interviews, 14 items were retained as is or with minor revisions, five items were retested and retained, two items were added and tested, and one item was dropped, for a total of 21 items for field testing. For Anxiety, of the 25 drafted items, including three verbatim PROMIS Pediatric items, 11 were retained as is or with minor modifications, 6 items were retested and retained, and 8 items were dropped, for a total of 17 items for field testing. For Depressive Symptoms, we drafted 16 items, of which 10 were retained as is or with minor modifications and 5 were dropped, resulting in a final item pool of 11 items. For all domains, reasons for removal included parent-perceived age-inappropriateness, overlap with other items, and lack of readability.

Item Bank Development and Psychometric Evaluation

EFAs for the Anger/Irritability, Anxiety, and Depressive Symptoms item pools resulted in ratios of the first to second eigenvalues equal to 9.45, 4.58, and 9.79, respectively. The first eigenvalues of the measures represented 62%, 62%, and 70% of the reliable variance, respectively. These results suggested that each of the three item banks was unidimensional. In total, 4 of the original 20 Anger/Irritability items were removed—1 had a residual correlation greater 0.20 and three others had modification index values ≥ 100 associated with their correlated errors terms. Four of the original 18 Anxiety items were removed due to having residual correlations > 0.20. One of the original 11 Depressive Symptoms items was removed because of a modification index value ≥ 100 associated with a correlated error term.

Results from the CFA models suggested that the single-factor models fit data from the respective scales well based on the a priori fit indices: Anger/Irritability, comparative fit index (CFI) = 0.99, Tucker-Lewis index (TLI) = 0.99, root mean square error of approximation (RMSEA) = 0.08; Anxiety, CFI = 0.97, TLI = 0.97, RMSEA = 0.10; Depressive Symptoms, CFI = 0.95, TLI = 0.95, RMSEA = 0.12. Although the RMSEA values for the three models were ≥0.08, the RMSEA has been shown to be sensitive to item skewness and scale length (Cook et al., 2009). The RMSEA threshold of 0.08 was relaxed based on previous work that has noted the sensibility of doing so when balancing fit with the inclusion of important content (Cook et al., 2009). Furthermore, each of the standardized factor loadings in the final CFA models > 0.60. No items had residual correlations > 0.20. At Wave 1, Cronbach’s alpha values for the full item banks for anger/irritability, anxiety, and depressive symptoms were .95, .96, and .95, respectively.

GRMs were fit for each of the emotional distress item banks—there were no items that exhibited poor fit across the three item sets. Moreover, DIF analyses did not result in the identification of any items that operated differentially across child age (3–5 vs. 1–2 year olds) or child sex (female vs. male).

Item Calibration, Scale Information, and Reliability

Using the Wave 2 sample as the representative sample for centering, multi-group item calibration analyses were conducted along with the combined Waves 1 and 2 sample for the 8-item anger/irritability, anxiety, and depressive symptoms short forms. See Table I for item descriptive statistics.

Table I.

Emotional Distress Measures Item Statistics

Wave 1
 Wave 2
Mean SD % floor (never) % ceiling (always) Mean SD % floor (never) % ceiling (always)
Anger/irritability
 My child broke or destroyed things during a temper tantruma 1.72 0.97 56.00 1.71 1.40 0.70 70.86 0.28
 My child became frustrated easilya 2.32 0.95 20.29 2.71 2.43 0.86 15.14 1.32
 My child had a temper tantruma 2.43 0.94 16.86 2.17 2.48 0.86 16.27 1.32
 My child lost his/her temper or had a temper tantrum when interacting with me or other parenta 2.18 0.99 30.43 2.00 2.26 0.86 21.19 0.95
 My child acted cranky 2.62 0.79 9.86 1.71 2.54 0.76 11.26 0.76
 My child became angry quickly 2.04 1.00 35.86 2.29 2.26 0.88 23.33 0.95
 My child stayed angry for a long time 1.60 0.87 59.29 1.29 1.49 0.70 60.26 0.28
 My child lost his/her temper or had a temper tantrum when out in public 1.85 0.95 45.57 1.71 1.74 0.82 47.59 0.38
 My child got mad 2.67 0.85 10.14 2.71
 My child had a temper tantrum when upset or angry 2.60 1.07 17.29 4.86
 My child screamed angrily at someone 2.19 1.03 31.29 2.43
 My child screamed loudly when frustrated 2.57 1.06 18.57 4.86
 When my child got mad, he/she stayed mad 1.79 0.86 43.43 1.14
 My child lost his/her temper 2.19 0.97 28.00 1.86
 My child had a hot/explosive temper 1.81 1.01 52.29 1.71
 My child had a temper tantrum till exhausted 1.67 0.93 58.00 1.29
Anxiety
 My child seemed scared or fearfula 1.78 0.92 47.43 1.57 1.56 0.70 55.72 0.09
 My child seemed nervousa 1.72 0.93 53.57 1.43 1.48 0.70 62.44 0.19
 My child seemed fearful or worried when out in publica 1.55 0.88 64.71 1.43 1.44 0.67 64.90 0.19
 My child was inconsolable when separating from me or other parent in a familiar settinga 1.78 1.00 53.57 1.86 1.52 0.77 62.72 0.38
 My child seemed worried 1.67 0.93 57.29 1.71 1.48 0.69 61.68 0.09
 My child seemed tense 1.53 0.86 65.86 0.86 1.54 0.69 57.14 0.09
 My child was fearful or worried at bedtime 1.52 0.84 66.14 0.71 1.43 0.69 68.12 0.19
 My child seemed fearful or worried at daycare, school, or other familiar settings 1.48 0.86 70.14 1.14 1.46 0.72 65.37 0.19
 My child got scared really easy 1.88 0.94 44.00 1.14
 My child talked about something really bad happening 1.62 0.93 62.00 0.86
 My child worried about what could happen to him/her 1.62 0.93 62.00 1.43
 My child woke up at night scared 1.72 0.94 54.86 1.00
 My child seemed fearful or worried at home 1.45 0.81 70.14 0.71
 My child froze because he/she was so scared 1.47 0.83 69.29 1.00
Depressive symptoms
 My child seemed sada 1.80 0.87 42.57 1.43 1.45 0.65 62.72 0.09
 My child was withdrawna 1.55 0.91 65.71 1.86 1.34 0.65 73.89 0.38
 My child wasn't interested in doing things he/she usually likesa 1.69 0.98 56.67 2.71 1.30 0.60 75.97 0.47
 My child acted sad during fun activitiesa 1.44 0.79 70.71 0.86 1.45 0.64 62.44 0.09
 My child kept crying even when I or other parent tried to comfort him/her 1.85 0.98 45.86 2.43 1.54 0.75 59.98 0.38
 My child had a hard time having fun 1.44 0.82 71.29 0.86 1.30 0.56 74.65 0.19
 My child seemed too sad to eat 1.31 0.73 80.00 0.71 1.20 0.49 83.54 0.09
 My child acted withdrawn when in a group of children 1.54 0.92 66.86 1.86 1.42 0.73 69.82 0.47
 My child seemed uninterested in eating food he/she usually likes 1.66 0.93 58.71 1.29
 My child said negative things about him/herself 1.55 0.91 66.29 0.86
Open in a new tab

Note. Bold = 8-item short form; all items used a 5-point Likert response scale: 1 (Never), 2 (Almost Never), 3 (Sometimes), 4 (Almost Always), and 5 (Always).

a

4-item short form.

Anger/Irritability

Item discrimination values ranged from 1.74 (“My child broke or destroyed things during a temper tantrum.”) to 2.97 (“My child lost his/her temper or had a temper tantrum when interacting with me or other parent.”). Threshold parameters ranged from −1.76 to 3.67. Based on response pattern scores, Waves 1 and 2 had marginal reliability estimates for the 8-item form equal to 0.91 and 0.90, respectively. Score-level reliability was ≥ 0.90 for Waves 1 and 2 from theta = −0.8 to +2.8.

Anxiety

Item discrimination values ranged from 1.39 (“My child was inconsolable when separating from me or other parent in a familiar setting.”) to 3.62 (“My child seemed worried.”). Threshold parameters ranged from 0.14 to 4.19. Based on response pattern scores, Waves 1 and 2 had marginal reliability estimates for the 8-item form equal to 0.82 and 0.79, respectively. Score-level reliability was ≥ 0.90 for Waves 1 and 2 from theta = 0 to 2.8.

Depressive Symptoms

Item discrimination values ranged from 1.99 (“My child kept crying even when I or other parent tried to comfort him/her.”) to 4.04 (“My child had a hard time having fun.”). Threshold parameters ranged from 0.28 to 3.47. Cronbach’s alpha for the 8-item anxiety measure was .90. Based on response pattern scores, Waves 1 and 2 had marginal reliability estimates for the 8-item form equal to 0.81 and 0.74, respectively. Score-level reliability was ≥ 0.90 for Waves 1 and 2 from theta = +0.4 to +2.8.

Across-Domain Associations and Known-Groups Analyses

The PROMIS EC emotional distress measures were all moderately negatively correlated with PROMIS EC Global Health (−0.32 to −0.36) and the PROMIS EC well-being (−0.18 to −0.45) measures. Alternatively, PROMIS EC Sleep Problems was positively correlated with the emotional distress measures (0.42–0.52), such that children with worse sleep have worse anxiety, anger/irritability, and depressive symptoms (see Table 4 in Lai et al., this issue, for full correlation matrix).

Table II includes the results from individual ANOVAs using the continuous emotional distress T-scores and three known-groups: high/low general health, a diagnosed EBD condition, and any chronic condition. Each of the emotional distress measures was significantly associated with EBD, any chronic condition, and general health values higher than their respective median values. These analyses indicated that young children with EBD and any chronic health conditions had small differences on the PROMIS EC emotional distress measures compared with their typically developing peers. However, medium-sized differences in the PROMIS EC emotional distress scales existed between children with compared with without general health scores below the median.

Table II.

Known-Groups Analyses Across PROMIS EC Emotional Distress Measures

Factor Score Group N Mean SD p-Value η2
Global Health Anxiety T-score Low 347 53.2 9.11
High 710 48.3 8.36 <.001 0.07
Anger/Irritability T-score Low 347 53.3 9.22
High 710 48.4 9.11 <.001 0.06
Depressive Symptoms T-score Low 347 53.7 9.09
High 710 48.2 7.98 <.001 0.09
EBD condition Anxiety T-score No 927 49.4 8.63
Yes 110 54.1 9.76 <.001 0.03
Anger/Irritability T-score No 927 49.3 9.21
Yes 110 54.6 9.93 <.001 0.03
Depressive Symptoms T-score No 927 49.4 8.46
Yes 110 54.2 9.65 <.001 0.03
Any condition Anxiety T-score No 770 49.2 8.56
Yes 285 52.4 9.37 <.001 0.05
Anger/Irritability T-score No 770 49.2 9.06
Yes 285 52.0 10.15 <.001 0.04
Depressive Symptoms T-score No 770 49.1 8.24
Yes 285 52.1 8.72 <.001 0.04
Open in a new tab

Note. EBD = emotional/behavioral/developmental; PROMIS EC = Patient-Reported Outcome Measurement Information System Early Childhood.

Convergent Validity

See Table III for correlations between the PROMIS EC emotional distress measures and external validation measures. See Supplementary Materials F–H for correlations between comparators and each of the three PROMIS EC emotional distress measures.

Anger/Irritability

As hypothesized, the PROMIS EC Anger/Irritability forms were most highly correlated and demonstrated evidence of convergent validity with the MAP-DB Temper Loss (0.71) and CBCL Externalizing Symptoms scales (0.55). There was also evidence of convergent validity between the PROMIS EC Anger/Irritability forms and the separation anxiety scales from the MAPS-INT and ITSEA (≥0.40).

Anxiety

The 8-item PROMIS EC Anxiety short form had positive correlations ≥0.40 with each of the hypothesized comparators—MAPS-INT General Anxiety, MASP-INT-Separation Anxiety, CBCL Anxious/Depressed, CBCL Internalizing, SDQ Emotional Problems, ITSEA Separation Distress, and ITSEA Anxiety scales—providing evidence of convergent validity with these measures. The high correlation with the MAPS-INT General Anxiety scale is not surprising; seven of the eight PROMIS EC Anxiety items are contained in the MAPS-INT General Anxiety, albeit with different response scales. The 8-item Anxiety short form also had moderate correlations with the MAP-DB Temper Loss (0.44) scale.

Depressive Symptoms

As hypothesized, the 8-item PROMIS EC Depressive Symptoms short form had positive correlations >0.40 with the MAPS-INT Depression and the CBCL Internalizing scales. Although we hypothesized that the correlations between the PROMIS EC Depressive Symptoms measure and the CBCL Anxious/Depressed, CBCL Withdrawn, SDQ Emotional Problems, and ITSEA Depression/Withdrawal scales would be >0.40, they were all between 0.23 and 0.40. However, it is worth noting that the sample reliabilities for the ITSEA Depression/Withdrawal were only 0.50. As was the case with PROMIS EC Anxiety, all of the PROMIS EC Depressive Symptoms items are contained in the MAPS-INT Depressive Symptoms scale, albeit with different response scales. The 8-item PROMIS EC Depressive Symptoms short form also had moderate correlations with the MAP-DB Temper Loss (0.43) scales.

Discussion

We developed, calibrated, and evaluated the reliability and validity of PROMIS measures of emotional distress that now cover from 5 years and older, down to 1 year of age. These new developmentally based measures provide a PROMIS instrument for emotional distress assessment in young children. These early childhood assessments are conceptually linked to parallel measures in older children, thus providing a common emotional distress framework for ages 1–17. Through the application of the MAPS framework to the PROMIS measure development process, these item banks were conceptually aligned with the PROMIS Pediatric parent proxy Anger, Anxiety, and Depressive Symptoms domains. Initial item banks were developed through expert meetings and parent concept elicitation and cognitive interviews. Finally, full-item banks, which can be used in CAT administration, as well as 4- and 8-item short forms were constructed for the three domains based on rigorous IRT analyses after ensuring scale unidimensionality and the absence of local independence and DIF across sex and age.

Having validated these three domains for the early childhood life stage, researchers now have access to brief, developmentally sensitive and pragmatic measures of emotional distress that reliably cover the entire pediatric period providing a tool that can be used continuously in the monitoring of emotional distress for children who develop early onset pediatric conditions. Researchers interested in studying the intersections between health and emotional distress in early childhood and later life stages are encouraged to make use of these measures, which offer a variety of forms that can be administered both cross-disease and across developmental periods and are highly feasible for use in primary care.

The increasing salience of early emotional distress and its prevention for pediatric outcomes is evident from a burgeoning literature. Irritability (captured by the PROMIS EC Anger/Irritability instrument) has transdiagnostic utility for both mood and behavioral problems and is a high salience comorbidity with many pediatric illnesses, making it an efficient way for providers to identify broad risk of impairing problems associated with emotional distress (Beauchaine & Tackett, 2020; Price et al., 1990; Wakschlag et al., 2019). It is especially useful for flagging signs of early vulnerability to these because it is measurable from infancy as we have demonstrated here. Anxiety frequently co-occurs with pediatric conditions and is exacerbated for those that require frequent medical procedures. Pragmatic preventions have been validated for early childhood anxiety prevention; these can provide critical coping tools to improve adherence and quality of life for such children (Rapee, 2002). Early childhood depressive behaviors may also occur in response to parental stress and depression exacerbated by challenging medical conditions and or amplifying the likelihood of these, and may represent early indication of an intergenerational risk pathway (Leiferman, 2002; Raposa et al., 2014).

Limitations and Future Directions

A major strength of the PROMIS EC emotional distress measures is that they were validated on a sample drawn from the general U.S. population recruited via address-based probability sampling, which enables individual and group comparisons to the “average” 1- to 5-year-old U.S. child. Furthermore, their intended use as cross-condition indicators also heightens importance due to frequent comorbidities and the opportunity this provides for clinical trial comparisons that transcend a disease area. However, this study does not guarantee that these measures are reliable among all subpopulations. Future work should evaluate the generalizability of these instruments to children with specific chronic conditions and in other countries, which is already ongoing (De Young et al., 2021; Vasileva et al., 2021).

Comprehensively assessing developmental functioning and emotional distress in young children is complex and requires a multi-faceted approach as has been extensively detailed in the literature (Bufferd et al., 2016; Wakschlag et al., 2005, 2010). Multi-method assessments are often useful in providing comprehensive information about child biobehavioral functioning and family context and history. However, driven by the pragmatic principles that are the backbone of PROMIS and more recently incorporated into developmental measurement frameworks (Morris et al., 2020), the level and nature of assessments must be clearly tied to the objectives of a study or setting. In pediatric psychology, brief, efficient patient-reported outcomes that are feasible for clinical use are fundamental. This does not preclude the need for more in-depth assessment for some children. Rather, surveys such as those in the PROMIS EC emotional distress instruments were designed to identify levels of emotional distress, capturing key features in a developmentally meaningful manner, relative to same age, representative peers. They are not intended to be diagnostic, and their use does not preclude additional assessments (indeed, it may point to the need for more in-depth assessments). Importantly, the PROMIS EC emotional distress measures were not designed to provide etiologic information (e.g., neurologic mechanisms). They are intended to provide a method for assessing young children’s emotional distress in a manner that is developmentally meaningful and feasible for clinical follow-up. This express intent will allow for the efficient monitoring of changes over time in response to treatment or developmental course. It is also critical to underscore that there are now many survey measures validated to assess psychopathology-related constructs in young children (e.g., ITSEA, SDQ, etc.), so the constructs themselves are not in question (Carter et al., 2004). Rather, the PROMIS EC emotional distress measures build on this sound evidence base to expand the reach and utility of the widely used PROMIS instruments. Examining the clinical and incremental value of the PROMIS EC emotional distress measures in relation to more intensive measures using modeling methods that identify when a level of measurement adds clinical utility (beyond statistical significance) in determining outcomes (Luby et al., 2019) will be an important area for future research. See Cella et al. (this issue) and Lai et al. (this issue) for broader discussions of the limitations associated with the qualitative and quantitative methods, respectively, used to create the PROMIS EC emotional distress measures.

Further, drawing on the developmental specification framework, a unique feature of the PROMIS EC measures, is the inclusion of items that are contextually bound, which is key to early childhood characterization. Understanding the express intent of the PROMIS EC emotional distress measures, researchers are encouraged to use these instruments in the context of larger studies to investigate their associational and predictive relationships with related constructs of interest. This point is particularly important given the high correlations observed between the PROMIS EC Anxiety and Depressive Symptoms measures—future research can help determine whether separate values for each construct, versus a single overarching score will have more clinical or predictive value. More work is also needed to understand the effects of specific environmental factors (i.e., vulnerabilities) and their interactions and reinforcement of emotional distress dimensions and their associated outcomes. Furthermore, previous research has highlighted the importance of the child–parent, interactional perspective in understanding child behavior (Bornstein, 2013; Collins et al., 2000; Dennis, 2006). To this end, researchers interested in exploring causal mechanisms for early child emotional distress should consider including parent-child attachment styles, child neurological maturity, family context, and parental mental status. Access to specialized mental health assessments and services is associated with significant health disparities, and the use of a pragmatic tool, like the PROMIS EC emotional distress instruments, which can be easily implemented in a broad range of health settings will advance equity in identification, prevention, and treatment (Alegría et al., 2015). Another important direction will be to psychometrically establish lifespan coherence of these early childhood measures with parallel PROMIS measures in older children, to provide statistical anchoring for the conceptual links achieved in this work. This would amplify the utility of PROMIS for the field of pediatric psychology, providing sound measurement of emotional distress constructs that often co-occur with pediatric health conditions (e.g., eczema, diabetes) and are key to their effective management (Hilliard et al., 2012; Schmitt et al., 2011). Such work could enable longitudinal follow-up throughout childhood, using common constructs of emotional distress.

Supplementary Data

Supplementary data can be found at: https://academic.oup.com/jpepsy.

Funding

Research reported in this publication was supported by the Environmental influences on Child Health Outcomes (ECHO) program, Office of The Director, National Institutes of Health, under Award Number U24OD023319 with cofunding from the Office of Behavioral and Social Sciences Research (OBSSR; Person-Reported Outcomes Core). The When to Worry Study which provided additional validation data were supported by NIMH R01MH107652.

Conflicts of interest: None declared.

Supplementary Material

jsac029_Supplementary_Data
Click here for additional data file. (46.6KB, docx)

References

  1. Achenbach T. M., Rescorla L. A. (2000). Manual for the ASEBA preschool-age forms & profiles. University of Vermont Research Center for Children, Youth, & Families.
  2. Alegría M., Green J. G., McLaughlin K. A., Loder S. (2015). Disparities in child and adolescent mental health and mental health services in the US . William T. Grant Foundation. https://wtgrantfoundation.org/library/uploads/2015/09/Disparities-in-Child-and-Adolescent-Mental-Health.pdf. Retrieved 1 March 2022.
  3. Alink L. R. A., Mesman J., Van Zeijl J., Stolk M. N., Juffer F., Koot H. M., Bakermans-Kranenburg M. J., Van IJzendoorn M. H. (2006). The early childhood aggression curve: Development of physical aggression in 10- to 50-month-old children. Child Development, 77(4), 954–966. 10.1111/j.1467-8624.2006.00912.x [DOI] [PubMed] [Google Scholar]
  4. Beauchaine T. P., Tackett J. L. (2020). Irritability as a transdiagnostic vulnerability trait: Current issues and future directions. Behavior Therapy, 51(2), 350–364. 10.1016/j.beth.2019.10.009 [DOI] [PubMed] [Google Scholar]
  5. Blackwell C. K., Wakschlag L. S., Krogh-Jespersen S., Buss K. A., Luby J., Bevans K., Lai J.-S., Forrest C. B., Cella D. (2020). Pragmatic health assessment in early childhood: The PROMIS® of developmentally based measurement for pediatric psychology. Journal of Pediatric Psychology, 45(3), 311–318. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bornstein M. H. (2013). Parenting and child mental health: A cross-cultural perspective. World Psychiatry, 12(3), 258–265. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Briggs-Gowan M., Carter A. (2006). ITSEA/BITSEA: Infant-toddler and brief infant-toddler social and emotional assessment. The Psychological Corporation. [Google Scholar]
  8. Briggs-Gowan M. J., Carter A. S., Irwin J. R., Wachtel K., Cicchetti D. V. (2004). The brief infant-toddler social and emotional assessment: Screening for social-emotional problems and delays in competence. Journal of Pediatric Psychology, 29(2), 143–155. 10.1093/jpepsy/jsh017 [DOI] [PubMed] [Google Scholar]
  9. Bufferd S. J., Dyson M. J., Hernandez I. G., Wakschlag L. S. (2016). Explicating the “developmental” in preschool psychopathology. In D. Cicchetti (Ed.), Developmental psychopathology: Maladaptation and psychopathology (pp. 152–186). John Wiley & Sons, Inc. [Google Scholar]
  10. Buss K. A., Davis E. L., Kiel E. J., Brooker R. J., Beekman C., Early M. C. (2013). Dysregulated fear predicts social wariness and social anxiety symptoms during kindergarten. Journal of Clinical Child and Adolescent Psychology, 42(5), 603–616. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Carbonneau R., Boivin M., Brendgen M., Nagin D., Tremblay R. E. (2016). Comorbid development of disruptive behaviors from age 1½ to 5 years in a population birth-cohort and association with school adjustment in first grade. Journal of Abnormal Child Psychology, 44(4), 677–690. 10.1007/s10802-015-0072-1 [DOI] [PubMed] [Google Scholar]
  12. Carter A. S., Briggs-Gowan M. J., Davis N. O. (2004). Assessment of young children's social-emotional development and psychopathology: Recent advances and recommendations for practice. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 45(1), 109–134. 10.1046/j.0021-9630.2003.00316.x [DOI] [PubMed] [Google Scholar]
  13. Carter A. S., Briggs-Gowan M. J., Jones S. M., Little T. D. (2003). The Infant–Toddler Social and Emotional Assessment (ITSEA): Factor structure, reliability, and validity. Journal of Abnormal Child Psychology, 31(5), 495–514. 10.1023/A:1025449031360 [DOI] [PubMed] [Google Scholar]
  14. Carter A. S., Gray S. A., Baillargeon R. H., Wakschlag L. S. (2013). A multidimensional approach to disruptive behaviors: Informing lifespan research from an early childhood perspective. In Tolan P.,Leventhal B. (Eds.), Disruptive behavior disorders (pp. 103–135). Springer. [Google Scholar]
  15. Casey B., Oliveri M., Insel T. (2014). A neurodevelopmental perspective on the Research Domain (RDoC) framework. Biological Psychiatry, 76(5), 350–353. [DOI] [PubMed] [Google Scholar]
  16. Cella D., Yount S., Rothrock N., Gershon R., Cook K., Reeve B., Ader D., Fries J. F., Bruce B., Rose M.; PROMIS Cooperative Group. (2007). The Patient-Reported Outcomes Measurement Information System (PROMIS): Progress of an NIH roadmap cooperative group during its first two years. Medical Care, 45(5 Suppl 1), S3–S11. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2829758/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Cohen S. (1988). Statistical power analysis for the behavioral sciences. Routledge Academic. [Google Scholar]
  18. Collins W. A., Maccoby E. E., Steinberg L., Hetherington E. M., Bornstein M. H. (2000). Contemporary research on parenting: The case for nature and nurture. American Psychologist, 55(2), 218–232. [PubMed] [Google Scholar]
  19. Cook K. F., Kallen M. A., Amtmann D. (2009). Having a fit: Impact of number of items and distribution of data on traditional criteria for assessing IRT’s unidimensionality assumption. Quality of Life Research, 18(4), 447–460. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Damme K., Wakschlag L. S., Briggs-Gowan M., Norton E., Mittal V. (2020). Developmental patterning of irritability enhances prediction of psychopathology in pre-adolescence: Improving RDoC with developmental science. bioRxiv. https://doi.org/10.1101/2020.04.30.070714 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. De Young A. C., Vasileva M., Boruszak-Kiziukiewicz J., Demipence Seçinti D., Christie H., Egberts M. R., Anastassiou-Hadjicharalambous X., Marsac M. L., Ruiz G.; COVID-19 Unmasked Global Collaboration. (2021). COVID-19 Unmasked Global Collaboration, 2021. COVID-19 Unmasked Global Collaboration Protocol: Longitudinal cohort study examining mental health of young children and caregivers during the pandemic. European Journal of Psychotraumatology, 12(1), 1940760. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Dennis T. (2006). Emotional self-regulation in preschoolers: The interplay of child approach reactivity, parenting, and control capacities. Developmental Psychology, 42(1), 84–97. [DOI] [PubMed] [Google Scholar]
  23. Devine J., Klasen F., Moon J., Herdman M., Hurtado M. P., Castillo G., Haller A. C., Correia H., Forrest C. B., Ravens-Sieberer U. (2018). Translation and cross-cultural adaptation of eight pediatric PROMIS® item banks into Spanish and German. Quality of Life Research, 27(9), 2415–2430. [DOI] [PubMed] [Google Scholar]
  24. DeWalt D. A., Gross H. E., Gipson D. S., Selewski D. T., DeWitt E. M., Dampier C. D., Hinds P. S., Huang I. C., Thissen D., Varni J. W. (2015). PROMIS® pediatric self-report scales distinguish subgroups of children within and across six common pediatric chronic health conditions. Quality of Life Research, 24(9), 2195–2114. 10.1007/s11136-015-0953-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Egger H. L., Angold A. (2006). Common emotional and behavioral disorders in preschool children: Presentation, nosology, and epidemiology. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 47(3–4), 313–337. 10.1111/j.1469-7610.2006.01618.x [DOI] [PubMed] [Google Scholar]
  26. Goodman R. (1997). The Strengths and Difficulties Questionnaire: A research note. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 38(5), 581–586. 10.1111/j.1469-7610.1997.tb01545.x [DOI] [PubMed] [Google Scholar]
  27. Hay D. F., Waters C. S., Perra O., Swift N., Kairis V., Phillips R., Jones R., Goodyer I., Harold G., Thapar A., van Goozen S. (2014). Precursors to aggression are evident by 6 months of age. Developmental Science, 17(3), 471–480. 10.1111/desc.12133 [DOI] [PubMed] [Google Scholar]
  28. Hilliard M. E., Harris M. A., Weissberg-Benchell J. (2012). Diabetes resilience: A model of risk and protection in type 1 diabetes. Current Diabetes Reports, 12(6), 739–748. [DOI] [PubMed] [Google Scholar]
  29. Irwin D., Stucky B., Langer M., Thissen D., DeWitt E., Lai J.-S., Yeatts K., Varni J., DeWalt D. (2012). PROMIS Pediatric Anger Scale: An item response theory analysis. Quality of Life Research, 21(4), 697–706. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Irwin D. E., Stucky B., Langer M. M., Thissen D., Dewitt E. M., Lai J. S., Varni J. W., Yeatts K., DeWalt D. A. (2010). An item response analysis of the pediatric PROMIS anxiety and depressive symptoms scales. Quality of Life Research, 19(4), 595–607. 10.1007/s11136-010-9619-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Krogh-Jespersen S., Kaat A., Gray L., Petitclerc A., Burns J., Adam H., Briggs-Gowan M., Wakschlag L. S. (2021). Calibrating tantrum severity in the transition to toddlerhood: Implications for developmental science. Applied Developmental Science. Advance online publication, 1–14. https://doi.org/10.1080/10888691.2021.1995386 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Leiferman J. (2002). The effect of maternal depressive symptomatology on maternal behaviors associated with child health. Health Education & Behavior, 29(5), 596–607. [DOI] [PubMed] [Google Scholar]
  33. Luby J., Allen N., Estabrook R., Pine D. S., Rogers C., Krogh-Jespersen S., Norton E. S., Wakschlag L. S. (2019). Mapping infant neurodevelopmental precursors of mental disorders: How synthetic cohorts & computational approaches can be used to enhance prediction of early childhood psychopathology. Behaviour Research and Therapy, 123, 103484. 10.1016/j.brat.2019.103484 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Luby J. L. (2010). Preschool depression: The importance of identification of depression early in development. Current Directions in Psychological Science, 19(2), 91–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Morris A. S., Wakschlag L. S., Krogh-Jespersen S., Fox N., Planalp B., Perlman S. B., Shuffrey L. C., Smith B., Lorenzo N. E., Amso D., Coles C. D., Johnson S. P. (2020). Principles for Guiding the Selection of Early Childhood Neurodevelopmental Risk and Resilience Measures: HEALthy Brain and Child Development Study as an Exemplar. Adversity and Resilience Science, 1(4), 247–221. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Petitclerc A., Briggs‐Gowan M. J., Estabrook R., Burns J. L., Anderson E. L., McCarthy K. J., Wakschlag L. S. (2015). Contextual variation in young children’s observed disruptive behavior on the DB‐DOS: Implications for early identification. Journal of Child Psychology and Psychiatry and Allied Disciplines 56(9), 1008–1016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Price C. E., Rona R. J., Chinn S. (1990). Associations of excessive irritability with common illnesses and food intolerance. Paediatric and Perinatal Epidemiology, 4(2), 156–160. 10.1111/j.1365-3016.1990.tb00632.x [DOI] [PubMed] [Google Scholar]
  38. PROMIS Cooperative Group. (2013). PROMIS® Instrument Development and Validation Scientific Standards Version 2.0. https://www.healthmeasures.net/images/PROMIS/PROMISStandards_Vers2.0_Final.pdf.
  39. Rapee R. (2002). The development and modification of temperamental risk for anxiety disorders: Prevention of a lifetime of anxiety? Biological Psychiatry, 52(10), 947–957. [DOI] [PubMed] [Google Scholar]
  40. Raposa E., Hammen C., Brennan P., Najman J. (2014). The long-term effects of maternal depression: Early childhood physical health as a pathway to offspring depression. The Journal of Adolescent Health, 54(1), 88–93. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Samejima F. (1997). The graded response model. In van der Linden W. J., Hambleton R. (Eds.), Handbook of modern item response theory. (pp. 85–100). Springer-Verlag. [Google Scholar]
  42. Schmitt J., Chen C. M., Apfelbacher C., Romanos M., Lehmann I., Herbarth O., Schaaf B., Kraemer U., Von Berg A., Wichmann H. E., Heinrich J.; LISA‐plus Study Group. (2011). Infant eczema, infant sleeping problems, and mental health at 10 years of age: The prospective birth cohort study LISAplus. Allergy, 66(3), 404–411. [DOI] [PubMed] [Google Scholar]
  43. Varni J. W., Magnus B., Stucky B. D., Liu Y., Quinn H., Thissen D., Gross H. E., Huang I. C., DeWalt D. A. (2014). Psychometric properties of the PROMIS® pediatric scales: Precision, stability, and comparison of different scoring and administration options. Quality of Life Research, 23(4), 1233–1243. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Vasileva M., Alisic E., De Young A. C. (2021). COVID-19 unmasked: Preschool children’s negative thoughts and worries during the COVID-19 pandemic in Australia. European Journal of Psychotraumatology, 12(1), 1924442. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Wakschlag L. S., Briggs-Gowan M. J., Choi S. W., Nichols S. R., Kestler J., Burns J. L., Carter A. S., Henry D. (2014). Advancing a multidimensional, developmental spectrum approach to preschool disruptive behavior. Journal of the American Academy of Child and Adolescent Psychiatry, 53(1), 82–96.e83. 10.1016/j.jaac.2013.10.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Wakschlag L. S., Choi S., Carter A., Hullsiek H., Burns J., McCarthy K., Leibenluft E. M., Briggs-Gowan M. (2012). Defining the developmental parameters of temper loss in young children: Implications for developmental psychopathology. Journal of Child Psychology and Psychiatry, 53(11), 1099–1108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Wakschlag L. S., Estabrook R., Petitclerc A., Henry D., Burns J., Perlman S., Voss J., Pine D., Leibenluft L., Briggs-Gowan M. (2015). Clinical implications of a dimensional approach: The normal:abnormal spectrum of early irritability. Journal of the American Academy of Child and Adolescent Psychiatry, 54(8), 626–634. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Wakschlag L. S., Leventhal B. L., Briggs-Gowan M. J., Danis B., Keenan K., Hill C., Egger H. L., Cicchetti D., Carter A. S. (2005). Defining the “disruptive” in preschool behavior: What diagnostic observation can teach us. Clinical Child and Family Psychology Review, 8(3), 183–201. 10.1007/s10567-005-6664-5 [DOI] [PubMed] [Google Scholar]
  49. Wakschlag L. S., Perlman S. B., Blair R. J., Leibenluft E., Briggs-Gowan M. J., Pine D. S. (2018). The neurodevelopmental basis of early childhood disruptive behavior: Irritable and callous phenotypes as exemplars. The American Journal of Psychiatry, 175(2), 114–130. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Wakschlag L. S., Roberts M. Y., Flynn R. M., Smith J. D., Krogh-Jespersen S., Kaat A. J., Gray L., Walkup J., Marino B. S., Norton E. S., Davis M. M. (2019). Future directions for early childhood prevention of mental disorders: A road map to mental health, earlier. Journal of Clinical Child and Adolescent Psychology, 48(3), 539–554. 10.1080/15374416.2018.1561296 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Wakschlag L. S., Sherlock P., Blackwell C. K., Krogh-Jespersen S., Cella D., Gershon R., Buss K., Luby J. (2021). The normal:abnormal spectrum of early childhood internalizing behaviors: A clinical-developmental approach. Under review. [DOI] [PMC free article] [PubMed]
  52. Wakschlag L. S., Tolan P. H., Leventhal B. L. (2010). Ain’t misbehavin’: Towards a developmentally-specified nosology for preschool disruptive behavior. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 51(1), 3–22. 10.1111/j.1469-7610.2009.02184.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Wiggins J., Briggs-Gowan M., Brotman M., Leibenluft E., Wakschlag L. S. (2021). Towards a developmental nosology for disruptive mood dysregulation disorder (DMDD) in early childhood. Journal of the American Academy of Child & Adolescent Psychiatry, 60(3), 388–397. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

jsac029_Supplementary_Data
Click here for additional data file. (46.6KB, docx)

Articles from Journal of Pediatric Psychology are provided here courtesy of Oxford University Press

RESOURCES