NIH Toolbox® Emotion Batteries for Children: Factor-Based Composites and Norms

Abstract

The recently released National Institutes of Health Toolbox (NIHTB) batteries for neurological and behavioral function were designed to serve as standardized, common measures in clinical and epidemiological research. The current study aimed to examine constructs assessed by the self-report and parental proxy-report scales in the NIHTB Emotion Battery (NIHTB-EB) for Children by using factor analyses on data from the U.S. national normative sample of 2,916 English-speaking children. This battery contains 31 scales designed to assess both positive and negative aspects of social and emotional functioning that are considered developmentally relevant at each of three age ranges (3-7, 8-12, and 13-17 years). Results revealed four similar self-report factors for ages 8 to 12 years and 13 to 17 years. Proxy reports for ages 3 to 7 years revealed three factors, and for ages 8 to 12 years two factors. Based on the standardization sample data, age- and gender-corrected norms are presented for all NIHTB-EB individual scales and factor-based composites.

Mood, emotionality, and affective tendencies are consistently shown to be related to important factors in children’s lives, such as school performance (e.g., Graziano, Reavis, Keane, & Calkins, 2007; Gumora & Arsenio, 2002), social competence (e.g., Lemerise & Arsenio, 2000), and risk for developing mood and anxiety disorders in early adolescence to adulthood (e.g., Roza, Hofstra, van der Ende, & Verhulst, 2003). These findings highlight the importance of assessing emotional functioning throughout childhood. The costs of using most well-known, proprietary measures (e.g., Child Behavior Checklist; Achenbach & Rescorla, 2000, 2001), however, limit their use in many clinical and research settings. This, in turn, may prevent parents from receiving appropriate emotional assessments and subsequent treatments for their children.

One promising, nonproprietary alternative for assessing emotional functioning is the NIH Toolbox® Emotion Battery (NIHTB-EB), which constitutes a component of the comprehensive NIH Toolbox for the Assessment of Neurological and Behavioral Function. As with the other components of the NIHTB (cognition, motor, and sensory), the NIHTB-EB was created for the purpose of establishing a set of easily administered, highly accessible, computerized assessments to use across studies and across a wide age range (3-85 years; Gershon et al., 2010; Gershon et al., 2013). The battery is also meant to promote the use of reliable and consistent scores in longitudinal and meta-analytic research. NIHTB measures have several advantages over traditional paper-and-pencil measures that can enhance quality and ease of clinical research, including user-friendly iPad administrations taking no more than 30 minutes to complete, state-of-the-art scale development (described previously; Salsman et al., 2013), automatic calculation of normed and composite scores, secure local or cloud-based data storage, and national conorming with all other NIHTB domain assessments (“NIH Toolbox® Scoring and Interpretation Guide,” 2016). The NIHTB norms were derived from the nationally representative normative sample, recruited from 10 sites across the United States (Los Angeles, CA; Columbus, OH; Philadelphia, PA; Cincinnati, OH; Atlanta, GA; Phoenix, AZ; Minneapolis, MN; St. Louis, MO; Dallas, TX; and Chicago, IL). Although the sampling plan was stratified by only age, sex, and preferred language (English or Spanish), the final sample contained proportions of race/ethnicity very similar to that of the 2010 U.S. Census (see Method section; U.S. Census Bureau, 2011). The NIHTB-EB additionally is quite comprehensive, as it includes 31 measures that assess a wide range of social and emotional functioning domains. Unlike other common measures of child emotional functioning that focus only on problematic (i.e., negative) emotional functioning, the NIHTB-EB importantly assesses both negative and positive functioning, consistent with the well-established theory that negative and positive emotions are relatively independent constructs that do not exist simply as opposite ends of one continuum (Watson & Tellegen, 1985). Furthermore, paper-and-pencil versions of individual measures are free to use and can be easily obtained from the NIH Toolbox website (http://www.healthmeasures.net/explore-measurement-systems/nih-toolbox/obtain-and-administer-measures). Nonproprietary instruments encourage open science, and more readily allow the study of psychometric properties. These qualities make the NIHTB an excellent resource for use in clinical research.

The NIHTB-EB assesses four theoretically defined constructs considered fundamental to emotional functioning: (a) negative affect, (b) psychological well-being, (c) stress and self-efficacy, and (d) social relationships (Gershon et al., 2013; Nowinski, Victorson, Debb, & Gershon, 2013; Salsman et al., 2013). The items and scales that comprise the NIHTB-EB were either selected from existing measures (based on psychometric soundness, ease of use, and applicability across age ranges and diverse groups) or newly developed by building on existing items from multiple surveys (see Gershon et al., 2013; Salsman et al., 2013, for details regarding the process of selecting and developing appropriate instruments).

To reliably and validly assess emotional functioning across the life span, the NIHTB-EB consists of different batteries of tests for adults (aged 18-85 years), and for children grouped by various age ranges based on expert judgment about age-appropriate emotional experiences and expression (Salsman et al., 2013). The NIHTB-EB for adults contains 17 individual self-report measures. The NIHTB-EB for children younger than the age of 18 years contains different measures for narrower age groups. See Table 1 for details regarding which measures are included for the assessment of each youth age group. Children aged 8 to 17 years receive self-report measures, while children aged 3 to 12 years are assessed via proxy-report measures (usually completed by the mother). Of note, this results in the collection of both self- and proxy-report data for children aged 8 to 12 years. Among the self-report measures for children aged 8 to 17 years, those assessing psychological well-being and stress and self-efficacy are different for children aged 8 to 12 years than for children aged 13 to 17 years. Self-report measures for negative affect and social relationships are standard for all children within the age range of 8 to 17 years. Among the proxy-report measures for children aged 3 to 12 years, those assessing negative affect and one measure of psychological well-being are different for children aged 3 to 7 years than for children aged 8 to 12 years. The remaining proxy-report measures are standard for all children within the age range of 3 to 12 years. Importantly, however, the NIHTB-EB for children aged 3 to 7 years do not include measures assessing the stress and self-efficacy subdomain. The limited cognitive and emotional capacity of children in this age range and difficulty for proxies to evaluate and rate these children’s stress and self-efficacy were considered likely to make any measurement of this construct inaccurate (Salsman et al., 2013).

Table 1.

Measures in Each Theoretical Subdomain of the NIH Toolbox Emotion Batteries for Children.

Emotion subdomain	Proxy-report for ages 3 to 7 years	Proxy-report for ages 8 to 12 years	Self-report for ages 8 to 12 years	Self-report for ages 12 to 17 years
Negative Affect	Anger	Anger	Angera	Angera
	Sadness	Sadness	Sadnessb	Sadnessb
	Over Anxious Separation Anxiety	Fear	Fearc	Fearc
Psychological Well-Being	Positive Affect General Life Satisfactiond	Positive Affect General Life Satisfactiond	Positive Affect General Life Satisfaction	Positive Affect General Life Satisfaction
Stress and Self-Efficacy	(not assessed)	Perceived Stress Self-Efficacy	Self-Efficacy	Perceived Stress Self-Efficacy
Social Relationships	Social Withdrawale	Social Withdrawale	Friendshipi	Friendshipi
	Positive Peer Interactionf	Positive Peer Interactionf	Lonelinessj	Lonelinessj
	Peer Rejectiong	Peer Rejectiong	Emotion Supportk	Emotion Supportk
	Empathic Behaviorsh	Empathic Behaviorsh	Perceived HostilityI Perceived Rejectionm	Perceived Hostilityl Perceived Rejectionm

Note. NIH = National Institutes of Health. Matched superscripts “a” through “m” indicate identical measures.

By separating children into the two age ranges that receive self-report measures and the two age ranges that have proxy-report measures, we can consider four distinct NIHTB-EBs for children: (a) proxy-report for ages 3 to 7 years, (b) proxy-report for ages 8 to 12 years, (c) self-report for ages 8 to 12 years, and (d) self-report for ages 13 to 17 years. Negative affect, psychological well-being, stress and self-efficacy, and social relationships are the four theoretically proposed emotion constructs that guided selection of the individual measures for each of these batteries; however, no studies to date have empirically determined the factor structure of these measures. Furthermore, while each of the 31 total measures (i.e., 16 proxy-report and 15 self-report) that comprise the NIHTB-EBs for children can be considered separately as unique entities, the creation of composite scores may be useful in examining broader constructs of emotional functioning. Composite scores have been created for other NIHTB assessments, including the NIHTB Cognition Battery for adults (Heaton et al., 2014), the NIHTB Cognition Battery for children (Akshoomoff et al., 2013), and the NIHTB-EB for adults (Babakhanyan, Casaletto, & Heaton, 2015; Babakhanyan, I., McKenna, B. S., Casaletto, K. B., & Heaton, R. K. 2018).

The primary goal of the current study was to empirically examine the major constructs assessed by each of the four NIHTB-EBs for children, using factor analyses of data from the U.S. national normative sample of 2,916 English-speaking children. Given the aforementioned age-related differences in individual scales across these batteries, and differences that may result from self-report versus parental proxy-reporting, we expected only general commonalities among positive and negative factors within the four batteries. However, more consistency was expected for the two self-report batteries for ages 8 to 12 years and 13 to 17 years, because 8 of the 15 component scales were overlapping (as opposed to only 5 of the 16 scales in the two proxy-report batteries). In addition, age- and gender-corrected norms are presented for all factor-based composites and individual NIHTB-EB scales. These demographic corrections were considered desirable for interpretation of results in relation to typical social and emotional development and expression of boys and girls within fairly broad age ranges. Similar normative adjustments have been provided for previously published assessments such as the Child Behavior Checklist (Achenbach & Rescorla, 2000, 2001) and the Behavior Assessment System for Children (Reynolds & Kamphaus, 2006).

Method

Participants

The NIHTB U.S. national normative sample of children consisted of 3,412 typically developing, community-dwelling children aged 3 to 17 years. Participants were recruited based on a stratified sampling strategy in which strata were defined by age, sex, and primary language (Beaumont et al., 2013). While English-speaking children of all ages (i.e., 3 to 17 years) were enrolled, only Spanish-speaking children aged 3 to 7 years were enrolled in the normative sample. The NIHTB team chose not to recruit Spanish-speaking children aged 8 to 17 years, because census data estimate that less than 1.5% of U.S. children in this age range use Spanish as their dominant language (Beaumont et al., 2013). As a result, only 496 Spanish-speaking children (all aged 3 to 7 years) were included in the normative sample. Because of the relatively small size and restricted age range of this Spanish-speaking subsample, they were excluded from the current factor analytic work. The final sample of 2,916 English-speaking children was used for creation of normed individual measure scores and composite scores reported here.

Personal and household demographic information was collected via proxy-report for child participants of all ages (see Table 2). This additionally includes information about child health with response options ranging from 5 (excellent health) to 1 (poor health), as well as school grades with response options ranging from 5 (mostly A’s) to 1 (mostly F’s). The child’s proxy was most often the child’s biological mother (76.0%), followed by the child’s biological father (19.1%), and various others including the child’s adoptive parent, stepparent, sibling, grandparent, or other relative (4.9%).

Table 2.

Sample Characteristics of Each Child Age Group.

	Ages 3-7 years (n = 668)	Ages 8-12 years (n = 1,120)	Ages 13-17 years (n = 1,128)
Age, years	5.14 (1.4)	10.01 (1.4)	14.98 (1.4)
Mother’s education, years	12.56 (2.4)	12.46 (2.5)	12.42 (2.5)
Gender, male	339 (50.7%)	559 (49.9%)	566 (50.2%)
Race/ethnicity
Wh ite	341 (51.0%)	657 (58.7%)	664 (58.9%)
Black	115 (17.2%)	187 (16.7%)	193 (17.1%)
Hispanic	124 (18.6%)	179 (16.0%)	175 (15.5%)
Other	88 (13.2%)	96 (8.6%)	96 (8.5%)
2009 Household income <$40,000, %	30.2	28.6	24.5
Household size	4.41 (1.3)	4.52 (1.4)	4.43 (1.6)
Child healtha	4.67 (0.5)	4.54 (0.7)	4.46 (0.7)
Proxy healtha	4.11 (0.8)	4.03 (0.9)	4.07 (0.8)
School gradesb	4.62 (0.7)	4.48 (0.7)	4.27 (0.8)

Note. Values are M (SD) or N (%). Proxy health refers to the proxy’s own self-rated health.

^aResponse options ranged from 5 (excellent health) to 1 (poor health).

^bResponse options ranged from 5 (mostly A’s) to 1 (mostly F’s).

NIHTB-EBs for Children

The NIHTB-EBs for children consist of computerized assessments for the evaluation of children’s emotions. They are available in the NIHTB iPad app, with paper-and-pencil versions of individual measures obtainable from the NIH Toolbox website (http://www.healthmeasures.net/explore-measurement-systems/nih-toolbox/obtain-and-administer-measures). All instruments included in these batteries were based on existing psychometrically valid measures or combinations of measures, as described previously. The NIHTB Scoring and Interpretation Guide provides detailed descriptions of each of these measures (Slotkin et al., 2012; “NIH Toolbox® Scoring and Interpretation Guide,” 2016). While each measure varies in length (i.e., number of items), all items use Likert-type response options on 4-point, 5-point, 6-point, or 7-point scales. Each battery was designed to require no more than 30 minutes to complete. Administration of available supplemental measures, however, can increase completion time. Supplemental measures were not included in the creation of composite scores because they were not administered to everyone in the normative sample.

Similar to the NIHTB-EB for adults, 27 of the measures used in the NIHTB-EBs for children are scored using item response theory methods, and are reported as scaled estimates of latent theta scores (M = 0, SD = 1). Four of the proxy-reported measures, however, are scored as a raw sum (i.e., Proxy-Reported Life Satisfaction, Positive Peer Interaction, Social Withdrawal, and Peer Rejection). Both item response theory estimates of theta scores and raw summed scores were converted to age- and gender-corrected T-scores for interpretation, as follows.

Age- and Gender-Corrected T-Score Derivation

Drawing from normative corrections used in widely accepted and psychometrically validated measures such as the Child Behavior Checklist (i.e., age and gender corrections; Achenbach & Rescorla, 2000, 2001) and the Behavior Assessment System for Children (Reynolds & Kamphaus, 2006), and further considering the developmental differences in emotional functioning across age and gender that may influence scores (Twenge & Nolen-Hoeksema, 2002), it was determined that correcting for age and gender would provide the best normative standard for interpreting children’s emotional health using the NIHTB-EBs for children. T-scores were derived using methods similar to those used for deriving the age-corrected and fully corrected T-scores for the NIHTB Cognition Battery (Casaletto et al., 2015); however, normed scores for the NIHTB-EBs for children were not calculated separately by racial/ethnic group, and were only regressed on age and gender, not education (in children, age and years of schooling are extremely highly correlated). Prior to demographic corrections, we first generated scaled scores (M = 10, SD = 3). This initial step put all scores within the same age category (i.e., 3-7 years, 8-12 years, and 13-19 years) on the same metric, and normalized them. Scaled scores are then used in formulas for computing age- and gender-adjusted T-scores (M = 50; SD = 10), which represent the child’s emotional functioning compared with children of the same age and gender.

Table S1 (All supplementary tables are available in online version of the article.) displays the individual measures’ associations to age and gender before normative corrections were applied. These raw scores represent the standardized theta scores (M = 0, SD = 1) that are automatically calculated and provided by the computerized program on which the NIHTB is administered. The tables for converting raw scores into scaled scores are provided in Tables S2 to S7. Formulas for computing age- and gender-corrected T-scores for all individual measures in the NIHTB-EBs for children are provided in Table S8. The method used to create these formulas (described in Casaletto et al., 2015) uses fractional polynomials to regress test scores on continuous predictors (i.e., age), allowing the fitting of nonlinear terms. Thus, for cases in which a nonlinear fit better explains variability in the outcome compared with a linear fit, the formula to apply the age correction takes the natural log of age. In addition, the next scheduled update of the NIHTB iPad app (expected in late 2017) automatically calculates the T-scores for the individual NIHTB-EB measures, and for all of the factor-based composites presented here.

Creation of Composite Scores

To create composite scores for emotional domains of each of the four pediatric versions of the NIHTB-EB, we evaluated the latent structure of the age- and gender-corrected measures for each group using factor analytic methods with single sample cross-validation methodologies. Specifically, each age group was randomly split into two samples stratified on gender and age. One subsample (3- to 7-year-old age group n = 468; 8- to 12-year-old age group n = 920; 13- to 17-year-old age group n = 928) was used for exploratory factor analyses (EFAs) and another subsample (n = 200 for each age group) was used for confirmatory factor analyses (CFAs). All analyses were conducted using the “lavaan” package for the R programming language (Rosseel, 2012). EFAs were first performed using maximum likelihood estimation and oblique rotation to estimate the number and structure of the underlying factors. We used parallel analysis (PA) and the minimum Bayesian information criterion (BIC) to determine of the number of factors to retain from EFAs. If multiple solutions were suggested by these methods, we aimed to select the most parsimonious model. Specifically, because PA has been known to retain too many factors in some contexts (e.g., due to item testlets or when scores are not entirely multivariate normal), we examined the “elbows” in scree plots and the simulated cutoff lines produced by PA, to determine whether the PA was suggesting inclusion of a factor with a small and indistinct eigenvalue. We also examined cross-loadings in the factor structures such that any scale loading at 0.32 or higher on two or more factors, or within a 0.13 margin difference, would be considered cross-loading (Costello & Osborne, 2005).

CFAs were then performed on the separate sample with the best solutions from the EFA step. Models allowed scales to load onto their primary (largest loading) factors determined from the EFAs with no cross-loadings or residual correlations. Means were fixed to 0, variances were fixed to 1, and all covariance parameters were freely estimated. Model parameters were estimated using maximum likelihood estimation with robust (Huber-White) standard errors. Multiple measures of model fit were assessed to determine how well the CFA model fit the data (Hoyle, 2000). These included (a) the comparative fit index, which compares the target model with a baseline null model that specifies no factors (values above 0.90 indicate adequate model fit and values above 0.95 indicate good model fit); (b) the root mean square error of approximation, which adjusts fit by weighting values by the number of parameters estimated (values less than .05 indicate good model fit, while values between 0.06 and 0.08 indicate adequate fit); and (c) standardized root mean square residual, which is an absolute measure of fit defined as the standardized difference between the observed correlation and the predicted correlation (values less than 0.08 indicate good model fit; Hu & Bentler, 1999; Kline, 2015; Steiger, 1990). Reliability was determined using both Cronbach’s alpha (α) as a lower bound estimate, and omega (ω), which is an estimate of reliability-based factor analysis (McDonald, 1999).

If a model was deemed to fit the data well, composite scores for the latent domains were created from the combined data (3- to 7-year-old age group n = 668; 8- to 12-year-old age group n = 1,120; 13- to 17-year-old age group n = 1,128). We combined the split samples at this step to have the most precise estimates for the purpose of creating summary scores that have research and clinical utility. Specifically, weighted averages were calculated for each domain/factor using the standardized, absolute CFA factor loadings as weights. T-scores for measures that had negative factor loadings were reverse scored (i.e., 100 – T-score) in composite score formulas to keep the direction consistent within each composite domain. Composite scores were then standardized to have a mean of 50 and standard deviation of 10.

Results

Exploratory Factor Analyses

For the proxy-report battery for children aged 3 to 7 years, the EFA best supported a three-factor structure with both the BIC statistic and the PA indicating three factors. Three measures (i.e., Positive Affect, General Life Satisfaction, and reverse-coded Anger) loaded most strongly onto Factor 1. Two measures (i.e., Over Anxious and Separation Anxiety) loaded most strongly onto Factor 2. Five measures (i.e., Social Withdrawal, Peer Rejection, Sadness and reverse-coded Positive Peer Interaction and Empathic Behavior) loaded most strongly onto Factor 3. See Table 3 for factor loadings, factor correlations, and the covariance matrix associated with this battery. Empathic Behavior cross-loaded onto Factor 1; however, it was kept in Factor 3 because most scales in Factor 3 had factor loadings <0.50 (Costello & Osborne, 2005). Omega for Factor 1 was 0.78, Factor 2 was 0.78, and Factor 3 was 0.85. Cronbach’s α for Factor 1 was 0.67, Factor 2 was 0.68, and Factor 3 was 0.76. The three factors were named based on the underlying construct captured by the combination of measures that comprise each. Factor 1 was named “Psychological Well-Being,” Factor 2 was named “Anxiety,” and Factor 3 was named “Negative Psychosocial Functioning.” Again, measures comprising certain factors were reverse scored to allow for better characterization of the theme of the underlying construct (e.g., Psychological Well-Being). This is consistent with certain factors in the other three emotion batteries for children as well.

Table 3.

Factor Loadings, Factor Correlations, and Covariance Matrix for Proxy-Report Battery for Children Aged 3 to 7 Years (n = 468).

	Factor loadings			Factor correlations		Covariance matrix
	Factor 1	Factor 2	Factor 3	Factors	Pearson r	Positive Affect	Anger	General Life Satisfaction	Over Anxious	Separation Anxiety	Sadness	Positive Peer Interaction	Social Withdrawal	Peer Rejection	Empathic Behavior
Positive Affect	0.55	−0.06	−0.15	Factors 1 and 2	−.45	99.45
Anger	−0.70	0.10	−0.15			−38.66	100.22
General Life Satisfaction	0.58	−0.06	0.05			40.93	−38.04	94.41
Over Anxious	−0.15	0.60	0.10	Factors 1 and 3	−.63	−33.35	35.28	−26.75	100.61
Separation Anxiety	−0.07	0.60	0.09			−28.35	27.15	−27.14	51.30	100.45
Sadness	−0.27	−0.10	0.46			−37.79	46.84	−40.18	43.28	36.93	99.19
Positive Peer Interaction	−0.02	−0.02	−0.86	Factors 2 and 3	.46	39.60	−26.74	28.85	−28.31	−22.47	−37.17	100.02
Social Withdrawal	−0.11	0.23	0.49			−35.47	30.08	−30.79	38.48	26.61	45.05	−49.54	102.13
Peer Rejection	−0.11	0.26	0.40			−36.14	31.72	−31.62	32.82	29.15	39.31	−46.28	41.67	98.17
Empathic Behavior	0.34	0.31	−0.48			35.64	−28.69	28.94	−9.88	−4.98	−28.34	43.54	−32.88	−22.78	102.28

Note. Bold values indicate factor loadings for measures that comprise each factor. Factor 1 = Psychological Well-Being; Factor 2 = Anxiety; Factor 3 = Negative Psychosocial Functioning.

For the proxy-reported battery for children aged 8 to 12 years, the EFA best supported a two-factor structure, with the BIC statistic indicating two factors and the PA indicating three factors. The most parsimonious two-factor model was supported by both scree and PA plots (i.e., the bend in the scree plot was at two factors and the PA cutoff line intersected the observed data line at the three-factor solution). Examination of cross-loadings also supported the two-factor model, as the three-factor model had meaningful crossloadings on at least one measure that would be problematic for factor interpretation and for the CFA models. Seven measures (i.e., Positive Affect, Self-Efficacy, General Life Satisfaction, and reverse-coded Sadness, Anger, Fear, and Perceived Stress) loaded most strongly onto Factor 1. Three measures (i.e., Social Withdrawal, Peer Rejection, and reverse-coded Positive Peer Interaction) loaded most strongly onto Factor 2. See Table 4 for factor loadings, factor correlations, and the covariance matrix associated with this battery. The Empathic Behavior measure did not load saliently onto any factors in any solutions and was therefore excluded from the factor solution. Omega reliability for Factor 1 was 0.91 and Factor 2 was 0.82. Cronbach’s α for Factor 1 was 0.86 and Factor 2 was 0.75. After considering the constructs captured by the measures that comprise each factor, Factor 1 was named “Psychological Well-Being” and Factor 2 was named “Negative Peer Relations.”

Table 4.

Factor Loadings, Factor Correlations, and Covariance Matrix for Proxy-Report Battery for Children Aged 8 to 12 Years (n = 920).

	Factor loadings		Factor correlations		Covariance matrix
	Factor 1	Factor 2	Factors	Pearson r	Positive Affect	Sadness	Anger	Fear	Self- Efficacy	Perceived Stress	General Life Satisfaction	Positive Peer Interaction	Social Withdrawal	Peer Rejection
Positive Affect	0.58	−0.08			99.04
Sadness	−0.61	0.13			−47.79	98.75
Anger	−0.65	0.05			−47.71	52.08	100.74
Fear	−0.57	0.09			−32.63	45.32	40.58	100.13
Self-Efficacy	0.73	0.08	Factors 1 and 2	−.59	53.24	−40.62	−44.44	−37.88	100.32
Perceived Stress	−0.91	−0.06			−55.52	55.93	60.62	50.47	−64.44	100.06
General Life Satisfaction	0.53	−0.10			52.52	−41.94	−39.84	−34.55	46.20	−52.81	103.71
Positive Peer Interaction	−0.08	−0.78			37.09	−32.18	−31.96	−28.00	34.69	−35.05	36.80	99.78
Social Withdrawal	−0.04	0.66			−36.56	37.68	34.72	28.28	−26.14	39.65	−31.28	−51.62	99.01
Peer Rejection	−0.20	0.56			−37.84	40.90	38.81	37.71	−37.74	46.95	−40.06	−50.28	49.19	100.15

Note. Bold values indicate factor loadings for measures that comprise each factor. Factor 1 = Psychological Well-Being; Factor 2 = Negative Peer Relations.

The results of the EFA on both of the self-report batteries (i.e., those for children aged 8 to 12 years and children aged 13 to 17 years) supported similar four-factor structures. For the self-report battery for children aged 8 to 12 years, both the BIC statistic and the PA indicated four factors. Three measures (i.e., Anger, Sadness, and Fear) loaded most strongly onto Factor 1, three measures (i.e., Positive Affect, General Life Satisfaction, and Self-Efficacy) loaded most strongly onto Factor 2, two measures (i.e., Perceived Hostility and Perceived Rejection) loaded most strongly onto Factor 3, and three measures (i.e., Friendship, Emotional Support, and reverse-coded Loneliness) loaded most strongly onto Factor 4. See Table 5 for factor loadings, factor correlations, and the covariance matrix associated with this battery. Although the standardized factor loading for Perceived Hostility on Factor 3 was greater than one, suggesting a high degree of multicollinearity (see Jöreskog, 1999), this factor was retained based on the BIC statistic, the PA, and the CFA results (see Table 5). Omega for Factor 1 was 0.91, Factor 2 was 0.81, Factor 3 was 0.80, and Factor 4 was 0.83. Cronbach’s α for Factor 1 was 0.85, Factor 2 was 0.73, Factor 3 was 0.73, and Factor 4 was 0.74.

Table 5.

Factor Loadings, Factor Correlations, and Covariance Matrix for Self-Report Battery for Children Aged 8 to 12 Years (n = 920).

	Factor loadings				Factor correlations		Covariance matrix
	Factor 1	Factor 2	Factor 3	Factor 4	Factors	Pearson r	Friendship	Loneliness	Emotional Support	Perceived Hostility	Perceived Rejection	Anger	Sadness	Fear	Positive Affect	General Life Satisfaction	Self- Efficacy
Friendship	0.13	0.22	−0.03	0.63	Factors 1 and 2	−.45	103.53
Loneliness	0.35	0.14	0.07	−0.69			−49.39	102.05
Emotional Support	−0.02	0.36	−0.06	0.43	Factors 1 and 3	.59	52.57	−46.81	97.76
Perceived Hostility	−0.04	0.01	1.03a	0.02			−19.52	42.42	−27.58	96.11
Perceived Rejection	0.28	−0.09	0.36	−0.21	Factors 1 and 4	−.51	−25.07	49.37	−35.58	56.20	100.37
Anger	0.65	−0.09	0.19	0.08			−20.21	46.08	−33.10	47.39	42.39	97.79
Sadness	0.74	−0.12	0.02	−0.09	Factors 2 and 3	−.39	−28.21	54.04	−40.12	44.65	52.19	63.40	98.66
Fear	0.80	−0.02	−0.06	−0.02			−22.46	47.81	−28.60	37.28	43.34	63.46	65.15	93.80
Positive Affect	−0.08	0.75	0.00	−0.04	Factors 2 and 4	.55	35.73	−30.20	44.72	−25.42	−33.05	−30.65	−38.39	−29.87	99.43
General Life Satisfaction	−0.10	0.61	−0.07	−0.02			33.12	−30.80	38.26	−29.92	−36.58	−30.46	−41.99	−27.46	52.88	99.95
Self-Efficacy	0.03	0.64	0.02	0.09	Factors 3 and 4	−.47	43.64	−25.34	44.98	−17.77	−21.74	−24.59	−28.36	−24.51	48.92	41.22	99.75

Note. Bold values indicate factor loadings for measures that comprise each factor. BIC = Bayesian information criterion; CFA = confirmatory factor analysis; PA = parallel analysis; Factor 1 = Negative Affect; Factor 2 = Psychological Well-Being; Factor 3 = Negative Social Perception; Factor 4 = Social Satisfaction.

^aSuggests high degree of multicollinearity (this factor was retained based on the BIC statistic, the PA, and CFA results).

For the self-report battery for children aged 13 to 17 years, the EFA best supported a four-factor structure, with the BIC statistic indicating four factors and the PA indicating five factors. The most parsimonious four-factor model was supported by both scree and PA plots (i.e., the bend in the scree plot was at four factors and the PA cutoff line intersected the observed data line at the five-factor solution). Examination of cross-loadings further supported the four-factor model, as the five-factor model had meaningful cross-loadings on at least one measure. Four measures (i.e., Positive Affect, General Life Satisfaction, Self-Efficacy, and reverse-coded Perceived Stress) loaded most strongly onto Factor 1. This is most similar to Factor 2 of the self-report battery for 8- to 12-year-olds with the addition of Perceived Stress, a self-report measure unique to the battery for 13- to 17-year-olds. Three measures (i.e., Anger, Sadness, and Fear) loaded most strongly onto Factor 2, identical to Factor 1 of the self-report battery for 8- to 12-year-olds. Two measures (i.e., Perceived Hostility and Perceived Rejection) loaded most strongly onto Factor 3, identical to Factor 3 of the self-report battery for 8- to 12-year-olds. Last, three measures (i.e., Friendship, Emotional Support, and reverse-coded Loneliness) loaded most strongly onto Factor 4, identical to Factor 4 of the self-report battery for 8- to 12-year-olds. See Table 6 for factor loadings, factor correlations, and the covariance matrix associated with the self-report battery for children aged 13 to 17 years. Omega for Factor 1 was 0.89, Factor 2 was 0.91, Factor 3 was 0.88, and Factor 4 was 0.88. Cronbach’s α for Factor 1 was 0.81, Factor 2 was 0.82, Factor 3 was 0.77, and Factor 4 was 0.77.

Table 6.

Factor Loadings, Factor Correlations, and Covariance Matrix for Self-Report Battery for Children Aged 13 to 17 Years (n = 928).

	Factor loadings				Factor correlations		Covariance matrix
	Factor 1	Factor 2	Factor 3	Factor 4	Factor	Pearson r	Friendship	Loneliness	Emotional Support	Perceived Hostility	Perceived Rejection	Anger	Sadness	Fear	Positive Affect	General Life Satisfaction	Self- Efficacy	Perceived Stress
Friendship	0.10	0.07	−0.01	0.70	Factors 1 and 2	−.65	97.09
Loneliness	0.09	0.38	0.11	−0.61			−52.22	98.26
Emotional Support	0.28	0.04	−0.11	0.49	Factors 1 and 3	−.55	49.26	−53.38	101.48
Perceived Hostility	0.02	−0.05	0.92	0.00			−25.57	38.18	−36.13	103.83
Perceived Rejection	−0.07	0.19	0.48	−0.20	Factors 1 and 4	.57	−35.06	56.17	−50.64	62.38	99.79
Anger	−0.19	0.51	0.27	0.11			−16.08	38.80	−31.74	57.30	50.29	102.20
Sadness	−0.24	0.59	0.09	−0.08	Factors 2 and 3	.64	−28.32	59.00	−42.11	47.68	58.68	63.07	101.53
Fear	−0.03	0.79	−0.02	−0.04			−18.32	46.61	−26.30	35.96	47.52	57.97	64.59	105.01
Positive Affect	0.70	−0.03	0.00	0.05	Factors 2 and 4	−.51	33.54	−39.39	43.11	−32.72	−41.40	−39.55	−52.12	−34.94	99.93
General Life Satisfaction	0.70	0.04	−0.11	0.00			28.88	−39.25	44.96	−37.13	−45.31	−40.82	−49.94	−31.81	55.28	99.90
Self-Efficacy	0.58	−0.03	0.09	0.14	Factors 3 and 4	−.56	36.23	−33.96	37.95	−23.38	−29.84	−31.25	−37.53	−32.22	46.81	43.40	100.29
Perceived Stress	−0.63	0.29	0.09	0.12			−25.24	42.65	−39.79	43.83	51.69	58.30	63.77	52.16	−55.37	−56.67	−48.11	100.01

Note. Bold values indicate factor loadings for measures that comprise each factor. Factor 1 = Psychological Well-Being; Factor 2 = Negative Affect; Factor 3 = Negative Social Perception; Factor 4 = Social Satisfaction.

Factors were given names based on the content and theme of the measures in each factor. Factor 1 of the self-report battery for children aged 8 to 12 years and Factor 2 of the self-report battery for children aged 13 to 17 years were both named “Negative Affect,” as they are both composed of measures that assess negative emotions. Factor 2 of the self-report battery for children aged 8 to 12 years and Factor 1 of the self-report battery for children aged 13 to 17 years were both named “Psychological Well-Being.” In both of the self-report batteries, Factor 3 was named “Negative Social Perception,” and Factor 4 was named “Social Satisfaction.”

Confirmatory Factor Analyses

CFA results from each of the four batteries using the results of the factor structure from the EFA step indicated adequate model fit overall. See Table 7 for all CFA fit indices for each battery of the NIHTB-EB for children. Because the models fit the data adequately, split samples were combined to create composite scores with the most precise estimates. After age- and gender-corrected T-scores for all measures included in a composite score were multiplied by their corresponding CFA weights (calculated from a CFA on combined samples), means and standard deviations of participants’ nonnormalized composite scores were calculated to normalize the composite score distribution to have a mean of 50 and a standard deviation of 10. Composite score formulas with CFA weights, means, and standard deviations can be found in Tables S9 to S12.

Table 7.

Confirmatory Factor Analyses Fit Indices.

NIHTB Emotion Battery for Children	CFI	RMSEA	SRMR
Proxy-report for children aged 3 to  7 years	.900	.082	.062
Proxy-report for children aged 8 to  12 years	.952	.070	.034
Self-report for children aged 8 to  12 years	.910	.089	.058
Self-report for children aged 13 to  17 years	.915	.098	.051

Note. n = 200 for each age group. NIHTB = National Institutes of Health Toolbox; CFI = comparative fit index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual.

Discussion

The NIHTB-EB, which is nonproprietary and utilizes easy, user-friendly iPad administration, has the potential to immediately impact and improve clinical measurement in the field of child mental health. The NIHTB-EB’s sophisticated scale development, nationally representative norms, and relatively comprehensive nature assessing many domains of emotional functioning, additionally highlight this assessment’s potential strength and importance for both research and clinical work. This study sought to advance the use of this battery by empirically examining factor structure, developing composites, and reporting norms. By computing and interpreting composite scores, researchers and clinicians who administer the NIHTB-EB for children will be able to examine more general aspects of emotional functioning in each age group.

In support of our hypothesis, results demonstrated that the factor structures vary across age groups and method (i.e., self- or proxy-report), with more similarities across the two self-report batteries for children aged 8 to 12 years and children aged 13 to 17 years. Several authors (e.g., Haynes, Smith, & Hunsley, 2011) have noted that conditional factor structure is not uncommon, and that it should be properly acknowledged when present. Moreover, given developmental changes in children, the many nonoverlapping scales across age-based batteries that were intended to reflect these differences, and the differences between self- and proxy-report, it is methodologically important to allow data collected for each battery to dictate the final measurement models. Results also showed that all factors comprising all four batteries had at least adequate reliability (Haynes et al., 2011). The factors comprising the three batteries for children aged 8 to 17 years had the largest reliability coefficients values (ranging from good to excellent reliability), indicating that social and emotional functioning in older children (i.e., at least 8 years old), whether proxy- or self-reported, is more reliably assessed compared with that of younger children (i.e., aged 3 to 7 years). This was expected, as social and emotional functioning in early childhood is difficult for proxies to observe and interpret accurately, especially among children who have yet to enter school (Denham, 2006). Notably, one measure in the proxy-report battery for children aged 3 to 7 years (i.e., Empathic Behavior) cross-loaded onto another factor, and one measure (i.e., Perceived Hostility) in the self-report battery for children aged 8 to 12 years displayed a high degree of mulitcollinearity. This could potentially make the EFA-derived factor structures for these batteries inconsistent in future replications. However, retaining these measures in the factor structures was supported by the subsequent CFA fit indices.

The psychological constructs captured by these composite scores map reasonably well onto three of the four subdomains originally chosen on theoretical grounds by the team of researchers working to create the NIHTB-EBs (i.e., Negative Affect, Psychological Well-Being, Stress and Self-Efficacy, and Social Relationships; Gershon et al., 2013; Nowinski et al., 2013; Salsman et al., 2013). The Stress and Self-Efficacy subdomain, however, is not represented by a composite score in any of the child age groups (or in the adult English-speaking or Spanish-speaking groups; Babakhanyan et al., 2015; (Babakhanyan, McKenna, Casaletto and Heaton, 2018). This was expected in the proxy-report battery for children aged 3 to 7 years, in which this subdomain was purposefully not assessed (Salsman et al., 2013). The individual measures relating to Stress and Self-Efficacy in the other three NIHTB-EBs for children are included with measures comprising the Psychological Well-Being composite scores. This is likely because there are not enough measures comprising this subdomain to emerge as a factor in any battery (Table 1).

The composite scores presented here will complement the composite scores that have also been created for the NIHTB-EB for adults (Babakhanyan, McKenna, Casaletto and Heaton, 2018). Two sets of composite scores were created for this battery in adults (i.e., for English-speaking adults and for Spanish-speaking adults). In both groups, factor analyses supported composite scores that very similarly characterized Negative Affect, Social Satisfaction, and Psychological Well-Being. Individual measures that comprise the adult battery, and the two child self-report batteries are displayed in Table 8. These similarities in composite scores for all self-report NIHTB-EBs may allow for tracking emotional development and functioning across the life span beginning at age 8.

Table 8.

Similarities Between Composite Scores for the NIHTB-EB for Adults and the Two Self-Report NIHTB-EB for Children.

	Individual measures
Composite scores	Self-report battery for adults aged 18+ years	Self-report battery for children aged 8 to 12 years	Self-report battery for children aged 13 to 17 years
Negative Affect	Anger-Affect	Anger	Anger
	Anger-Hostility	Sadness	Sadness
	Sadness	Fear	Fear
	Fear-Affect
	Perceived Stress
Social Satisfaction	Friendship	Friendship	Friendship
	Emotional Support	Emotional Support	Emotional Support
	Instrumental Support	Loneliness	Loneliness
	Loneliness
	Perceived Rejection
Psychological Well-Being	Life Satisfaction	Life Satisfaction	Life Satisfaction
	Meaning and Purpose	Self-Efficacy	Self-Efficacy
	Positive Affect	Positive Affect	Positive Affect Perceived Stress
Negative Social Perception	—	Perceived Hostility	Perceived Hostility
	—	Perceived Rejection	Perceived Rejection

Note. NIHTB-EB = National Institutes of Health Toolbox Emotion Battery.

It is also important to note the differences in factor structure across age in order to be aware of limitations when using the NIHTB-EBs to track emotional functioning as individuals transition to a different age-based version of the instrument. Importantly, users of the NIHTB-EBs for children must be aware that factors with the same name are not identically assessed or defined in the different age groups, and thus cannot be considered entirely equivalent. Some of the differences in factor structure across the NIHTB-EBs may relate to the differences in individual measures that were given for each battery (i.e., the Anxiety factor in the proxy-report battery for 3- to 7-year-olds is composed of measures administered only in this battery; the Psychological Well-Being factor in the self-report battery for 13- to 17-year-olds includes an additional measure only administered in this battery). Other factor differences, however (i.e., similar individual measures loaded onto different factors for different age groups), may reflect changes across emotional development as well as differences in parent-perception and self-perception of emotional functioning to which the NIHTB-EBs for children are sensitive.

Of note, we decided to correct the scores on the individual measures for only age and gender because of the importance of assessing a child’s emotional functioning in comparison with expected gender-specific developmental trajectories. Additionally, age and gender corrections are utilized in other widely used measures of child emotional functioning such as the Child Behavior Checklist (Achenbach & Rescorla, 2000, 2001) and the Behavior Assessment System for Children (Reynolds & Kamphaus, 2006). Correcting for any other variables (e.g., race/ethnicity, parent education/socioeconomic status) was considered to be either excessive or a hindrance to interpretability of scores. Last, years of education completed is too highly correlated with age in children for it to have warranted its own correction.

The current study provides users of the NIHTB-EBs for children with empirically derived factor-based composites that are likely to enhance use and interpretability of scores across age ranges; however, a key limitation is that our normative sample may not be demographically representative of some children to whom the battery is administered. For example, the majority of children in each age group self-identified as non-Hispanic White (51.0% in the 3- to 7-year-old age group; 58.7% in the 8- to 2-year-old age group; and 58.9% in the 13- to 17-year-old age group); however, this is very consistent with 2010 census figures for the U.S. population in general. Additionally, beyond examining factor structure, the current report does not attempt to validate the NIHTB-EBs for children. Some initial evidence of convergent validity has previously been published (Salsman et al., 2013) and this report is primarily intended to provide a foundation for future validation studies of the NIHTB-EBs for children, as these batteries have already been implemented in research and clinical settings nationwide. Studies are needed to examine validity of NIHTB-EBs for children in detecting problematic emotional and social functioning in community or clinical groups, or in relation to everyday functioning (e.g., school and social functioning). In clinical groups, relationships may also exist with NIHTB-assessed acquired impairments in other domains (i.e., cognition, motor, and sensory). It will be necessary for future studies of patient populations to also investigate patterns and probabilities of rates of emotional distress as measured by composite scores compared with those of the individual measures that comprise them. Last, next steps also include obtaining a large enough sample of Spanish-speaking children to repeat this factor analytic work with the aim of evaluating equivalence in factor structures across different cultural samples.

The normed scores for the individual and composite measures described in this report may enhance the use of the NIHTB-EBs for children by providing reliable and clinically interpretable scores for various domains of social and emotional functioning. Calculations for the normative scores and composite scores presented in this report are being programmed into the next update of the NIHTB iPad app (scheduled for late 2017). Growing use of the NIHTB-EBs will allow for better cross-study comparisons in the future, potentially transforming research on emotional functioning across the life span.