Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Aug 1.
Published in final edited form as: Depress Anxiety. 2017 May 22;34(8):742–751. doi: 10.1002/da.22656

Latent Structure of Negative Valence Measures in Childhood

Minyoung Lee 1, Steven H Aggen 1, Dever M Carney 1, Shannon Hahn 1, Elizabeth Moroney 2, Laura Machlin 2, Melissa A Brotman 2, Kenneth E Towbin 2, Ellen Leibenluft 2, Daniel S Pine 2, Roxann Roberson-Nay 1, John M Hettema 1
PMCID: PMC5542863  NIHMSID: NIHMS874523  PMID: 28543958

Abstract

Background

Internalizing disorders (IDs), consisting of the syndromes of anxiety and depression, are common, debilitating conditions often having onsets in adolescence. Scientists have developed dimensional self-report instruments that assess putative negative valence system (NVS) trait-like constructs as complimentary phenotypes to clinical symptoms. These include various measures that index temperamental predispositions to IDs and correlate with neural substrates of fear, anxiety, and affective regulation. This study sought to elucidate the overarching structure of putative NVS traits and their relationship to early manifestations of ID symptomatology.

Methods

The sample consisted of 768 juvenile twin subjects ages 9–13. Together with ID symptoms, extant validated instruments were chosen to assess a broad spectrum of NVS traits: anxiety sensitivity, irritability, fearfulness, behavioral activation and inhibition, and neuroticism and extraversion. Exploratory and confirmatory factor analyses (EFA/CFA) were used to investigate the latent structure of the associations among these different constructs and ID symptoms. Bifactor modeling in addition to standard correlated-factor analytic approaches were applied.

Results

Factor analyses produced a primary tripartite solution comprising anxiety/fear, dysphoria, and positive affect among all these measures. Competing DSM-like correlated factors and an RDoC-like NVS bifactor structure provided similar fit to these data.

Conclusions

Our findings support the conceptual organization of a tripartite latent internalizing domain in developing children. This structure includes both clinical symptoms and a variety of self-report dimensional traits currently in use by investigators. These various constructs are, therefore, most informatively investigated using an inclusive, integrated approach.

Keywords: fear, anxiety, depression, phenotype, child, twin study

Introduction

Anxiety and depressive (i.e., internalizing) disorders (IDs) typically start in late childhood or adolescence (Cohen et al., 1993; Pine, Cohen, Gurley, Brook, & Ma, 1998) and attain high lifetime prevalence and chronicity in adulthood (Kessler et al., 2005). Prior studies have applied factor analytic approaches to examine the relationships between internalizing symptoms or disorders in large, community samples of adults (Krueger, 1999; Vollebergh et al., 2001) and juveniles (Eley et al., 2003; Lahey et al., 2008), with similar findings: evidence for a single, higher-order “internalizing” factor that accounts for correlations amongst anxiety and depressive disorders and differentiates itself from an “externalizing” factor. The internalizing factor is typically made up of two lower, highly-correlated first order factors, the first roughly corresponding to an “anxious misery” dimension onto which major depressive disorder (MDD), dysthymia, and generalized anxiety disorder (GAD) load, and the second a “fear” dimension with indicators of panic and phobic disorders. Such findings inform the classification and epidemiology of diagnostic syndromes but, by including only clinical phenotypes, are limited for elucidating developmental risk mechanisms for IDs.

Scientists have been investigating a variety of trait-like emotional, temperamental, and behavioral constructs that can be assessed via self-report as complimentary to ID symptoms. Extant studies have reported replicable associations between certain constructs and specific IDs. Neuroticism/negative affect (NA) and extraversion/positive affect (PA) are two core dimensions of temperament believed to be instrumental to the risk for, and course of, IDs (Clark, Watson, & Mineka, 1994). Whereas neuroticism/NA is considered to be etiologically relevant to the broadest range of IDs and their comorbidity (Bienvenu et al., 2001), extraversion/PA has been linked more specifically to social anxiety and depression (Brown, Chorpita, & Barlow, 1998; Mineka, Watson, & Clark, 1998). They are partially related to Gray’s constructs of behavioral inhibition (BI) and behavioral activation (BA), respectively, which have been shown to correlate in animal emotional/motivational systems (Gray, 1982). Extant research supports BA/BI scales as indicators of neural systems underlying human approach–withdrawal tendencies (Sutton & Davidson, 2000). NA and PA, together with autonomic arousal, form the tripartite model of anxiety and depression (Clark & Watson, 1991; Mineka et al., 1998), a well-researched psychological model for IDs.

Besides these broad dimensions of temperament, several trait-like constructs relate more specifically to particular IDs. Anxiety sensitivity (AS) is a tendency to attribute harmful physical, psychological, or social consequences to the experience of anxious symptoms (Reiss, Peterson, Gursky, & McNally, 1986) with relatively specific association to panic-related pathology (Zvolensky & Schmidt, 2007). Childhood fearfulness is expressed in diverse forms and severity as a function of developmental timing and individual differences (Gullone, 2000). Its overall degree remains relatively stable, with more fearful children tending to meet criteria for phobias and other IDs in adolescence and adulthood. Chronic irritability in youth is relatively common, impairing, and predicts both depression and anxiety in adulthood (Leibenluft, 2011; Stringaris, Cohen, Pine, & Leibenluft, 2009).

In summary, extant studies suggest that various dimensional, trait-like measures might index shared or specific components of ID risk. However, most studies have only examined the relationship of such constructs to selected diagnostic outcomes via case-control approaches rather than using more broadly informative, unselected study designs. Studies ascertained for particular phenotypes can be biased and limited in their generalizability. In addition, it is not clear whether these constructs, hypothesized to be risk factors for anxiety and depression, map in an integrated manner onto the known dimensions of ID symptoms in developing children. Thus, further clarification of these relationships in childhood and adolescence are important for understanding the developmental unfolding of psychopathology, since premorbid temperamental substrates indexed by various dimensions of trait anxiety or dysphoria are hypothesized to drive the tendency for individuals to experience frequent symptomatic states, and increase vulnerability to the development of IDs and their comorbidity.

The Research Domain Criteria (RDoC) initiative of the National Institute of Mental Health proposes a framework for studying mental disorders using fundamental dimensions that cut across traditional disorder categories. (Insel et al., 2010). As articulated in the RDoC hypothetical framework, the negative valence system (NVS) domain encompasses biological and psychological systems involved in the response to aversive, threatening, or harmful stimuli, including systems that have a putative relationship to IDs. Hypothetically some of these measures should align with broader constructs proposed in the RDoC matrix like acute threat (“fear”) or potential threat (“anxiety”) and elements of negative affect (loss, anhedonia, frustrative non-reward) rather than clinical diagnoses (Cuthbert, 2015). Our recent review summarizes available data for putative NVS endophenotypes that share genetic etiology with anxiety disorders (Savage, Sawyers, Roberson-Nay, & Hettema, 2016).

A crucial step in evaluating the potential use of these measures as NVS endophenotypes is developing a detailed understanding of their relationship to each other and to the broad range of ID symptoms. Although the factorial structure of psychiatric symptoms and disorders has been thoroughly investigated, this is not the case for these psychologically-constructed dimensional traits. Analyzing state-like symptoms and related trait-like dimensions in the same model can provide deeper insights into underlying psychological substrates that they commonly index. Therefore, this study aims to address the following key questions:

  1. How is this diverse set of traits related to symptoms of IDs in youth? Simply, which traits relate to which symptoms and to what relative degree?

  2. If interpretable patterns of association are identified, what are the number and structure of putative underlying latent constructs represented among these measures? That is, does this broader array of internalizing phenotypes adhere to a similar tripartite structure as has been shown for clinical symptoms alone? Or will additional factors representing distinct underlying constructs emerge?

Materials and Methods

Participants and Measures

Caucasian twins aged 9–13 years (mean 11.2 years) were recruited for the Virginia Commonwealth University Juvenile Anxiety Study (VCU-JAS) from the Mid-Atlantic Twin Registry (MATR) (Lilley & Silberg, 2013). This study used an epidemiological sampling design unselected for any particular outcome phenotypes. Fifty-three percent were female, and one-third of the pairs were monozygotic. Caucasian twins were recruited to minimize phenotypic and genetic heterogeneity for the overall project. VCU-JAS assessed a broad array of putative psychological and biological indicators of NVS involved in the development of IDs (Carney et al., 2016). These included survey instruments assessing dimensional aspects of children’s symptomatology or temperament and psychological and physiological measures collected during laboratory paradigms. Only the child self-report survey measures assessing the previously described constructs are included in the current analyses. Of the 398 complete twin pairs who participated in VCU-JAS, data from these measures were available from 768 children. This study was conducted in compliance with the Declaration of Helsinki and approved by the VCU Institutional Review Board. Written informed consent from parents and assent from minor children were obtained after the nature of the procedures was explained.

A battery of psychometrically sound self-report instruments was chosen to assess the broad-spectrum of relevant psychological traits as well as ID symptomatology. The Short Mood and Feelings Questionnaire (SMFQ) is a 13-item scale derived from a larger pool of depression-related questions from the full MFQ (Angold et al., 1995). The Screen for Childhood Anxiety Related Disorders (SCARED) - Child Version (Birmaher et al., 1997) assesses five anxiety symptom domains: Panic/Somatic Anxiety (PD), General Anxiety (GAD), Separation Anxiety (SAD), Social Phobia (SOC), and School Avoidance (SchA). Because of the poorer clinical specificity of the four SchA items, we excluded those from our analyses. The Childhood Anxiety Sensitivity Index (CASI) assesses AS (Silverman, Fleisig, Rabian, & Peterson, 1991) with good internal consistency and test-retest reliability in non-referred and clinical samples. The Affective Reactivity Index (ARI) is a 6-item inventory of irritability with elevated scores in children with mood and anxiety disorders (Stringaris et al., 2012; Stoddard et al., 2014). The 80-item Fear Survey Schedule for Children-Revised (FSSC-R) assesses general levels of fearfulness across common fear domains (Ollendick, 1983). The Behavioral Inhibition System/Behavioral Activation System (BIS/BAS) Scales (Carver & White, 1994) assess facets of BI and BA, respectively. The Eysenck Personality Questionnaire, Junior version (JEPQ) (Eysenck, 1965) measures the two broad personality traits included herein: neuroticism (JEPQ-N) (20 items), extraversion (JEPQ-E) (25 items).

Statistical Analyses

Data analyses were carried out using data from SCARED, SMFQ, CASI, FSSC-R, ARI, BIS, BAS, JEPQ-N, and JEPQ-E scales. Mean scores for each were created to account for possible item non-responses for each subject. The four SCARED subscales were also included in additional models to examine the effects of specific anxiety disorder symptoms in relation to the other scales. Pearson correlations were calculated between all the trait and symptom measures in order to estimate their relative patterns of association.

The approach employed to address the second study aim is exploratory factor analysis (EFA) followed by confirmatory factor analysis (CFA). These were conducted in Mplus 7.1 (Muthen & Muthen, 2012) using the maximum likelihood estimator with robust standard errors (MLR) in conjunction with the COMPLEX option to adjust standard errors and fit indexes for the dependency of co-twins nested within pairs. Since no study has ever examined the relationships between these diverse phenotypes which have demonstrated varied relationships with different clinical measures, no clear hypotheses can be confidently proposed for their structure. Rather, EFA was conducted in a random half of the sample to generate a set of preliminary hypotheses that were then confirmed via CFA in the remaining half. Twins were randomized individually, not pair-wise. For the EFA phase, an initial parallel analysis of the eigenvalues was performed to determine the number of factors to retain (HORN, 1965). Model specification for CFA was guided by the EFA parallel analysis, goodness of fit, and the patterning and interpretability of loadings. The root mean square error of approximation (RMSEA) and comparative fit index (CFI) were used to evaluate overall model fit. The Akaike information criterion (AIC) and Bayesian information criterion (BIC) were used to evaluate the relative fit of non-nested models; lower values indicate amore parsimonious fit to the data. Final parameters for the best-fitting CFA models were estimated using the full sample data.

To investigate alternative hypotheses regarding the underlying latent structure among the measures, both correlated factors and bifactor models were specified. The correlated factors model posits multiple oblique first-order latent dimensions to account for the associations among the observed scales. The bifactor model represents a different conceptual structure to account for the patterning of their associations (Reise, 2012). It posits a single general common factor with loadings on all the measures plus residual group factors with loadings for subsets that are orthogonal to the general factor. Each of the differently specified models were evaluated using CFA n the second half-sample to validate the factor structure suggested by the EFA results derived using the first half-sample.

Results

Means, standard deviations, and internal consistency estimates for each scale are shown in Table 1. There were no significant differences for the means and standard deviations of the scale scores between the split samples. Internal consistency estimates demonstrated acceptable reliability (α >.70).

Table 1.

Means, standard deviations of mean scale scores, and internal consistency of scales

Mean (SD)
Internal consistency†
α
Scale EFA sample
N=384		 CFA sample
N=384		 All sample
N=768
SCARED 0.60 (0.28) 0.61 (0.31) 0.61 (0.30) 0.899
SCARED-PD 0.39 (0.28) 0.41 (0.32) 0.40 (0.30) 0.789
SCARED-GAD 0.67 (0.40) 0.66 (0.42) 0.66 (0.41) 0.788
SCARED-SAD 0.63 (0.41) 0.68 (0.45) 0.66 (0.43) 0.782
SCARED-SOC 0.87 (0.47) 0.86 (0.48) 0.86 (0.47) 0.795
SMFQ 0.40 (0.32) 0.41 (0.32) 0.41 (0.32) 0.812
JEPQ-E 0.73 (0.18) 0.72 (0.17) 0.73 (0.17) 0.783
JEPQ-N 0.42 (0.23) 0.43 (0.25) 0.42 (0.24) 0.852
CASI 0.56 (0.33) 0.58 (0.34) 0.57 (0.33) 0.845
ARI 0.50 (0.42) 0.50 (0.44) 0.50 (0.43) 0.820
BAS 1.52 (0.47) 1.50 (0.46) 1.51 (0.47) 0.803
BIS 1.29 (0.58) 1.30 (0.60) 1.30 (0.59) 0.774
FSSC-R 0.67 (0.32) 0.69 (0.34) 0.68 (0.33) 0.959
Open in a new tab

Abbreviations: EFA=exploratory factor analysis; CFA=confirmatory factor analysis; SD=standard deviation; SCARED=screen for childhood anxiety related disorders; PD=panic disorder; GAD=general anxiety; SAD=separation anxiety; SOC=social phobia; SMFQ=short mood and feelings questionnaire; JEPQ-E=junior version of Eysenck personality questionnaire – extraversion; JEPQ-N=junior version of Eysenck personality questionnaire – neuroticism; CASI=childhood anxiety sensitivity index; ARI=affective reactivity index; BAS=behavioral activation system; BIS=behavioral inhibition system; FSSC-R=shortened fear survey schedule for children-revised.

†

Internal consistency α > .9 (Excellent), > .8 (Good), > .7 (Acceptable), > .6 (Questionable), > .5 (Poor), and < .5 (Unacceptable).

Table 2 presents the correlation patterning within blocks of related measures roughly indicating (1) anxiety and fear (symptoms or temperament), (2) negative affect and neuroticism, and (3) behavioral activation and extraversion. Besides higher correlations within each block, moderate correlations between NA/neuroticism and anxiety/fear measures were also found. Extraversion was positively correlated with BA and negatively correlated with social phobia.

Table 2.

Bivariate correlations for all self-report measures blocked by degree of relationship (N=768)

Self-report measures
1 2 3 4 5 6 7 8 9 10 11 12 13
  1. SCARED   1.00
  2. SCARED-PD   0.80*   1.00
  3. SCARED-GAD   0.77*   0.51*   1.00
  4. SCARED-SAD   0.77*   0.51*   0.43*   1.00
  5. SCARED-SOC   0.72*   0.40*   0.41*   0.44*   1.00
  6. CASI   0.71*   0.65*   0.56*   0.53*   0.42*   1.00
  7. BIS   0.65*   0.50*   0.66*   0.42*   0.41*   0.65*   1.00
  8. FSSC-R   0.62*   0.47*   0.46*   0.54*   0.43*   0.66*   0.58*   1.00
  9. JEPQ-N   0.64*   0.52*   0.58*   0.46*   0.39*   0.58*   0.64*   0.49*   1.00
10. SMFQ   0.54*   0.48*   0.45*   0.42*   0.29*   0.49*   0.41*   0.37*   0.67*   1.00
11. ARI   0.41*   0.36*   0.33*   0.31*   0.25*   0.40*   0.35*   0.28*   0.61*   0.50*   1.00
12. BAS   0.19*   0.23*   0.19*   0.11*   0.03   0.24*   0.23*   0.12*   0.18*   0.11*   0.17*   1.00
13. JEPQ-E −0.26* −0.14* −0.13* −0.18* −0.38* −0.17* −0.20* −0.29* −0.27* −0.26* −0.21*   0.33*   1.00
Open in a new tab
*

p<0.01

Abbreviations: SCARED=screening for childhood anxiety related disorders; PD=panic disorder; GAD=general anxiety; SAD=separation anxiety; SOC=social phobia; SMFQ=short mood and feelings questionnaire; JEPQ-E=junior version of Eysenck personality questionnaire – extraversion; JEPQ-N=junior version of Eysenck personality questionnaire – neuroticism; CASI=childhood anxiety sensitivity index; ARI=affective reactivity index; BAS=behavioral activation system; BIS=behavioral inhibition system; FSSC-R=shortened fear survey schedule for children-revised.

For the EFA, the scree plot of the eigenvalues of the observed correlation matrix are contrasted against parallel analysis eigenvalues derived from randomly generated data, indicating three factors without or with the SCARED subscales (Figure 1). Notably, the first eigenvalue was 3–4 times larger than the second and third, a pattern suggestive of a bifactor structure.

Figure 1. Scree plot from the results of EFA parallel analysis (EFA sample, N=384) using SCARED total score (A) and SCARED subscale scores (B).

Figure 1

Open in a new tab

Abbreviations: EFA=exploratory factor analysis; SCARED=screen for childhood anxiety related disorders.

For both the correlated factor and bifactor models, we selected three-factor solutions due to their acceptable fit to the data, consistency with the parallel analysis, and interpretability. Table 3 lists the factor loadings and inter-factor correlations for the three-factor models. Because the overall solutions for total SCARED and its sub-scales are similar (Tables 3A and 3B, respectively), we present them together here.

Table 3.

Factor loadings of standard and bifactor EFA in EFA sample (N=384, 3-factor solution)

A. Using SCARED total score
Scale Standard EFA Bifactor EFA
Factor 1 Factor 2 Factor 3 Factor 1 Factor 2 Factor 3
SCARED   0.669*   0.156   0.002   0.785* −0.053   0.024
CASI   0.841* −0.001   0.148   0.797* −0.154   0.174*
FSSC-R   0.909* −0.205 −0.023   0.722*   0.012 −0.302*
BIS   0.630*   0.207*   0.102   0.771* −0.003   0.120*
ARI −0.087   0.670* −0.009   0.478*   0.383* −0.019
JEPQ-N   0.012   0.926*   0.009   0.785*   0.505*   0.001
SMFQ   0.122   0.595* −0.047   0.625*   0.304* −0.048
BAS   0.295*   0.008   0.488*   0.212*   0.009   0.487*
JEPQ-E −0.002 −0.247*   0.938* −0.368* −0.009   0.919*
Inter-factor correlation
Factor 1   1.000   1.000
Factor 2   0.706*   1.000   0.000   1.000
Factor 3 −0.159 −0.079   1.000   0.000   0.000   1.000
B. Using SCARED subscale scores
Scale Standard EFA Bifactor EFA
Factor 1 Factor 2 Factor 3 Factor 1 Factor 2 Factor 3
SCARED-PD   0.687*   0.023   0.017   0.685* −0.065   0.141
SCARED-GAD   0.595*   0.203*   0.038   0.677*   0.144*   0.016
SCARED-SAD   0.625*   0.025 −0.033   0.636* −0.063   0.081
SCARED-SOC   0.452*   0.002 −0.320*   0.502* −0.098 −0.229*
CASI   0.810*   0.007   0.025   0.791* −0.089   0.171
FSSC-R   0.832* −0.157 −0.145*   0.716* −0.217   0.012
BIS   0.625*   0.131   0.011   0.768*   0.053   0.121
ARI −0.048   0.633* −0.006   0.450*   0.397* −0.024
JEPQ-N   0.020   0.938*   0.012   0.753*   0.581*   0.001
SMFQ   0.186   0.539* −0.065   0.616*   0.303* −0.038
BAS   0.280*   0.009   0.425*   0.202*   0.033   0.464*
JEPQ-E   0.001 −0.248*   0.985* −0.378* −0.013   0.961*
Inter-factor correlation
Factor 1   1.000   1.000
Factor 2   0.689*   1.000   0.000   1.000
Factor 3 −0.022 −0.072   1.000   0.000   0.000   1.000
Open in a new tab

Abbreviations: EFA=exploratory factor analysis; CFA=confirmatory factor analysis; SCARED=screening for childhood anxiety related disorders; PD=panic disorder; GAD=general anxiety; SAD=separation anxiety; SOC=social phobia; SMFQ=short mood and feelings questionnaire; JEPQ-E=junior version of Eysenck personality questionnaire – extraversion; JEPQ-N=junior version of Eysenck personality questionnaire – neuroticism; CASI=childhood anxiety sensitivity index; ARI=affective reactivity index; BAS=behavioral activation system; BIS=behavioral inhibition system; FSSC-R=shortened fear survey schedule for children-revised.

*

Indicates significant factor loadings

For the correlated factor model (left side of tables), the first factor displayed the highest loadings on the scales assessing anxiety or fear (ANX/FEAR): SCARED, CASI, FSSC-R, and BIS. The second factor had primary loadings for scales representing dysphoric mood (DYSPH): ARI, SMFQ, and JEPQ-N. This second factor was correlated with the first factor (r=.78). The third factor, uncorrelated with the first two, primarily reflected positive affect (PA) as assessed by items from the JEPQ-E and BAS.

Bifactor EFA modeling (right side of tables) revealed a general factor that underlies all the scales with significant loadings on each (Jennrich & Bentler, 2011). This general factor was assigned the suggestive label “NVS”. Anxiety/fear and dysphoria scales loaded > .60 (range .62 to .80), and positive affect scales had smaller loadings (range from .20 to .38). Two orthogonal group factors were identified that accounted for residual co-variance among dysphoria and positive affect scales, respectively. Smaller but significant loadings were also estimated for SCARED subscales GAD on factor 2 (DYSPH) and SOC on factor 3 (PA, negatively correlated).

To further validate the EFA results, three different CFA specifications of these models, with or without SCARED subscale scores, were fit to the second half-sample and compared. Table 4 reports the fit statistics for all competing CFA models. These were specified as simple structure versions by fixing low indicator cross-loadings to zero based on the results of EFA and interpretability of loadings. First, without the SCARED subscales, the correlated three-factor model (A2) was compared with the two-factor model (A1) to verify our findings from the EFA. As expected, the three-factor model fit better (CFI=.973, RMSEA=.070, AIC=705.2) than a two-factor model (CFI=.884, RMSEA=.139, AIC=852.5). In addition, the bifactor model (A3) with one general and two orthogonal group factors provided a slightly improved fit over the correlated three-factor model (CFI=.978, RMSEA=.065, AIC=700.6). The two residual group factors had loadings on the scales representing DYSPH and PA in addition to the common general factor NVS having loadings on all the scales.

Table 4.

Fit statistics for CFA models in CFA sample (N=384)

Model χ2 df CFI TLI RMSEA AIC BIC
A. Using SCARED total score
 A1. 2-factor 199.1 24 0.884 0.827 0.139 734.3 852.5
 A2. 3-factor 63.1 22 0.973 0.956 0.070 579.1 705.2
 A3. bifactor 54.5 21 0.978 0.962 0.065 570.5 700.6
B. Using SCARED subscale scores
 B1. 3-factor† 202.6 49 0.923 0.896 0.091 1618.6 1780.1
 B2. 3-factor‡ 162.6 46 0.942 0.916 0.082 1582.8 1756.1
 B3. bifactor 155.9 45 0.944 0.919 0.081 1573.4 1750.7
Open in a new tab

Abbreviations: CFI=comparative fit statistic; TLI=Tucker Lewis index; RMSEA=root mean square error of approximation; df = degrees of freedom; AIC=Akaike information criterion; BIC=Bayesian information criterion; SCARED=screening for childhood anxiety related disorders.

†

Model B1 retains the same structure as the 3-factor model using SCARED total score (Model A2) with all loadings of SCARED subscales to the first factor for anxiety/fear.

‡

Model B2 is based on the 3-factor EFA solution using SCARED subscale scores.

Given these findings with the SCARED total score, we separately fit a similar sequence of models including the four SCARED subscales to better understand the detailed latent structure in relation to different clinical domains. First, the correlated three-factor model (B1) was fit with the SCARED subscales loading only on the ANX/FEAR factor. It had a marginally adequate fit to the data (CFI=.923, RMSEA=.091); however, a correlated three-factor model (B2) provided an improved fit (CFI=.942, RMSEA=.082) when allowing the GAD and SOC subscales to cross-load on DYSP and PA, respectively. Again, the bifactor model specification (B3) including the separate SCARED subscales provided the best-fitting structure to these data. Finally, the full dataset was used to obtain parameter estimates for the best-fitting models. These competing structural models are illustrated in Figures 2 (correlated factors) and 3 (bifactor) separately for SCARED total score (part A) and subscales (part B).

Figure 2. Three-factor correlated models for factor structure of NVS (full sample, N=768) using SCARED total score (A) and SCARED subscale scores (B).

Figure 2

Open in a new tab

Abbreviation: NVS=negative valence system; ANX/FEAR=anxiety disorder and fear; DYSPH=dysphoria; PA=positive affect; SCARED=screen for childhood anxiety related disorders; PD=panic disorder; GAD=general anxiety; SAD=separation anxiety; SOC=social phobia; SMFQ=short mood and feelings questionnaire; JEPQ-E=junior version of Eysenck personality questionnaire - extraversion; JEPQ-N=junior version of Eysenck personality questionnaire - neuroticism; CASI=childhood anxiety sensitivity index; ARI=affective reactivity index; BAS=behavioral activation system; BIS=behavioral inhibition system; FSSC-R=fear survey schedule for children-revised.

Discussion

This study, conducted in an epidemiological sample of 768 twins aged 9–13, used exploratory and confirmatory factor analyses (EFA/CFA) to investigate possible latent structures of extant dimensional traits to serve as psychological probes of internalizing emotional dysregulation in children. The included constructs of neuroticism, extraversion, behavioral inhibition/activation (BI/BA), anxiety sensitivity (AS), fearfulness, and irritability have each been reported in prior studies to show differences among cases of one or more IDs and unaffected controls. As indicated in Table 2, we found rather complex relational patterns among these measures and between them and ID symptoms in children. Neuroticism exhibited moderate associations with all anxiety and depressive symptoms as well as the negative dimensional traits (r=0.4 to 0.6), while extraversion had fairly modest, negative associations with them (r=−0.1 to −0.3). Irritability had similarly widespread albeit smaller correlations with the other measures. BA had very small associations with the others, suggesting it is globally orthogonal to NVS traits. That is consistent with its conceived role in positive valence (approach) systems. Among anxious traits, AS exhibited a strong relationship to overall childhood anxiety symptoms (SCARED r=0.71) and a diminishing but significant association with each individual anxiety subscale (PD r=0.65, GAD r=0.56, SAD r=0.53, SOC r=0.42). It also had a moderate correlation with depression (SMFQ r=0.49). BI had a similar overall pattern as AS, although some details differ. Fearfulness had higher estimated correlations with AS (r=0.66) and BI (r=0.58) than with the anxiety symptom scales (0.43–0.54). Thus, many of these dimensional traits have rather widespread (“cross-cutting”) associations across the ID symptom domains in this age group. This is consistent with the often nebulous and changing patterns of ID symptoms seen in developing children and the fact that most of these children do not meet criteria for full diagnostic syndromes. It might also be partially explained by lack of stable personality at this age, making the traits less likely to achieve some of the more specific associations seen in adults.

The structural relationships among the measures were formally examined using EFA in a random half-sample of the children and cross-validated using CFA in the other half-sample. A three-factor model provided a parsimonious and good fit to these data. We then tested alternative models for the structure under which these three factors are organized. The correlated factor model that posits three underlying latent constructs at the same level of organization adequately fit the data. This model consisted of two highly correlated latent dimensions corresponding to anxiety and fear (ANX/FEAR) and dysphoria (DYSPH), respectively, plus a third that loaded on measures of positive affect (PA) (Figure 2). It does not appear that fear and anxiety represent distinct latent dimensions at this broad structural level even though they can be dissected at finer levels of resolution. Such a tripartite structure is reminiscent of models previously reported for internalizing clinical symptoms and disorders. These findings extend that structure to a broader array of constructs beyond symptoms alone.

Overall, an alternative bifactor model (Figure 3) provided a slightly better fit to the data than the correlated factors model. This specifies a conceptually different organization of the underlying structure of these phenotypes, with a single common factor accounting for covariation between all the measures plus two independent residual factors. A bifactor structure for Negative Affect had been previously reported in a large study of children ascertained from both community and clinical settings (Ebesutani et al., 2011). For the current data, the common factor might be interpreted as a latent RDoC-like NVS dimension with highest positive loadings on the ANX/FEAR measures, intermediate loadings on measures of dysphoric mood, and a modest negative loading on extraversion. That is, this broader NVS factor reflects elements of acute (fear) and potential (anxiety) threat response, anxious temperament (inhibition, neuroticism), and negative affect (depression, irritability). Indicators of the two residual factors are similar to those from DYSPH and PA dimensions seen in the correlated factor model but with the elements common to the NVS factor removed. Interestingly, besides positive loadings on extraversion and BA, this PA factor also includes modest but significant negative loadings on the SCARED social phobia subscale and FSSC-R fears.

Figure 3. Three-factor bifactor models for factor structure of NVS (full sample, N=768) using SCARED total score (A) and SCARED subscale scores (B).

Figure 3

Open in a new tab

Abbreviation: NVS=negative valence system; DYSPH=dysphoria; PA=positive affect; SCARED=screen for childhood anxiety related disorders; PD=panic disorder; GAD=general anxiety; SAD=separation anxiety; SOC=social phobia; SMFQ=short mood and feelings questionnaire; JEPQ-E=junior version of Eysenck personality questionnaire - extraversion; JEPQ-N=junior version of Eysenck personality questionnaire - neuroticism; CASI=childhood anxiety sensitivity index; ARI=affective reactivity index; BAS=behavioral activation system; BIS=behavioral inhibition system; FSSC-R=fear survey schedule for children-revised.

These factor-analytic findings substantially inform our understanding of the patterns of affective responding in children in ways that the individual correlations cannot. The global patterns of clinical symptoms, and even their associations with dimensional traits, is best explained by a familiar tripartite structure. Thus, while individual symptoms and trait constructs appear to be somewhat nebulous and non-specific in their inter-associations, the latent underlying structure is consistent with that seen in adult IDs. This suggests that neural systems in this age group are already providing a patterned substrate of affective responding at both the long-term trait and short-term state levels. Remarkably, this applies to children who have not yet passed through the adolescent period of increasing risk in which IDs more clearly manifest.

To our knowledge, this is the first study to jointly examine the relationships among such a broad set of putative NVS phenotypes and ID symptoms in children. Prior studies partially explored such relationships. Eley and colleagues reported associations between AS, panic/somatic symptoms, and heart-beat perception in children 8–11 years old (Eley, Stirling, Ehlers, Gregory, & Clark, 2004). Longitudinal twin studies from that group reported influences of AS at age 8 on various anxiety subscales at age 10 (Waszczuk, Zavos, & Eley, 2013) and cross-lagged effects between AS and symptoms of anxiety and depression between ages 15 and 17 (Zavos, Rijsdijk, & Eley, 2012).

Note that, like other studies that have examined the relationships between temperament and psychopathology, we started by estimating statistical correlations to examine the differential associations between constructs. Associations between outcome phenotypes arise either because one directly causes another, or because each are etiologically related to some common, possibly unmeasured, construct. Prior studies have suggested that common etiological (i.e., genetic and environmental) pathways likely account for the majority of shared variance between personality and IDs (Bienvenu, Hettema, Neale, Prescott, & Kendler, 2007; Hettema, Neale, Myers, Prescott, & Kendler, 2006). Herein we have taken the position that, while conceptualized and assessed differently, trait-like dimensions such as AS or BI on the one hand, and state-like symptoms like panic and depression on the other, are manifest indices of (unmeasured) neural systems that broadly underlie the regulation of fear, anxiety, and mood. In order to test the alternative hypothesis of a temporally causal pathway between traits and disorders, a longitudinal study design would be required.

When conceptualizing this study, we made the a priori decision to include in our factor analyses a broad array of potential internalizing constructs and corresponding measures to thoroughly investigate their structural relationships: trait-like measures (e.g., AS, extraversion), clinical symptoms (SCARED, SMFQ), and others that might fall somewhere in between (fear and irritability scales). Prior studies examining some of these measures in more restricted case-control designs, as well as their extensive covariance observed in aim 1, supports such an analytic strategy. An alternative approach might first conduct factor analyses without the clinical measures and then examine the latter as external validators for the solutions. When we tested this post-hoc, the resulting models for the traits reflected a similar overall tripartite structure. However, there were insufficient numbers of outcomes to adequately define some of the factors, making the models fit sub-optimally. Also, while we attempted to select a broad and representative group of surveys to assess constructs related to IDs, we did not include some others proposed under RDoC NVS (e.g., loss) or other nosological systems (e.g., harm avoidance; (Cloninger, Svrakic, & Przybeck, 1993)). Thus, we cannot determine how those omitted constructs might fit into the overall structure we found. Because of these considerations, our results support an overall latent tripartite or, possibly, higher-order NVS, structure; the detailed mix of included constructs and their respective loadings can vary based upon the measures studied. We note that the current RDoC matrix does not include dimensions of normal personality like neuroticism, but it does include “anxious traits” like AS and BI. Studies like this one provide opportunities to explore the boundaries of NVS and other domains to inform future RDoC formulations.

These findings should be considered in the context of other potential limitations. First, these measures were all assessed at the same time point, allowing only cross sectional analyses. As mentioned above, we could test only associational, not causal, effects, providing limited insights into mechanistic pathways of risk. However, the discovery and replication approach using the split-half sample tentatively supports the reliability of the latent structures we found. Second, we evaluated the relationship between the putative NVS endophenotypic measures and internalizing symptoms, not disorders. Although young children have limited ability to report on their emotional states, both SCARED and SMFQ have previously shown adequate validity with respect to interview-assessed IDs. Compared to a clinically ascertained sample, a community sample such as VCU-JAS typically has lower severity of internalizing symptoms, making generalization to patient samples uncertain. Third, the results apply to juveniles aged 9–13. This age group was recruited specifically because it represents a critical juncture between childhood and adolescence at which the risk for IDs sharply rises. It is interesting that the overall phenotypic structure is largely tripartite in nature similar to that seen previously for IDs in adults. This could reflect an essential, age-invariant organization of negative emotional systems at both clinical state and enduring trait levels. However, this would need to be confirmed in studies in both younger children and older adolescents. Fourth, given the numbers of individual items in each survey measure, we could not conduct item-level factor analyses among the constructs. Fifth, correlation between constructs may be inflated by overlapping item content that invoke similar responses in the participants. This, together with limited method variance (i.e., self-report in children), could create spurious factors or enhance the fit of one model over another. The former is less likely, given the interpretable tripartite structure we obtained. To address the latter, we present both a DSM-like correlated factors structure and RDoC-like NVS bifactor structure as reasonable competing hypotheses with comparable fits. The most optimal set of analyses would involve eliminating potential item contamination in the construction of a new assessment instrument, rewording retained items using similar time frames and response formats, and then examining the resulting factor structure. Construction of a such new instrument is outside of the intent and scope of this study. Sixth, only Caucasian families were included to minimize heterogeneity, precluding the ability to generalize our findings to children of other racial and ethnic groups. Finally, from an RDoC “units of analysis” perspective, we only included self-report measures. Future analyses in this sample will potentially include data from laboratory paradigms and genetic information that could help validate or further extend the structural models we identified.

The findings of this study have broad implications for future research into developmental risk factors for anxiety and depression. First, it supports the conceptual organization of an overall tripartite structured, latent NVS domain in developing children. Second, this structure applies not only to clinical symptoms but also a wide variety of dimensional trait measures currently in use by investigators. Third, many of these phenotypes have reported associations with neural circuitry, which implies an underlying set of biological mechanisms with elements that are construct-specific and others that are “cross-cutting” in nature. The overarching conclusion is that these various constructs would be most informatively investigated in future studies using an inclusive, integrated approach.

Acknowledgments

VCU-JAS was sponsored through the first RDoC funding request (RFA-MH-12-100) and funded by the National Institutes of Health (R01MH098055 to JMH and NIMH-IRP-ziamh002781 to DSP). The Mid-Atlantic Twin Registry was supported through the NIH Center for Advancing Translational Research Grant Number UL1TR000058.

Footnotes

This study was performed at the two institutions listed above.

There are no financial disclosures or conflicts of interest.

Preliminary results from this study were presented at the Anxiety and Depression Association of America (ADAA) Conference, April 2016, Philadelphia, PA.

Reference List

  1. Angold A, Costello EJ, Messer SC, Pickles A, Winder F, Silver D. The development of a short questionnaire for use in epidemiological studies of depression in children and adolescents. InternationalJournal of Methods in Psychiatric Research. 1995;5:1–12. [Google Scholar]
  2. Bienvenu OJ, Brown C, Samuels JF, Liang KY, Costa PT, Eaton WW, et al. Normal personality traits and comorbidity among phobic, panic and major depressive disorders. Psychiatry Research. 2001;102:73–85. doi: 10.1016/s0165-1781(01)00228-1. [DOI] [PubMed] [Google Scholar]
  3. Bienvenu OJ, Hettema JM, Neale MC, Prescott CA, Kendler KS. Low extraversion and high neuroticism as indices of genetic and environmental risk for social phobia, agoraphobia, and animal phobia. Am J Psychiatry. 2007;164:1714–1721. doi: 10.1176/appi.ajp.2007.06101667. [DOI] [PubMed] [Google Scholar]
  4. Birmaher B, Khetarpal S, Brent D, Cully M, Balach L, Kaufman J, et al. The Screen for Child Anxiety Related Emotional Disorders (SCARED): scale construction and psychometric characteristics. J Am Acad Child Adolesc Psychiatry. 1997;36:545–553. doi: 10.1097/00004583-199704000-00018. [DOI] [PubMed] [Google Scholar]
  5. Brown TA, Chorpita BF, Barlow DH. Structural relationships among dimensions of the DSM-IV anxiety and mood disorders and dimensions of negative affect, positive affect, and autonomic arousal. J Abnorm Psychol. 1998;107:179–192. doi: 10.1037//0021-843x.107.2.179. [DOI] [PubMed] [Google Scholar]
  6. Carney DM, Moroney E, Machlin L, Hahn S, Savage JE, Lee M, et al. The Twin Study of Negative Valence Emotional Constructs. Twin Res Hum Genet. 2016;19:456–464. doi: 10.1017/thg.2016.59. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Carver CS, White TL. Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS scales. Journal of Personality and Social Psychology. 1994;67:319–333. [Google Scholar]
  8. Clark LA, Watson D. Tripartite model of anxiety and depression: psychometric evidence and taxonomic implications. J Abnorm Psychol. 1991;100:316–336. doi: 10.1037//0021-843x.100.3.316. [DOI] [PubMed] [Google Scholar]
  9. Clark LA, Watson D, Mineka S. Temperament, personality, and the mood and anxiety disorders. J Abnorm Psychol. 1994;103:103–116. [PubMed] [Google Scholar]
  10. Cloninger CR, Svrakic DM, Przybeck TR. A psychobiological model of temperament and character. Archives of General Psychiatry. 1993;50:975–990. doi: 10.1001/archpsyc.1993.01820240059008. [DOI] [PubMed] [Google Scholar]
  11. Cohen P, Cohen J, Kasen S, Velez CN, Hartmark C, Johnson J, et al. An epidemiological study of disorders in late childhood and adolescence–I. Age- and gender-specific prevalence. J Child Psychol Psychiatry. 1993;34:851–867. doi: 10.1111/j.1469-7610.1993.tb01094.x. [DOI] [PubMed] [Google Scholar]
  12. Cuthbert BN. Research Domain Criteria: toward future psychiatric nosologies. Dialogues Clin Neurosci. 2015;17:89–97. doi: 10.31887/DCNS.2015.17.1/bcuthbert. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Ebesutani C, Smith A, Bernstein A, Chorpita BF, Higa-McMillan C, Nakamura B. A bifactor model of negative affectivity: fear and distress components among younger and older youth. Psychol Assess. 2011;23:679–691. doi: 10.1037/a0023234. [DOI] [PubMed] [Google Scholar]
  14. Eley TC, Bolton D, O’connor TG, Perrin S, Smith P, Plomin R. A twin study of anxiety-related behaviours in pre-school children. J Child Psychol Psychiatry. 2003;44:945–960. doi: 10.1111/1469-7610.00179. [DOI] [PubMed] [Google Scholar]
  15. Eley TC, Stirling L, Ehlers A, Gregory AM, Clark DM. Heart-beat perception, panic/somatic symptoms and anxiety sensitivity in children. Behav Res Ther. 2004;42:439–448. doi: 10.1016/S0005-7967(03)00152-9. [DOI] [PubMed] [Google Scholar]
  16. Eysenck SB. A new scale for personality measurements in children. Br J Educ Psychol. 1965;35:362–367. doi: 10.1111/j.2044-8279.1965.tb01824.x. [DOI] [PubMed] [Google Scholar]
  17. Gray JA. The Neuropsychology of Anxiety. New York: Oxford University Press; 1982. [Google Scholar]
  18. Gullone E. The development of normal fear: a century of research. Clin Psychol Rev. 2000;20:429–451. doi: 10.1016/s0272-7358(99)00034-3. [DOI] [PubMed] [Google Scholar]
  19. Hettema JM, Neale MC, Myers JM, Prescott CA, Kendler KS. A population-based twin study of the relationship between neuroticism and internalizing disorders. Am J Psychiatry. 2006;163:857–864. doi: 10.1176/ajp.2006.163.5.857. [DOI] [PubMed] [Google Scholar]
  20. HORN JL. A RATIONALE AND TEST FOR THE NUMBER OF FACTORS IN FACTOR ANALYSIS. Psychometrika. 1965;30:179–185. doi: 10.1007/BF02289447. [DOI] [PubMed] [Google Scholar]
  21. Insel T, Cuthbert B, Garvey M, Heinssen R, Pine DS, Quinn K, et al. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry. 2010;167:748–751. doi: 10.1176/appi.ajp.2010.09091379. [DOI] [PubMed] [Google Scholar]
  22. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62:593–602. doi: 10.1001/archpsyc.62.6.593. [DOI] [PubMed] [Google Scholar]
  23. Krueger RF. The structure of common mental disorders. Arch Gen Psychiatry. 1999;56:921–926. doi: 10.1001/archpsyc.56.10.921. [DOI] [PubMed] [Google Scholar]
  24. Lahey BB, Rathouz PJ, Van HC, Urbano RC, Krueger RF, Applegate B, et al. Testing structural models of DSM-IV symptoms of common forms of child and adolescent psychopathology. J Abnorm Child Psychol. 2008;36:187–206. doi: 10.1007/s10802-007-9169-5. [DOI] [PubMed] [Google Scholar]
  25. Leibenluft E. Severe mood dysregulation, irritability, and the diagnostic boundaries of bipolar disorder in youths. Am J Psychiatry. 2011;168:129–142. doi: 10.1176/appi.ajp.2010.10050766. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lilley EC, Silberg JL. The Mid-Atlantic Twin Registry, revisited. Twin Res Hum Genet. 2013;16:424–428. doi: 10.1017/thg.2012.125. [DOI] [PubMed] [Google Scholar]
  27. Mineka S, Watson D, Clark LA. Comorbidity of anxiety and unipolar mood disorders. Annu Rev Psychol. 1998;49:377–412. doi: 10.1146/annurev.psych.49.1.377. [DOI] [PubMed] [Google Scholar]
  28. Muthen LK, Muthen BO. Mplus User’s Guide. Seventh. Los Angeles, CA: Muthen & Muthen; 2012. Los Angeles, CA ed. [Google Scholar]
  29. Ollendick TH. Reliability and validity of the Revised Fear Surgery Schedule for Children (FSSC-R) Behav Res Ther. 1983;21:685–692. doi: 10.1016/0005-7967(83)90087-6. [DOI] [PubMed] [Google Scholar]
  30. Pine DS, Cohen P, Gurley D, Brook J, Ma Y. The risk for early-adulthood anxiety and depressive disorders in adolescents with anxiety and depressive disorders. Arch Gen Psychiatry. 1998;55:56–64. doi: 10.1001/archpsyc.55.1.56. [DOI] [PubMed] [Google Scholar]
  31. Reise SP. Invited Paper: The Rediscovery of Bifactor Measurement Models. Multivariate Behav Res. 2012;47:667–696. doi: 10.1080/00273171.2012.715555. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Reiss S, Peterson RA, Gursky DM, McNally RJ. Anxiety sensitivity, anxiety frequency and the prediction of fearfulness. Behav Res Ther. 1986;24:1–8. doi: 10.1016/0005-7967(86)90143-9. [DOI] [PubMed] [Google Scholar]
  33. Savage JE, Sawyers C, Roberson-Nay R, Hettema JM. The genetics of anxiety-related negative valence system traits. Am J Med Genet B Neuropsychiatr Genet. 2016 doi: 10.1002/ajmg.b.32459. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Silverman WK, Fleisig W, Rabian B, Peterson RA. Childhood anxiety sensitivity index. Journal of Clinical Child Psychology. 1991;20:162–168. doi: 10.1207/s15374424jccp2801_9. [DOI] [PubMed] [Google Scholar]
  35. Stoddard J, Stringaris A, Brotman MA, Montville D, Pine DS, Leibenluft E. Irritability in child and adolescent anxiety disorders. Depress Anxiety. 2014;31:566–573. doi: 10.1002/da.22151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Stringaris A, Cohen P, Pine DS, Leibenluft E. Adult outcomes of youth irritability: a 20-year prospective community-based study. Am J Psychiatry. 2009;166:1048–1054. doi: 10.1176/appi.ajp.2009.08121849. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Stringaris A, Goodman R, Ferdinando S, Razdan V, Muhrer E, Leibenluft E, et al. The Affective Reactivity Index: a concise irritability scale for clinical and research settings. J Child Psychol Psychiatry. 2012;53:1109–1117. doi: 10.1111/j.1469-7610.2012.02561.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Sutton SK, Davidson RJ. Prefrontal brain electrical asymmetry predicts the evaluation of affective stimuli. Neuropsychologia. 2000;38:1723–1733. doi: 10.1016/s0028-3932(00)00076-2. [DOI] [PubMed] [Google Scholar]
  39. Vollebergh WA, Iedema J, Bijl RV, de Graaf R, Smit F, Ormel J. The structure and stability of common mental disorders: the NEMESIS study. Arch Gen Psychiatry. 2001;58:597–603. doi: 10.1001/archpsyc.58.6.597. [DOI] [PubMed] [Google Scholar]
  40. Waszczuk MA, Zavos HM, Eley TC. Genetic and environmental influences on relationship between anxiety sensitivity and anxiety subscales in children. J Anxiety Disord. 2013;27:475–484. doi: 10.1016/j.janxdis.2013.05.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Zavos HM, Rijsdijk FV, Eley TC. A longitudinal, genetically informative, study of associations between anxiety sensitivity, anxiety and depression. Behav Genet. 2012;42:592–602. doi: 10.1007/s10519-012-9535-0. [DOI] [PubMed] [Google Scholar]
  42. Zvolensky MJ, Schmidt NB. Introduction to anxiety sensitivity: recent findings and new directions. Behav Modif. 2007;31:139–144. doi: 10.1177/0145445506296968. [DOI] [PubMed] [Google Scholar]

RESOURCES