Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Apr 1.
Published in final edited form as: J Child Psychol Psychiatry. 2014 Jul 23;56(4):415–422. doi: 10.1111/jcpp.12300

Criterion Validity of the General Factor of Psychopathology in a Prospective Study of Girls

Benjamin B Lahey 1, Paul J Rathouz 2, Kate Keenan 1, Stephanie D Stepp 3, Rolf Loeber 3, Alison E Hipwell 3
PMCID: PMC4435546  NIHMSID: NIHMS688361  PMID: 25052460

Abstract

Background

The best-fitting model of the structure of common psychopathology often includes a general factor on which all dimensions of psychopathology load. Such a general factor would be important if it reflects etiologies and mechanisms shared by all dimensions of psychopathology. Nonetheless, a viable alternative explanation is that the general factor is partly or wholly a result of common method variance or other systematic measurement biases.

Methods

To test this alternative explanation, we extracted general, externalizing, and internalizing factor scores using mother-reported symptoms across 5–11 years of age in confirmatory factor analyses of data from a representative longitudinal study of 2,450 girls. Independent associations between the three psychopathology factor scores and teacher-reported criterion variables were estimated in multiple regression, controlling intelligence and demographic covariates.

Results

The model including the general factor fit significantly better than a correlated two-factor (internalizing/externalizing) model. The general factor was robustly and independently associated with all measures of teacher reported school functioning concurrently during childhood and prospectively during adolescence.

Conclusions

These findings weaken the hypothesis that the general factor of psychopathology in childhood is solely a measurement artifact and support further research on the substantive meaning of the general factor.

Keywords: Validity, psychopathology, factor analysis

Prevalent dimensions of psychopathology are substantially correlated with one another from childhood through adulthood (Angold & Costello, 2009; Angold, Costello, & Erkanli, 1999; Krueger & Markon, 2006a). Furthermore, the magnitudes of those correlations vary in patterned ways that reliably give rise to second-order ‘externalizing’ and ‘internalizing’ factors (Achenbach, Conners, Quay, Verhulst, & Howell, 1989; Krueger & Markon, 2006a). It is less often noted, however, that the internalizing and externalizing factors are themselves substantially correlated with one another (Angold & Costello, 2009; Angold, et al., 1999; Krueger & Markon, 2006b). This is important because it suggests the testable hypothesis that internalizing and externalizing factors are correlated because they share etiologies and psychobiological mechanisms, which are distinguishable from the shared etiologic factors and mechanisms that cause the first-order dimensions of psychopathology to correlate together on the internalizing and externalizing factors. That is, there may be a general factor of psychopathology that reflects etiologies and mechanisms shared to varying degrees by all prevalent dimensions of common forms of psychopathology (Lahey et al., 2012). Related arguments have been made regarding a general factor of personality (Block, 1965; Musek, 2007; Pettersson, Turkheimer, Horn, & Menatti, 2012)

We initially tested the general factor of psychopathology hypothesis using phenotypic data from structured diagnostic interviews on psychopathology of 43,093 18–64 year old adults from the representative National Epidemiologic Study of Alcohol and Related Conditions (NESARC) (Grant et al., 2004). A hierarchical confirmatory factor analysis (CFA) model of 11 prevalent mental disorders that included two second-order internalizing factors (fears and distress) and an externalizing factor fit the data well. Like previous studies (Krueger & Markon, 2006b), however, the correlations among these second-order factors were substantial. Consistent with our hypothesis, an alternative model that also included a general factor fit the data significantly better (Lahey, et al., 2012). In addition, data from a study of 9–17 year old twins provided support for the general factor of psychopathology at the level of shared etiologic influences (Lahey, Van Hulle, Singh, Waldman, & Rathouz, 2011). A model of genetic covariances among 11 dimensions of psychopathology specifying orthogonal internalizing, externalizing, and general factors fit significantly better than a correlated two-factor (internalizing/externalizing) model, suggesting that the general factor of psychopathology partly reflects highly pleiotropic genetic influences that are shared to varying degrees by all of the first-order dimensions of psychopathology. In the same sample, associations of factors from a general factor model based on phenotypic covariances with dispositional constructs were described, but a formal test of the comparative fit of the general factor and correlated two-factor models was not conducted (Tackett et al., 2013). In addition, the general factor hypothesis was examined recently in young adults (Caspi et al., 2014). A formal test of improvement in fit was not conducted, however, and fit statistics suggested that a model specifying three correlated dimensions of psychopathology may fit as well as the general factor model. Thus, there is need for additional formal tests of the comparative fit of the general factor model across the lifespan.

If replicated, the hypothesized general factor of psychopathology could be important to our understanding of psychopathology. Indeed, it may force a hierarchical conceptualization of the nature of psychopathology in which the etiology and psychobiological mechanisms of varying dimensions are seen as both widely shared and dimension-specific. Before the field can seriously consider such an interpretation of the general factor of psychopathology, however, alternative explanations must be evaluated empirically. One viable alternative is the possibility that the general factor partly or wholly reflects systematic measurement biases rather than correlations among veridically measured dimensions of psychopathology. For example, the general factor could reflect specious correlations arising from common method variance—because the same informants and methods are used to assess each dimension of psychopathology (Campbell & Fiske, 1959; Podsakoff, MacKenzie, Lee, & Podsakoff, 2003). Similarly, the general factor could arise from respondents’ general tendency to rate all negatively worded items similarly (Pettersson et al., 2014) or from implicit theories that certain emotions and behaviors tend to occur together (Schneider, 1973).

Systematic measurement biases such as these could well result in reporting symptoms in correlated ways even when the symptoms are not actually experienced or observed. Indeed, it is difficult to imagine that the observed correlations among symptoms and dimensions of psychopathology do not reflect such forms of biased measurement to some degree. The essential question addressed in this paper is whether the general psychopathology factor represents only correlations among dimensions of psychopathology arising from biased measurement.

We approach this issue from the perspective of measurement theory, in which the rating of each symptomatic behavior is assumed to be composed of a ‘true’ score plus measurement error (Nunnally, 1978; Spearman, 1904). Random measurement error is only a problem in studies of the correlational structure of psychopathology to the extent that it attenuates correlations among true scores on psychopathology dimensions (Spearman, 1904; Thorndike, 1920). In contrast, systematic sources of measurement error could give rise to a specious general factor that reflects only systematic error. In measurement theory, this is a question of criterion validity (Cronbach & Meehl, 1955). If the general psychopathology factor reflects substantive correlations among dimensions of psychopathology, it will be correlated with important variables that are independent to the measurement of psychopathology, such as measures of daily functioning obtained from different informants. If the general factor reflects only systematically biased measurement, it will not be correlated with such external criteria.

In our earlier paper on the general factor of psychopathology in adults (Lahey, et al., 2012), we conducted limited tests of the criterion validity of the general factor using data from NESARC. Controlling for the internalizing and externalizing factors, the general factor of psychopathology was found to account for unique variance in associations with putative risk factors, measures of adaptive functioning, and future psychopathology three years later. Unfortunately, because each of these variables was reported by the same informant, these correlations could simply be due shared measurement error. In the current paper, we report the results of much stronger tests of the criterion validity of the general factor of psychopathology by using independent measures of adaptive functioning obtained from a different informant.

In the present analyses, we use CFA to compare the fit of alternative models of the phenotypic structure of psychopathology when the general factor is, and is not, included in a representative sample of girls whose psychopathology has been assessed repeatedly at approximately one-year intervals (Keenan et al., 2010). We model the structure of psychopathology using parent reports of symptoms across 5–11 years of age. Because the general factor model has not been tested in this young age range, these analyses will test the generality of this model. We then test the criterion validity of the general factor of psychopathology by evaluating its unique correlations with independent measures of functioning obtained both concurrently (at 5–11 years) and prospectively (at 12–16 years).

METHOD

Participants

The sample for the Pittsburgh Girls Study (PGS) was identified via an enumeration of 103,238 city households, targeting 100% of households in 23 disadvantaged neighborhoods and 50% of households in the remaining 66 neighborhoods (Hipwell et al., 2008; Hipwell et al., 2011; Keenan, et al., 2010). Sampling weights were used to adjust for this over-sampling. Of 2,875 families with a 5–8 year old girl who were contacted, 2,450 (85.3%) agreed to participate under informed consent. Compared with data from the 2000 Census, the proportion of girls successfully recruited did not differ across the 50% and 100% sampled neighborhoods. Participation rates in annual assessments were very high, ranging from 97.2% in wave 2 (girls aged 6–9 years) to 85.3% in wave 12 (youngest girls aged 16 years). Approximately half (52%) of the 2,450 girls were African American, 42% European American, and 6% of mixed or another ethnicity.

Measures

Demographic characteristics

Demographic data were collected via parent report which included questions about household structure (single or dual parent), race, parental educational level (< 12 years of education of higher) and whether the household received public assistance (e.g. WIC, food stamps, welfare).

Intelligence

Verbal and performance intelligence were assessed at 10 years of age using two verbal (Vocabulary and Similarities) and two performance (Block Design and Picture Completion) subtests of the Wechsler Intelligence Scale for Children-III-R (Wechsler, 1991). This short-form has demonstrated excellent psychometric properties relative to the full-length Verbal Scale (Donders, 1997; Kaufman, Kaufman, Balgopal, & McLean, 1996). The mean score was 97.0 (range 51–149; SD = 17.6) for the verbal scale and 95.1 (range 49–145; SD = 17.9) for the performance scale.

Measures of Psychopathology

In each annual assessment conducted when the girls were 5–11 years of age, their parents completed reliable and valid measures of the girls’ psychopathology. The Child Symptom Inventory (CSI-4) (Gadow & Sprafkin, 1994) was administered to assess inattention, hyperactivity-impulsivity, oppositional defiance, conduct disorder, and depression. Parents also completed the Screen for Child Anxiety Related Emotional Disorders (Birmaher et al., 1999; Birmaher et al., 1997; Monga et al., 2000; Wren et al., 2007) annually to assess the girls’ generalized anxiety disorder, social phobia, separation anxiety disorder, school phobia, and panic/somatic symptoms.

Independent Measures of Functioning

The criterion validity of the general psychopathology factor based on parent reports of symptoms was evaluated using independent measures of functioning relevant to psychopathology obtained from teachers over the course of the study. These reflect key adaptational demands for children.

School functioning

Each year, a teacher of an academic subject or subjects rated performance in reading, spelling, and mathematics relative to the child’s classmates on a 5-point scale ranging from 1 (far below grade level) to 5 (far above grade level). Teachers also reported on the girls’ behavior in the classroom relative to other students. Three items assessed how hard she was working, how appropriately she was behaving, and how happy she appeared to be. Each item was rated on a 7-point scale ranging from 1 (much less) to 7 (much more). Separate mean scores were calculated for each of these measures of school functioning across years 5–11 and 12–16 years.

The reliability of these items was conservatively estimated based on all correlations among ratings of each item by each child’s teacher at different ages, indexing the correlations with Cronbach’s alpha. For example, the 21 correlations among ratings of mathematics performance by different teachers during the seven assessments when the girls were 5–11 years ranged from r = .45 to .64 (median = .59). Based on such correlations, alphas for childhood reading, spelling, and mathematics were .91, .90, and .91, respectively, and .90, .90, and .89 during adolescence. Alphas for ratings of working hard, behavior, and mood in childhood were .85, .84, and .76, respectively, and .70, .80, and .68 during adolescence.

Grade retention and special education services in school

Each year the teacher reported whether the girl was currently repeating last year’s grade in school between ages 5–11 years and between 12–16 years. Teachers also reported on whether the girl had been evaluated for special school services that year between ages 5–11 and if she had received special education services for behavior or emotional problems between 12–16 years. These variables were included as ecologically valid indicators of school functioning related to psychopathology that are based on the judgments of many individuals regarding the girls’ difficulties in meeting the demands of the classroom without assistance.

Global impairment

The overall functioning of the child was assessed annually using the rating on the Children’s Global Assessment Scale (Setterberg, Bird, Gould, Shaffer, & Fisher, 1992) by the child’s teacher in each year. This scale assesses the child’s highest level of functioning during the two months prior to each assessment. Lower scores indicate poorer functioning. Mean CGAS scores were computed across the age ranges of 8–11 years and for 12–16 years.

Statistical Analyses

Correcting Biases due to Missing Data

Of the 2,450 girls with symptom data during 5–11 years, 2,419 (98.7%) had nonmissing data on the criterion measures at 5–11 years, providing little opportunity for bias. Among these 2,450 girls, 2,238 (91.3%) had nonmissing criterion scores during adolescence, with some individual criterion measures having slightly lower Ns. Comparing the 2,230 girls with data on criterion measures in both childhood and adolescence to the 185 girls with missing criterion data in adolescence, the girls without adolescent data were less likely to be non-Hispanic white (40.4% to 51.4%, χ2 (1, N = 2415) = 8.52, p < .005) and their families were more likely to have been on public assistance 57.7% to 41.4%, χ2, (1, N = 2419) = 18.62, p < .0001), but did not differ on intelligence scores or teacher CGAS scores during childhood. Therefore, the analyses included the demographic variables as covariates to estimate population parameters. In addition, all analyses were weighted to account for the varying probabilities of selection into the sample.

Confirmatory Factor Analyses

Based on the analytic methods used in previous studies of the hierarchical structure of psychopathology (Caspi, et al., 2014; Lahey, et al., 2012; Tackett, et al., 2013), we compared the fit of two alternative structural models of psychopathology scores across 5–11 years. CFA models were estimated using maximum likelihood with a robust variance estimator (MLR) to account for departures from normality and to allow for sample weights in Mplus 7.1 (Muthén & Muthén, 2013). Comparative tests of nested alternative models were based on the scaled Satorra–Bentler chi-square statistic appropriate for MLR (Satorra & Bentler, 2001).

Model 1

A correlated (oblique) 2-factor model was estimated in which parent-reported depression, generalized anxiety disorder, social phobia, school phobia, and panic disorder scores loaded on an internalizing factor, and parent-reported inattention, hyperactivity-impulsivity, oppositional defiant, and conduct disorder scores loaded on an externalizing factor.

Model 1A

In order to identify the strongest 2-factor against which to test the general factor model, we tested the comparative fit of an alternative correlated 2-factor model in which depression symptoms scores were allowed to load on both the internalizing and externalizing factors specified in Model 1. Model 1A was based on previous evidence from a separate study of child and adolescent twins that depression is atypical in being correlated as strongly with externalizing as with internalizing dimensions of psychopathology at both phenotypic (Lahey et al., 2008) and genetic levels (Lahey, et al., 2011).

Model 2

This model was identical to Model 1A except that a general factor of psychopathology was specified as a bifactor (Gibbons & Hedeker, 1992) on which every dimension of psychopathology loaded. Consistent with the assumptions underlying bifactor models (Brown, 2006), covariances between the internalizing and externalizing factors and between both of those factors and the general factor were fixed to zero.

Criterion Validity Analyses

Tests of the criterion validity were conducted of internalizing, externalizing, and general psychopathology factor scores estimated in the best-fitting CFA model in Mplus and standardized (mean = 0; SD = 1). The independent association of each such factor score with each external criterion variable was estimated in multiple linear (for continuous criterion measures) and logistic (for dichotomous criterion measures) regressions in PROC SURVEYREG and PROC SURVEYLOGISTIC in SAS 9.3 to account for the oversampling of lower income neighborhoods.

RESULTS

Confirmatory Factor Analyses

Initially, the fit of the correlated 2-factor model was poor: Sattora-Bentler χ2 = 881.26, df = 34; RMSEA = 0.101 (90% CI: 0.095 – 0.107), CFI = 0.881; BIC = 99298.98, but modification indices indicated that ODD and CD and inattention and hyperactivity-impulsivity were each substantially correlated beyond the correlations accounted for by the externalizing factor. Because these modification indices were substantially greater than any others, and the correlations were theoretically sensible based on current knowledge of the correlational structure of psychopathology in children (Achenbach, et al., 1989; Lahey, et al., 2008), the residual deviations for ODD and CD and for inattention and hyperactivity-impulsivity dimensions were allowed to correlate. Nevertheless, the fit of Model 1 was marginal even with correlated residuals, Sattora-Bentler χ2 = 701.38, df = 32; RMSEA = 0.092 (90% CI: 0.087 – 0.098); CFI = 0.906; BIC = 99084. Model 1A, in which depression loaded on both the internalizing and externalizing factors, RMSEA = 0.073 (90% CI: 0.067 – 0.080); CFI = 0.942; BIC = 98747, fit better than Model 1 (Δ χ2 = 327.65, df = 1, p < .0001). Nonetheless, the general factor Model 2 (see Figure 1) fit significantly better than Model 1A (Δ χ2 = 192.11, df = 9, p < .0001, RMSEA = 0.061 (90% CI: 0.054 – 0.068); CFI = 0.972; BIC = 98488.

Figure 1.

Figure 1

Open in a new tab

Best-fitting structural model of parent-reported psychopathology at 5–11 years of age (Model 2).

Associations with Intelligence

Caspi et al. (2013) found that the general factor of psychopathology in adults was inversely correlated with intelligence. In this sample, controlling demographic covariates in multiple regression, verbal intelligence was slightly but significantly associated with the general (β = −0.85, t = −2.24, p < 0.05), internalizing (β = −0.85, t = −2.35, p < 0.05), and externalizing (β = −1.23, t = −3.27, p < 0.005) factors. Similarly, performance intelligence was associated with the general (β = −1.38, t = −3.81, p < 0.0001), internalizing (β = −0.94, t = −2.54, p < 0.05), and externalizing (β = −1.02, t = −2.85, p < 0.005) factors. That is, for example, two individuals differing by 1 SD on their general factor scores will differ by 1.23 points in performance intelligence on average, with higher general factor scores associated with lower intelligence scores.

Criterion Validity: Independent Associations of Factor Scores with Validity Indicators

Table 1 presents the independent associations of each psychopathology factor score derived from CFA models 1A and 2, controlling for the other psychopathology factor scores, demographic covariates, and verbal and performance intelligence scores. In the correlated factors Model 1A, externalizing was inversely associated with every measure of adaptive school functioning, both concurrently and prospectively, whereas internalizing was positively associated with multiple measures of adaptive school functioning, controlling for the variance accounted for by the externalizing and internalizing factor scores and all covariates.

Table 1.

Descriptive statistics for criteria and independent concurrent and prospective associations of the general, internalizing, and externalizing psychopathology factor predictors estimated across 5–11 years of age with criterion variables, controlling intelligence and demographic covariates.

Concurrent Associations with Criteria Measured across 5–11 years of age
Continuous Criterion Measures Associations of Factor Scores with Criteria
Correlated 2-Factor Model (Model 1A)
General Factor Model (Model 2)
Proportion or Mean (SD) Internalizing Externalizing General Internalizing Externalizing
β t β t β t β t β t
Teacher-rated classroom behavior and mood
 Behaves 3.39 (1.33) 0.26 9.02**** −0.59 −19.33**** −0.24 −9.02**** 0.23 8.53**** −0.30 −10.64****
 Works hard 3.60 (1.24) 0.13 4.73**** −0.40 −12.71**** −0.24 −8.79**** 0.18 6.99**** −0.13 −4.73****
 Happy 3.49 (0.98) 0.00 0.08 −0.27 −11.51**** −0.22 −10.17**** 0.07 3.34*** −0.13 −6.31****
Teacher-rated academic performance
 Reading 2.93 (0.86) 0.00 0.34 −0.14 −8.21**** −0.13 −8.76**** 0.06 4.13**** −0.02 −1.55
 Spelling 2.86 (0.82) 0.01 0.55 −0.14 −8.11**** −0.12 −7.33**** 0.06 3.72*** −0.03 −1.66
 Math 3.05 (0.84) −0.02 −1.05 −0.12 −7.34**** −0.14 −9.05**** 0.05 3.61*** −0.02 −1.02
Teacher CGAS 80.83 (13.42) 0.97 3.11* −4.31 −11.65**** −2.76 −9.45**** 1.57 5.57**** −2.02 −5.69****
Binary Criterion Measures
 Grade ret 12.69% 0.03 0.17 0.30 15.70**** 0.30 16.56**** −0.12 2.74 0.10 2.13
 Spec educ 59.91% −0.04 0.45 0.36 29.56**** 0.24 17.51**** −0.08 2.17 0.14 5.26*
Prospective Associations with Criteria Measured across 12–16 years of age
Continuous Criterion Measures
Teacher-rated classroom behavior and mood
 Behaves 3.26 (1.37) 0.17 5.28**** −0.39 −11.03**** −0.12 −4.01**** 0.10 3.58*** −0.24 −7.66****
 Works hard 3.58 (1.31) 0.10 2.92** −0.28 −7.78**** −0.15 −4.80**** 0.12 3.83**** −0.12 −3.72***
 Happy 3.56 (1.06) 0.02 0.65 −0.24 −8.72**** −0.14 −5.43**** 0.02 0.80 −0.16 −6.42****
Teacher-rated academic performance
 Reading 2.98 (0.90) 0.05 2.62** −0.18 −8.80**** −0.09 −5.19**** 0.05 2.99** −0.07 −3.89****
 Spelling 3.00 (0.83) 0.04 2.19* −0.16 −8.77**** −0.10 −5.69**** 0.06 3.39*** −0.06 −3.51***
 Math 3.05 (0.09) −0.00 −0.13 −0.13 −6.02**** −0.11 −5.81**** 0.03 1.76 −0.04 −2.02*
Teacher CGAS 79.01 (12.82) 1.03 3.39*** −4.11 −12.31**** −1.98 −7.01**** 0.92 3.60*** −2.52 −8.10****
Binary Criterion Measures
 Grade ret 4.31% 0.16 2.24 0.22 3.92* 0.36 12.44*** −0.07 0.27 −0.00 0.00
 In spec educ 36.21% −0.11 2.59 0.48 43.00**** 0.27 18.90**** −0.14 5.32* 0.20 11.34***
Open in a new tab
*

p < .05;

**

p < .01;

***

p < .001;

****

p < .0001

Note: All regression analyses controlled for race-ethnic categories, receipt of public assistance, and verbal and performance intelligence. CGAS = Children’s Global Assessment Scale; spec educ = evaluated for any special education service; in spec educ = received special education for emotional or behavior problems; grade ret = grade retention.

In Model 2, the general psychopathology factor was inversely associated with every measure of adaptive school functioning, both concurrently and prospectively, controlling for the variance accounted for by the externalizing and internalizing factor scores and all covariates. In addition, externalizing factor scores accounted for significant independent variance in school impairment and the need for special services, both concurrently and prospectively, controlling for the other psychopathology factor scores and covariates. The externalizing factor score also accounted for significant unique variance in rated school performance at later ages, but not during childhood. When the general psychopathology and externalizing scores and other covariates were controlled, the internalizing factor score was positively associated with an even greater number of teacher-rated measures of good academic performance and positive mood and behavior in school, and was inversely associated with evaluation for special education services at 5–11 years and receipt of special education services for emotional and behavior problems at 12–16 years of age.

DISCUSSION

The present findings replicate in 5–11 year old girls the improved fit of the general factor model to correlational data found in adults (Lahey, et al., 2012). Furthermore, these findings are inconsistent with predictions based on the alternative hypothesis that the general factor of psychopathology reflects only biased measurement. Had the general factor only been a manifestation of common method variance or other systematic measurement biases, the general factor would not have been related to independent measures of impaired school functioning. Instead, the general psychopathology factor was robustly associated with measures indicating difficulties in academic performance and in meeting the behavioral demands of the classroom, after accounting for variance associated with the externalizing and internalizing factors, intelligence, and demographic covariates. This was the case for both concurrently and prospectively collected teacher data. These prospective associations of the general factor with school functioning in adolescence provide a stringent test of the criterion validity of the general factor, given that teachers of the girls at 12–16 years had little or no knowledge of the girls when parent ratings of psychopathology were obtained at 5–11 years of age. These findings are consistent with longitudinal findings in adults that the general factor robustly predicts future mental health diagnoses when other factors are controlled (Lahey, et al., 2012) and that a total count of symptoms of personality disorder predicts future functioning after variance associated with elements of each disorder that are independent of the total score were controlled (Hopwood et al., 2011). Unlike the present study, however, the same informant reported on symptoms at both times in these studies of adults.

Controlling for the general factor, the internalizing factor, and all covariates, the externalizing factor accounted for additional variance in teacher-rated school functioning, both currently and prospectively. In addition, the externalizing factor in childhood in the general factor model was inversely related to academic functioning in middle and high school when academic demands are greater.

In both the correlated 2-factor model (Model 1A) and in the general factor model (Model 2), the internalizing factor was positively correlated with teacher-rated academic performance and adaptive school behavior, particularly during adolescence, when the other psychopathology factor(s) and all covariates were controlled. That is, higher scores on the internalizing factor in these multiple regression models predicted better academic performance and more adaptive school behavior as judged by teachers. In the general factor model, the internalizing factor was even positively correlated with concurrent teacher ratings of happiness. These findings suggest that the internalizing factor measures child characteristics that teachers view as adaptive, particularly when the general factor of psychopathology is controlled. One possibility is that individual differences in some processes linked with greater emotional problems in girls, including greater inhibition of negative emotions, compliance, and empathy (Keenan & Hipwell, 2005), are viewed as positive qualities by teachers. Consistent with this interpretation, the internalizing factor was found to be correlated with ratings of restrained, obedient, and prosocial behavior in a sample of older youth in a general factor model (Tackett, et al., 2013). It is possible, however, that internalizing scores are still associated with distress in the child, even though teachers view the behavior of children with higher internalizing scores as adaptive. Indeed, teachers may mistakenly view such well-behaved children as being happy. Alternatively when the general factor is controlled, many children with higher internalizing scores may indeed be happy in school because they are successful and teachers respond positively to them.

The present findings support future research to test the substantive hypothesis that the general factor of psychopathology reflects etiologies and mechanisms that are broadly shared by all of the prevalent forms of psychopathology (Lahey, et al., 2011). If this hypothesis is supported, it will mean that studies of the etiology and mechanisms of psychopathology would be more efficient if they take a hierarchical perspective. To take molecular genetic research as an example, our earlier twin study suggested the hypothesis that a substantial proportion of the genetic influences on psychopathology is highly pleiotropic, meaning that these variants simultaneously influence risk for multiple dimensions of psychopathology, whereas other genetic influences are specific to only one dimension of psychopathology (Lahey, et al., 2011). If this view is even partly correct, the identification of molecular genetic risk factors, and the biopsychological mechanisms through which they operate, would require study of each level of the hierarchy while modeling the entire hierarchy. In contrast, our present approach of studying molecular genetic risk factors for one dimension of psychopathology at a time is optimized only for the identification of dimension-specific risk variants. There are similar important implications for the hierarchical study of other risk factors and mechanisms (Lahey, et al., 2011).

Much remains to be learned before the implications of the general factor of psychopathology for clinical applications are understood. The present findings suggest, however, that clinical assessments should be comprehensive and not limited to a single suspected disorder. Furthermore, they raise the possibility that the total number of symptoms conveys important information beyond the number of symptoms in each dimension of psychopathology. More research is needed to test this hypothesis, however.

Key Points.

  • Some theorists have posited a general factor of psychopathology that reflects broadly shared etiological influences and mechanisms.

  • The alternative hypothesis that the general factor is a measurement artifact with no substantive meaning must be tested, however.

  • In confirmatory factor analyses of data on mother-reported symptoms of common forms of psychopathology at 5–11 years of age from a longitudinal study of a representative sample of 2,450 girls, the best-fitting model included general, externalizing, and internalizing factors, replicating previous findings in adults.

  • Controlling intelligence and demographic factors, the general factor accounted for independent variance in teacher-reported measures of functioning, indicating criterion validity.

  • These findings weaken the alternative hypothesis and thereby support studies of the substantive meaning of a general factor of psychopathology.

Acknowledgments

Supported by grants MH056630 and P50MH94267. We gratefully acknowledge the support of the Pittsburgh Public Schools and all other school districts that made the collection of teacher data possible.

References

  1. Achenbach TM, Conners CK, Quay HC, Verhulst FC, Howell CT. Replication of empirically derived syndromes as a basis for taxonomy of child and adolescent psychopathology. Journal of Abnormal Child Psychology. 1989;17:299–323. doi: 10.1007/BF00917401. [DOI] [PubMed] [Google Scholar]
  2. Angold A, Costello EJ. Nosology and measurement in child and adolescent psychiatry. Journal of Child Psychology and Psychiatry. 2009;50:9–15. doi: 10.1111/j.1469-7610.2008.01981.x. [DOI] [PubMed] [Google Scholar]
  3. Angold A, Costello EJ, Erkanli A. Comorbidity. Journal of Child Psychology and Psychiatry and Allied Disciplines. 1999;40:57–87. [PubMed] [Google Scholar]
  4. Birmaher B, Brent DA, Chiappetta L, Bridge J, Monga S, Baugher M. Psychometric properties of the Screen for Child Anxiety Related Emotional Disorders (SCARED): A replication study. Journal of the American Academy of Child and Adolescent Psychiatry. 1999;38:1230–1236. doi: 10.1097/00004583-199910000-00011. [DOI] [PubMed] [Google Scholar]
  5. Birmaher B, Khetarpal S, Brent D, Cully M, Balach L, Kaufman J, Neer SM. The screen for child anxiety related emotional disorders (SCARED): Scale construction and psychometric characteristics. Journal of the American Academy of Child and Adolescent Psychiatry. 1997;36:545–553. doi: 10.1097/00004583-199704000-00018. [DOI] [PubMed] [Google Scholar]
  6. Block J. The challenge of response sets. East Norwalk, CT: Appleton-Century-Crofts; 1965. [Google Scholar]
  7. Brown TA. Confirmatory factor analysis for applied research. New York: Guilford; 2006. [Google Scholar]
  8. Campbell DT, Fiske DW. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin. 1959;56:81–105. doi: 10.1037/h0046016. [DOI] [PubMed] [Google Scholar]
  9. Caspi A, Houts RM, Belsky DW, Goldman-Mellor SJ, Harrington H, Israel S, Moffitt TE. The p factor: One general psychopathology factor in the structure of psychiatric disorders? Clinical Psychological Science. 2014;2:119–137. doi: 10.1177/2167702613497473. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cronbach LJ, Meehl PE. Construct validity in psychological tests. Psychological Bulletin. 1955;52:281–302. doi: 10.1037/h0040957. [DOI] [PubMed] [Google Scholar]
  11. Donders J. A short form of the WISC-III for clinical use. Psychological Assessment. 1997;9:15–20. [Google Scholar]
  12. Gadow K, Sprafkin J. Child symptom inventories Manual-4. Stonybrook, NY: Checkmate Plus; 1994. [Google Scholar]
  13. Gibbons RD, Hedeker DR. Full-information item bifactor analysis. Psychometrika. 1992;57:423–436. [Google Scholar]
  14. Grant BF, Stinson FS, Dawson DA, Chou SP, Dufour MC, Compton W, Kaplan K. Prevalence and co-occurrence of substance use disorders and independent mood and anxiety disorders - Results from the national epidemiologic survey on alcohol and related conditions. Archives of General Psychiatry. 2004;61:807–816. doi: 10.1001/archpsyc.61.8.807. [DOI] [PubMed] [Google Scholar]
  15. Hipwell AE, Keenan K, Kasza K, Loeber R, Stouthamer-Loeber M, Bean T. Reciprocal influences between girls’ conduct problems and depression, and parental punishment and warmth: A six year prospective analysis. Journal of Abnormal Child Psychology. 2008;36:663–677. doi: 10.1007/s10802-007-9206-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Hipwell AE, Stepp S, Feng X, Burke J, Battista DR, Loeber R, Keenan K. Impact of oppositional defiant disorder dimensions on the temporal ordering of conduct problems and depression across childhood and adolescence in girls. Journal of Child Psychology and Psychiatry. 2011;52:1099–1108. doi: 10.1111/j.1469-7610.2011.02448.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Hopwood CJ, Malone JC, Ansell EB, Sanislow CA, Gill CM, McGlashan TH, Morey LC. Personality assessment in DSM-5: Empirical support for rating severity, style, and traits. Journal of Personality Disorders. 2011;25:305–320. doi: 10.1521/pedi.2011.25.3.305. [DOI] [PubMed] [Google Scholar]
  18. Kaufman A, Kaufman J, Balgopal R, McLean J. Comparison of three WISC-III short forms: Weighing psychometric, clinical and practical factors. Journal of Clinical Child Psychology. 1996;25:97–105. [Google Scholar]
  19. Keenan K, Hipwell A, Chung T, Stepp S, Stouthamer-Loeber M, Loeber R, McTigue K. The Pittsburgh Girls Study: Overview and initial findings. Journal of Clinical Child and Adolescent Psychology. 2010;39:506–521. doi: 10.1080/15374416.2010.486320. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Keenan K, Hipwell AE. Preadolescent clues to understanding depression in girls. Clinical Child and Family Psychology Review. 2005;8:89–105. doi: 10.1007/s10567-005-4750-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Krueger RF, Markon KE. Reinterpreting comorbidity: A model-based approach to understanding and classifying psychopathology. Annual Review of Clinical Psychology. 2006a;2:111–133. doi: 10.1146/annurev.clinpsy.2.022305.095213. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Krueger RF, Markon KE. Reinterpreting comorbidity: A model-based approach to understanding and classifying psychopathology. Annual Review of Clinical Psychology. 2006b;2:111–133. doi: 10.1146/annurev.clinpsy.2.022305.095213. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Lahey BB, Applegate B, Hakes JK, Zald DH, Hariri AR, Rathouz PJ. Is there a general factor of prevalent psychopathology during adulthood? Journal of Abnormal Psychology. 2012;121(4):971–977. doi: 10.1037/a0028355. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Lahey BB, Rathouz PJ, Applegate B, Hulle CV, Garriock HA, Urbano RC, Waldman ID. Testing structural models of DSM-IV symptoms of common forms of child and adolescent psychopathology. Journal of Abnormal Child Psychology. 2008;36:187–206. doi: 10.1007/s10802-007-9169-5. [DOI] [PubMed] [Google Scholar]
  25. Lahey BB, Van Hulle CA, Singh AL, Waldman ID, Rathouz PJ. Higher-order genetic and environmental structure of prevalent forms of child and adolescent psychopathology. Archives of General Psychiatry. 2011;68:181–189. doi: 10.1001/archgenpsychiatry.2010.192. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Monga S, Birmaher B, Chiappetta L, Brent D, Kaufman J, Bridge J, Cully M. Screen for child anxiety-related emotional disorders (SCARED): Convergent and divergent validity. Depression and Anxiety. 2000;12:85–91. doi: 10.1002/1520-6394(2000)12:2<85::AID-DA4>3.0.CO;2-2. [DOI] [PubMed] [Google Scholar]
  27. Musek J. A general factor of personality: Evidence for the Big One in the five-factor model. Journal of Research in Personality. 2007;41:1213–1233. [Google Scholar]
  28. Muthén B, Muthén L. Mplus 7.1. Los Angeles: Muthén & Muthén; 2013. [Google Scholar]
  29. Nunnally JC. Psychometric theory. 2. New York: McGraw-Hill; 1978. [Google Scholar]
  30. Pettersson E, Mendle J, Turkheimer E, Horn EE, Ford DC, Simms LJ, Clark LA. Do maladaptive behaviors exist at one or both ends of personality traits? Psychological Assessment. 2014 doi: 10.1037/a0035587. http://dx.doi.org/10.1037/a0035587. [DOI] [PubMed]
  31. Pettersson E, Turkheimer E, Horn EE, Menatti AR. The general factor of personality and evaluation. European Journal of Personality. 2012;26:292–302. [Google Scholar]
  32. Podsakoff PM, MacKenzie SB, Lee JY, Podsakoff NP. Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology. 2003;88:879–903. doi: 10.1037/0021-9101.88.5.879. [DOI] [PubMed] [Google Scholar]
  33. Satorra A, Bentler PM. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika. 2001;66:507–514. doi: 10.1007/s11336-009-9135-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Schneider DJ. Implicit personality theory: Review. Psychological Bulletin. 1973;79:294–309. doi: 10.1037/h0034496. [DOI] [PubMed] [Google Scholar]
  35. Setterberg S, Bird H, Gould M, Shaffer D, Fisher P. Parent and interviewer versions of the Children’s Global Assessment Scale (C-GAS) New York: Colombia University; 1992. [Google Scholar]
  36. Spearman C. The proof and measurement of association between two things. American Journal of Psychology. 1904;15:72–101. [PubMed] [Google Scholar]
  37. Tackett JL, Lahey BB, Van Hulle CA, Waldman ID, Krueger RF, Rathouz PJ. Common genetic influences on negative emotionality and a general psychopathology factor in childhood and adolescence. Journal of Abnormal Psychology. 2013;122:1142–1153. doi: 10.1037/a0034151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Thorndike EL. A constant error in psychological ratings. Journal of Applied Psychology. 1920;4:25–29. [Google Scholar]
  39. Wechsler D. The Wechsler intelligence scale for children. 3. San Antonio, TX: Psychological Corporation; 1991. [Google Scholar]
  40. Wren FJ, Berg EA, Heiden LA, Kinnamon CJ, Ohlson LA, Bridge JA, Bernal MP. Childhood anxiety in a diverse primary care population: Parent-child reports, ethnicity and SCARED factor structure. Journal of the American Academy of Child and Adolescent Psychiatry. 2007;46:332–340. doi: 10.1097/chi.0b013e31802f1267. [DOI] [PubMed] [Google Scholar]

RESOURCES