The wide‐ranging life outcome correlates of a general psychopathology factor in adolescent psychopathology

Odilia M Laceulle; Joanne M Chung; Wilma AM Vollebergh; Johan Ormel

doi:10.1002/pmh.1465

. 2019 Aug 13;14(1):9–29. doi: 10.1002/pmh.1465

The wide‐ranging life outcome correlates of a general psychopathology factor in adolescent psychopathology

Odilia M Laceulle ^1,^✉, Joanne M Chung ², Wilma AM Vollebergh ³, Johan Ormel ⁴

PMCID: PMC7140177 PMID: 31407875

Abstract

Background

The structure of psychopathology has been much debated within the research literature. This study extends previous work by providing comparisons of the links between psychopathology and several life outcomes (temperamental, economic, social, psychological and health) using a three‐correlated‐factors model, a bifactor model, a revised‐bifactor model and a higher‐order model.

Methods

Data from a sample of Dutch adolescents were used (n = 2 230), and psychopathology factors were modelled using self‐reported and parent‐reported longitudinal data from youth across four assessments during adolescence, from ages 11 to 19. Outcome variables were assessed at age 22 using adolescent‐reports and parent‐reports and more objective measures (e.g. body mass index).

Results

While no measurement model was clearly superior, we found modest associations between the psychopathology factors and life outcomes. Importantly, after taking into account a general factor, the associations with life outcomes decreased for the residual parts of thought problems (across all domains) and internalizing problems (for temperamental and psychological outcomes), but not for externalizing problems, compared with the traditional three‐correlated‐factors model. Patterns were similar for adolescent‐reported and parent‐reported data.

Conclusions

Findings suggest that a general factor is related to psychopathology and life outcomes in a meaningful way. Results are discussed in terms of individual differences in propensity to psychopathology and more broadly in light of recent developments concerning the structure of psychopathology. © 2019 The Authors Personality and Mental Health Published by John Wiley & Sons Ltd

Introduction

The structure of psychopathology has been much debated in the research literature. Traditionally, researchers have posited that the structure of psychopathology is characterized by the dimensions of internalizing (INT) and externalizing (EXT) psychopathology (and sometimes a thought problem dimension (THO)1, 2; for the INT/EXT/THO approach, see3, 4). More recently, it has been suggested that psychopathology is better defined by either a bifactor model or a higher‐order model reflecting one or more higher‐order dimensions in addition to INT, EXT and THO and subsequent lower‐order symptoms.5, 6, 7, 8, 9, 10, 11 Over the last years, the bifactor model proposing a general psychopathology factor (GEN), in addition to INT, EXT and THO, has been robustly replicated. However, interpretation and use of the GEN factor models have been shown to be complicated.12, 13 The current study aims at contributing to unveiling the construct validity of the GEN factor providing a direct comparison of the links between psychopathology and life outcomes using a three‐correlated‐factors model (model A; INT/EXT/THO), a bifactor model (model B; INT/EXT/THO/GEN), a revised‐bifactor model (model ′B; INT/EXT/GEN; excluding the THO factor after the THO variance was found to be almost completely subsumed by GEN3, 4;) and a higher‐order model (model C).

Across the literature, definitions of the GEN factor vary. Often, the GEN factor—and higher‐order dimensions more broadly—is assumed to reflect some kind of vulnerability.5, 7, 14 Additionally, the GEN factor may reflect a proneness towards distress that is inherent across diagnostic categories, and as such, people who are high in this proneness may be more likely to meet criteria for any given disorder.15 Further, the GEN factor has been suggested to parallel compromised cognitive or impulse control,16, 17, 18 poor intellectual function,5, 19 disordered form and content of thoughts5 or cybernetic dysfunction, which is failure to make progress towards important goals because of failure of characteristic adaptations.20 These approaches posit that the GEN factor reflects the propensity for adverse (mental health) outcomes.

Risks of general factor modelling have been described repeatedly.21, 22 Specifically, the sometimes‐suggested better fit of a bifactor approach compared with traditional approaches may often be due to its greater complexity, rather than a better reflection of the true structure of psychopathology.23, 24, 25, 26 As a result, superior model fit statistics (i.e. Bayes information criterion and Akaike information criterion) may not provide robust evidence that the GEN factor model better reflects the true structure of psychopathology than do traditional approaches.

In sum, whereas the replicability of the GEN factor has now been well established, its conceptual meaning is debated and it is unclear what the substantive meaning is of the residual parts of INT and EXT after taking into account the GEN factor.27 One way of unveiling the usefulness and meaning of the GEN factor is to investigate the link between the GEN factor and stable individual differences.17, 27 Some studies indeed have found support for the GEN factor being related to personality and temperament traits.5, 9, 17, 19 For example, Tackett and colleagues suggested that the GEN factor is largely accounted for by neuroticism.9 In addition, the construct validity of the GEN factor has been studied by examining associations with school functioning and academic outcomes5, 19, 28 and brain functioning.5, 15 For example, Lahey and colleagues provided evidence of robust and independent associations with a range of teacher‐reported school functioning measures from childhood to adolescence.29 Finally, studies have suggested that the GEN factor is related to more adverse family and developmental history, compared with INT and EXT.5, 15, 29

Taken together, in addition to the robust support that the structure of psychopathology is well reflected by a model including a GEN factor, research has increasingly assessed the GEN factor in terms of criterion validity to examine its scientific and clinical utility. Findings can provide insight for the GEN factor being reflective of a propensity for all types of psychopathology but also being important for a variety of life domains and partially accounting for the traditional link between INT/EXT and life outcomes. Moreover, findings seem to suggest that the GEN factor subsumes most of the ‘severe pathology' of the traditional INT, EXT and THO factors. As such, when the GEN factor is added to the model, the remaining traditional factors may reflect some modest, non‐pathological tendencies for problem behaviours, which may be less important for predicting long‐term life outcomes. Moreover, in our previous work14 and in the study by Caspi and colleagues,5 the THO factor (often including severe pathological symptoms such as psychotic experiences) was completely subsumed by the GEN factor.

Whereas the studies mentioned previously provide support for links between the GEN factor and a range of psychological constructs, they generally do not contrast the three‐correlated‐factors model to GEN factor models in terms of their relations with life outcomes. Moreover, to illuminate the extent that (a) the GEN factor accounts for the links between INT/EXT/THO and life outcomes and (b) the remaining INT/EXT/THO factors augment the GEN factor in explaining individual differences in life outcomes, it is crucial to compare the links between the various factors and life outcomes across models. In the current study, we compare the various psychopathology factors and their respective links with a range of major life outcomes. Specifically, with regard to the measurement models, the current study builds on our previous work replicating the approach of Caspi and colleagues.5, 14 That is, we compared the traditional correlated‐three‐factors model (INT/EXT/THO; model A) with the following three alternative models: a bifactor model (INT/EXT/THO and GEN; model B), a revised‐bifactor model (INT/EXT and GEN; model ′B) and a higher‐order model (model C). In the subsequent structural models, a range of life outcomes were added to the models. Life outcomes were assessed in line with previous work using the same data30 in five major life domains: temperament, economic outcomes, social outcomes, psychological outcomes and health behaviours. Based on previous correlational evidence,5 we hypothesized that the GEN factor is related to more adverse outcomes in all domains, whereas INT, EXT and THO are expected to be related to only some of the outcome variables after the GEN factor is taken into account. Specifically, in the three‐correlated‐factors model, EXT was expected to show the strongest links with social outcomes, whereas INT is expected to show the strongest associations with psychological outcomes. Additionally, we hypothesized that the GEN factor accounts for a substantial part of the links among INT, EXT, THO and life outcomes, as revealed in the literature in which factor scores were based on the traditional three‐correlated‐factors models. As such, we expected that most of the links between respectively INT, EXT, THO and life outcomes would decrease or would not remain significant in a model including a GEN factor. Finally, to test the robustness of the findings and possible effects of common method variance, relations among parent‐reported psychopathology and (primarily adolescent‐reported) life outcomes were also analysed.

Methods

Sample

The TRacking Adolescents' Individual Lives Survey (TRAILS) is a large prospective cohort study of Dutch adolescents who were followed biennially or triennially from 11 to at least 25 years of age. The current research uses data from all five assessment waves (T1 to T5). Children born between 1 October 1989 and 30 September 1991 were eligible to participate, providing they met the inclusion criteria and their schools were willing to participate.31 Over 90% of the schools enrolling a total of 2 935 eligible children agreed to participate in the study. Seventy‐six percent of these children and their parents consented to participate (T1, n = 2 230, mean age = 11.1 years, standard deviation = 0.6, 50.8% girls). Subsequent data collection waves had good retention rates (T2 mean age 13.6, 96%; T3 mean age 16.3, 81%; T4 mean age 19.1, 84%; and T5 mean age 22.3, 80%). Non‐response and attrition during follow‐ups was somewhat higher in males and in adolescents of non‐Western ethnicity, with divorced parents, low socio‐economic status, low intelligence quotient and academic achievement and poor physical health and with behaviour and substance use problems.32 Non‐response showed little to no association with urbanization, parental religiousness, being an only child or recent self‐reports of anxiety and mood problems.33 Each assessment wave was approved by the national ethical committee (CCMO, http://www.ccmo.nl).

TRAILS data are not open source but accessible for researchers outside the TRAILS consortium by submitting a publication proposal. For information on access to the data, see https://www.trails.nl/en/hoofdmenu/data/data-use. All codebooks are available from https://easy.dans.knaw.nl/ui/home. To enable the reproducibility of our analyses, all descriptions and correlations for all study variables are reported in Table S2. All Mplus output files are available at https://osf.io/8275s/.

Measures

A list of variable names and response categories are presented in the supporting information.

Psychopathology. Psychopathology was assessed using a range of adolescent‐report and parent‐report measures. Adolescent‐report measures included the Youth Self Report (T1, T2 and T3), the Adult Self Report (ASR; T4), the Revised Child Anxiety and Depression Scale (T1, T2 and T3) and the Community Assessment of Psychic Experiences (T3). The Youth Self Report and ASR were used to assess anxious‐depressed, withdrawn‐depressed, aggressive behaviour, delinquent behaviour, attention‐hyperactivity problems and thought problems.34, 35 The Revised Child Anxiety and Depression Scale was used to asses generalized anxiety disorder, social anxiety, separation anxiety, panic disorder and obsessive‐compulsive disorder.36, 37 The Community Assessment of Psychic Experiences was used to assess psychotic experiences (both frequency and distress38;).

Parent‐reported psychopathology was also used in the current study. Parent‐reported psychopathology of the adolescent was assessed using the Child Behavior Checklist (T1, T2 and T3), one of the most commonly used parent‐report questionnaires in child and adolescent psychiatric research.39 The symptom dimensions covered by the Child Behavior Checklist are anxious‐depressed, withdrawn‐depressed, aggressive behaviour, delinquent behaviour, attention‐hyperactivity problems and thought problems.

Outcomes. Outcomes covering five life domains were included in the analyses: temperament, economic outcomes, social outcomes, psychosocial outcomes and health behaviours. Outcome variables were selected to correspond with previous work on the TRAILS data.30 Continuous outcome measures were utilized as much as possible. For the few outcome measures for which this was not possible (e.g. being pregnant as a teenager vs. not being pregnant as a teenager), we used binary variables. All outcomes were assessed at T5 when participants were 22 years old, with the exception of suicidal ideation, which was assessed at T4 (and not at T5).

Five temperament traits were included: effortful control, frustration, fearfulness, affiliation and shyness. Temperament was assessed using the Early Adolescent Temperament Questionnaire (parent‐reported40;).

Four economic outcomes were included: (1) attained educational level or current level if still studying; (2) receiving social security benefits due to unemployment or long‐term illness; (3) absenteeism from work; and (4) serious financial difficulties in the past 2 years (adolescent‐reported).

Five social outcomes were included: (1) antisocial behaviour (assessed with the Anti‐social Behavior Questionnaire; adolescent‐reported41); (2) teenage pregnancy (adolescent‐reported); and (3) being let down by a friend or relative, (4) having a serious conflict with somebody at least twice and (5) physical assault (including rape). Teenage pregnancy, being let down, serious conflict and physical assault were all assessed with the Life Event Checklist, asking the adolescents for events that occurred in the last 2 years (adolescent‐reported42).

Six psychological outcomes were included: (1) lifetime serious suicidal ideation (measured at T4; adolescent‐reported43;); (2) use of specialty mental health services as registered in the Psychiatric Case Register North Netherlands: lifetime day treatment or inpatient care; (3) use of specialty mental healthcare in the past 2 years; (4) low levels of happiness and/or life satisfaction (adolescent‐reported; TRAILS questionnaire); (5) poor sleep quality as indexed by the Nottingham Health ;); and (6) feeling lonely in the past 6 months (adolescent‐reported; T5 ASR35;).

Five health behaviours were included: (1) daily smoking (10+ cigarettes per day); (2) alcohol use; (3) cannabis use; (4) body mass index (BMI; as measured during physical examinations or, if unavailable, by adolescent report); and (5) subjective physical health. Health behaviours were assessed with five questions (adolescent‐reported except for BMI; TRAILS questionnaire).

Statistical analyses

All analyses were performed within a structural equation modelling framework in Mplus 8 (45). In the current study, we examined how a three‐correlated‐factors model, a bifactor model, a revised‐bifactor model and a higher‐order model of psychopathology compared with each other when examining their associations with a range of life outcomes. To do so, we specified two measurement models presented in the work of Laceulle and colleagues (14): model A (including INT, EXT and THO) and model ′B (including the GEN factor, and INT and EXT, with INT and EXT being allowed to correlate, and the THO factor being omitted from the model). We extended the previous work by also specifying model B (included the THO factor) and model C (including a GEN factor that was indicated by INT, EXT and THO). We began our model specification by specifying measurement models such that scale scores for the various psychopathology measures were used as manifest indicators of latent symptom variables, and then, depending on the model, latent symptom variables were specified as indicators of the INT, EXT, THO factors and/or a GEN factor. We then specified structural models in which we entered scale scores for the outcome variables as manifest variables, allowed the outcome variables to covary and then regressed them onto the INT, EXT, THO factors and/or GEN factor. We chose a Bayesian approach for estimating our models. This approach differs from the frequentist approach (e.g. maximum likelihood) in the conceptualization and computation of estimates, such that it uses both prior parameter distributions and the likelihood to create posterior distributions from which estimates are drawn. For the applied researcher, this approach is recommended for models that have shown convergence problems, including negative residual variances,46 and for models that are structurally similar to bifactor models, such as multitrait multimethod models.47, 48 Each model was run at least twice, such that proportional scale reduction factor was examined after the first run, and iterations were increased by at least a factor of 2 using the FBITER command in Mplus. Model convergence was determined by obtaining a proportional scale reduction value of <1.1.

The Bayesian estimation approach also provides statistics that differ from traditional analytic techniques. Specifically, this approach does not yield model fit indices that can be interpreted in the same way as a reader might interpret values for popular incremental fit indices such as the Tucker–Lewis index or the comparative fit index and absolute measures of fit such as the root mean square of approximation. Instead, the Bayesian estimation approach provides the comparative measures of the Bayesian information criteria (BIC) and deviance information criteria (DIC49, 50;). Values of the BIC and DIC indicate how well a model might fit the data when compared with an alternative model, with lower values indicating better model fit. Model fit indices are shown in Table 1.

Table 1.

Model fit statistics for measurement and structural models using adolescent reports and parent reports

		Model
		Model A (correlated‐three‐factors)	Model B (bifactor)^*	Model ′B (revised‐bifactor)	Model C (higher‐order)
Adolescent report
Measurement	DIC	180,215.27	179,306.82	179,624.57	180,246.32
	pD	179.11	−6.82	184.84	176.24
	BIC	181,235.80	180,667.64	180,688.32	181,272.20
Structural	DIC	256,566.04	254,944.08	255,309.84	256,425.34
	pD	877.66	680.91	637.48	565.15
	BIC	259 638.35	258,713.64	258,934.59	259,756.40
Parent report
Measurement	DIC	72,396.18	72,248.29	72,087.93	72,180.93
	pD	80.44	−177.16	83.47	79.53
	BIC	72,841.90	73,300.64	72,565.44	72,636.04
Structural	DIC	147,979.35	147,644.26	147,646.69	148,171.16
	pD	518.85	544.92	525.39	466.83
	BIC	151,012.42	150,859.59	150,707.39	150,931.14

Open in a new tab

Fit statistics should be interpreted with caution as the measurement model B did not converge adequately for both adolescent report and parent report. DIC, deviance information criterion; pD, posterior mean of the deviance minus the deviance of the posterior mean; BIC, Bayesian information criterion.

Additionally, the Bayesian estimation approach provides posterior standard deviations—the variance of the posterior distribution for the parameter—and can be seen as a Bayesian analogue to standard errors. Furthermore, in general, given the large sample size and number of estimated associations caution is needed with interpreting the p‐values. Instead, interpretation of the effects is based on standardized effects (βs) and explained variance (R ²). To test the robustness of the findings and effects of common method variance, parent‐reported psychopathology was also analysed. Descriptive statistics and correlations among all study variables are shown in the supporting information.

Results

Descriptive statistics

Descriptive statistics and correlations among all study variables are reported in the supporting information. Plots showing the associations between psychopathology and the various outcomes are also shown in the supporting information. Plots were created in R51 using the ggplot2 package.52

Associations between psychopathology and the various outcomes

Adolescent‐reported psychopathology and life outcomes. Model fit statistics for both the structural and measurement models are shown in Table 1. Path diagrams, factor loadings and ancillary statistics for each of the measurement models can be found in the supporting information. Measurement model B, the bifactor model, did not converge according to the criteria described previously, and as such, fit statistics should be interpreted with caution.5, 14 All other models, including the structural model B, converged according to the criteria described previously. When examining model fit indices such as DIC and BIC, differences between models were small, indicating that there was no clear superior model but rather that the relative quality of the models when compared with another was similar.

The structural models showed a range of associations between the psychopathology factors and outcomes in the various models (Table 2). First, the THO factor showed modest to moderate links with several outcomes in model A (the three‐correlated‐factors model). While more thought problems were generally related to less adaptive outcomes, there were a few exceptions where more thought problems were related to more adaptive outcomes (e.g. shyness: β = −0.25, alcohol use: β = −0.08). When adding GEN to the model (model B), these associations with maladaptive outcomes decreased in effect size as a substantial amount of variance shifted from the THO factor to the GEN factor. Additionally, substantial decreases were, for example, found for fear (β = 0.13 to 0.04), social security benefits (β = 0.20 to 0.10), interpersonal conflicts (β = 0.25 to 0.15) and cannabis use (β = 0.16 to 0.08). In model ′B, the separate THO factor was not specified, forcing all variance into the GEN factor. Second, the INT factor showed moderate to strong links with most temperamental and psychological outcomes in model A (the three‐correlated‐factors model). When adding a GEN factor to the model (model B) associations decreased in effect size as a substantial amount of variance shifted from the INT factor to the GEN factor. For example, the link with shyness decreased from β = 0.43 in the three‐correlated‐factors model to β = 0.08 in the revised‐bifactor model. Also, several links were revealed with the health, social and economic outcomes, indicating that adolescents reporting more INT were lower on, for example, smoking, cannabis and alcohol use, substantial antisocial behaviour and interpersonal conflict and higher on attained educational level. These patterns with INT being related to adaptive outcomes became sometimes even more prominent after adding the GEN factor. Third, the EXT factor showed mainly associations with health outcomes and with a few temperamental and social outcomes. The associations that were found in model A (e.g. less adaptive health outcomes, lower effortful control and more substantial antisocial behaviour) remained in models B and ′B when adding the GEN factor to the model, indicating that little variance shifted to the GEN factor. Fourth, the GEN factor showed substantial links with almost all outcomes in models B, ′B and the higher‐order model (model C). This was also the case for more severe outcomes such as suicidal ideation and recent psychiatric hospitalization, outcomes that were not related to INT and EXT in any of the models. In model ′B (after forcing all THO variance to the GEN factor), however, the strength of the associations was larger than in model B. Strongest associations were found for temperamental outcomes (e.g. effortful control (β = −0.23), frustration (β = 0.29) and fear (β = 0.34)) and psychological outcomes (e.g. suicidal ideation (β = 0.30), specialist mental healthcare (β = 0.30), being unhappy (β = 0.34) and loneliness (β = 0.36)). Models ′B and C showed highly similar associations between the GEN factor and the various outcomes. Finally, variance explained by all psychopathology factors together was calculated for each of the outcome variables. In general, the amount of explained variance was rather modest, with the largest amounts being found for the temperamental outcomes (8–16%). Plots showing the standardized regression coefficients for each factor model and for each outcome domain can be found in the supporting information.

Table 2.

Associations between adolescent‐reported problems and various life outcome variables

		Model A (three‐correlated‐factors)							Model B (bifactor)
						95% CI							95% CI
		B	Posterior SD	β	p	Lower 2.5%	Upper 2.5%	R ²	B	Posterior SD	β	p	Lower 2.5%	Upper 2.5%	R ²
Temperament
Effortful control	GEN							0.161	−0.076	0.021	−0.104	0.000	−0.117	−0.035	0.159
	EXT	−0.263	0.039	−0.360	0.000	−0.344	−0.190		−0.233	0.032	−0.319	0.000	−0.296	−0.171
	INT	0.042	0.051	0.058	0.196	−0.070	0.138		0.013	0.026	0.018	0.315	−0.038	0.065
	THO	−0.001	0.070	−0.002	0.493	−0.132	0.145		0.007	0.034	0.009	0.421	−0.059	0.075
Frustration	GEN							0.141	0.139	0.020	0.198	0.000	0.099	0.179	0.131
	EXT	0.210	0.038	0.300	0.000	0.138	0.290		0.188	0.032	0.269	0.000	0.127	0.251
	INT	0.098	0.057	0.139	0.040	−0.033	0.204		0.024	0.026	0.035	0.174	−0.027	0.075
	THO	−0.041	0.072	−0.058	0.265	−0.196	0.088		−0.038	0.035	−0.054	0.137	−0.106	0.031
Fear	GEN							0.119	0.219	0.019	0.321	0.000	0.181	0.257	0.110
	EXT	−0.020	0.038	−0.029	0.296	−0.094	0.053		0.010	0.030	0.015	0.365	−0.050	0.068
	INT	0.180	0.072	0.264	0.027	−0.060	0.284		0.043	0.026	0.063	0.052	−0.009	0.094
	THO	0.089	0.069	0.130	0.090	−0.045	0.231		0.026	0.033	0.039	0.208	−0.037	0.093
Affiliation	GEN							0.072	0.025	0.017	0.044	0.071	−0.008	0.060	0.115
	EXT	−0.129	0.033	−0.221	0.000	−0.201	−0.068		−0.061	0.027	−0.105	0.011	−0.116	−0.009
	INT	−0.057	0.052	−0.097	0.093	−0.163	0.041		−0.149	0.034	−0.257	0.008	−0.194	−0.104
	THO	0.140	0.066	0.240	0.007	0.032	0.286		0.076	0.032	0.130	0.010	0.016	0.137
Shyness	GEN							0.079	0.116	0.024	0.145	0.000	0.068	0.163	0.160
	EXT	−0.029	0.047	−0.035	0.274	−0.115	0.071		−0.083	0.038	−0.104	0.016	−0.157	−0.008
	INT	0.343	0.134	0.426	0.026	−0.242	0.506		0.298	0.061	0.373	0.008	0.231	0.365
	THO	−0.201	0.103	−0.249	0.009	−0.426	−0.039		−0.126	0.047	−0.158	0.007	−0.209	−0.042
Economic outcomes
Attained educational level (low)	GEN							0.071	0.005	0.020	0.007	0.407	−0.033	0.043	0.083
	EXT	0.088	0.037	0.136	0.013	0.012	0.158		0.093	0.031	0.145	0.002	0.031	0.154
	INT	−0.141	0.076	−0.218	0.028	−0.264	0.093		0.053	0.027	0.083	0.025	0.000	0.103
	THO	0.173	0.074	0.267	0.006	0.051	0.335		0.076	0.035	0.117	0.015	0.009	0.144
Frequent absenteeism work/study	GEN							0.024	0.500	0.178	0.085	0.002	0.152	0.849	0.024
	EXT	−0.014	0.336	−0.002	0.483	−0.688	0.634		−0.044	0.284	−0.007	0.436	−0.608	0.501
	INT	0.159	0.418	0.027	0.345	−0.699	0.968		0.476	0.236	0.081	0.024	0.007	0.934
	THO	0.744	0.595	0.127	0.093	−0.394	1.985		0.316	0.303	0.054	0.144	−0.262	0.941
Social security benefits	GEN							0.047	0.034	0.008	0.121	0.000	0.017	0.051	0.043
	EXT	0.005	0.016	0.017	0.386	−0.028	0.036		0.008	0.014	0.028	0.287	−0.020	0.034
	INT	−0.001	0.022	−0.004	0.477	−0.048	0.040		0.017	0.011	0.060	0.068	−0.006	0.039
	THO	0.058	0.030	0.204	0.021	0.003	0.121		0.029	0.016	0.104	0.026	0.000	0.060
Serious financial difficulties	GEN							0.093	0.129	0.026	0.149	0.000	0.078	0.180	0.091
	EXT	0.113	0.049	0.130	0.015	0.013	0.208		0.122	0.041	0.141	0.003	0.039	0.201
	INT	−0.009	0.068	−0.011	0.441	−0.148	0.121		0.018	0.032	0.021	0.287	−0.045	0.082
	THO	0.172	0.089	0.199	0.023	0.003	0.360		0.098	0.046	0.114	0.016	0.011	0.189
Social outcomes
Substantial antisocial behaviour	GEN							0.129	0.003	0.004	0.024	0.218	−0.004	0.010	0.158
	EXT	0.045	0.007	0.354	0.000	0.031	0.059		0.051	0.006	0.403	0.000	0.040	0.063
	INT	−0.018	0.011	−0.142	0.046	−0.036	0.011		−0.014	0.006	−0.112	0.009	−0.024	−0.004
	THO	0.001	0.013	0.010	0.457	−0.024	0.027		−0.006	0.006	−0.044	0.188	−0.018	0.007
Teenage pregnancy	GEN							0.053	−0.005	0.007	−0.019	0.253	−0.019	0.009	0.069
	EXT	0.031	0.014	0.118	0.014	0.003	0.057		0.027	0.012	0.102	0.013	0.003	0.049
	INT	−0.026	0.020	−0.101	0.075	−0.064	0.019		0.012	0.010	0.046	0.102	−0.007	0.031
	THO	0.028	0.025	0.106	0.117	−0.019	0.079		0.021	0.013	0.079	0.051	−0.005	0.046
Recent let down experience	GEN							0.078	0.058	0.014	0.124	0.000	0.030	0.085	0.074
	EXT	0.048	0.027	0.102	0.040	−0.006	0.101		0.061	0.022	0.130	0.004	0.017	0.105
	INT	−0.005	0.036	−0.011	0.439	−0.081	0.064		−0.004	0.018	−0.009	0.403	−0.039	0.031
	THO	0.077	0.049	0.165	0.046	−0.016	0.179		0.036	0.025	0.076	0.070	−0.014	0.083
Recent interpersonal conflicts	GEN							0.084	0.048	0.013	0.113	0.000	0.023	0.072	0.082
	EXT	0.043	0.025	0.101	0.048	−0.009	0.091		0.045	0.021	0.107	0.019	0.003	0.085
	INT	−0.035	0.038	−0.081	0.150	−0.111	0.044		−0.006	0.016	−0.014	0.355	−0.038	0.026
	THO	0.105	0.047	0.246	0.009	0.023	0.206		0.066	0.024	0.154	0.008	0.020	0.112
Physical assault, including rape	GEN							0.025	0.006	0.006	0.033	0.135	−0.005	0.018	0.029
	EXT	−0.004	0.011	−0.022	0.353	−0.028	0.017		−0.004	0.009	−0.019	0.347	−0.023	0.014
	INT	−0.019	0.017	−0.101	0.093	−0.053	0.017		0.007	0.008	0.038	0.177	−0.008	0.022
	THO	0.041	0.022	0.216	0.019	0.003	0.088		0.025	0.011	0.132	0.013	0.004	0.046
Psychological outcomes
Suicidal ideation	GEN							0.124	0.063	0.009	0.224	0.000	0.046	0.080	0.124
	EXT	−0.003	0.016	−0.012	0.416	−0.035	0.028		−0.008	0.014	−0.030	0.262	−0.036	0.018
	INT	0.071	0.031	0.252	0.027	−0.027	0.117		0.064	0.016	0.228	0.008	0.042	0.088
	THO	0.036	0.029	0.128	0.105	−0.023	0.093		0.029	0.015	0.103	0.025	0.000	0.057
Psychiatric hospitalization	GEN							0.048	0.024	0.006	0.124	0.000	0.013	0.035	0.047
	EXT	0.016	0.010	0.084	0.053	−0.004	0.037		0.009	0.009	0.047	0.148	−0.008	0.026
	INT	0.037	0.018	0.195	0.028	−0.010	0.067		0.027	0.009	0.141	0.008	0.012	0.042
	THO	−0.007	0.020	−0.037	0.347	−0.050	0.029		0.001	0.009	0.004	0.468	−0.018	0.019
Specialist mental healthcare	GEN							0.113	0.148	0.022	0.196	0.000	0.104	0.191	0.115
	EXT	0.119	0.040	0.157	0.002	0.042	0.198		0.078	0.033	0.103	0.010	0.012	0.142
	INT	0.142	0.069	0.189	0.030	−0.042	0.257		0.139	0.037	0.184	0.008	0.081	0.196
	THO	0.035	0.074	0.046	0.314	−0.119	0.175		0.037	0.036	0.049	0.148	−0.034	0.107
Unhappy and/or dissatisfied	GEN							0.153	0.441	0.045	0.299	0.000	0.353	0.528	0.166
	EXT	0.054	0.085	0.036	0.255	−0.107	0.229		0.011	0.068	0.007	0.435	−0.124	0.143
	INT	0.523	0.195	0.352	0.026	−0.295	0.775		0.378	0.085	0.256	0.008	0.260	0.497
	THO	0.029	0.161	0.019	0.425	−0.315	0.327		0.013	0.074	0.009	0.421	−0.128	0.161
Sleep problems	GEN							0.073	0.187	0.033	0.173	0.000	0.122	0.253	0.086
	EXT	−0.031	0.063	−0.029	0.307	−0.155	0.091		−0.054	0.055	−0.050	0.159	−0.165	0.050
	INT	0.068	0.087	0.063	0.212	−0.116	0.235		0.150	0.048	0.139	0.008	0.064	0.233
	THO	0.255	0.116	0.236	0.013	0.041	0.495		0.180	0.062	0.166	0.007	0.067	0.298
Loneliness	GEN							0.211	0.234	0.018	0.385	0.000	0.199	0.269	0.208
	EXT	−0.005	0.036	−0.009	0.440	−0.072	0.069		−0.018	0.028	−0.029	0.262	−0.071	0.037
	INT	0.344	0.122	0.560	0.026	−0.260	0.461		0.152	0.036	0.250	0.008	0.099	0.203
	THO	−0.078	0.076	−0.128	0.114	−0.243	0.048		−0.045	0.031	−0.074	0.068	−0.106	0.016
Health behaviours
Cigarette smoking (daily)	GEN							0.075	0.009	0.011	0.025	0.209	−0.013	0.030	0.096
	EXT	0.080	0.021	0.220	0.000	0.037	0.120		0.107	0.017	0.294	0.000	0.072	0.140
	INT	−0.065	0.037	−0.180	0.030	−0.125	0.041		−0.038	0.016	−0.105	0.011	−0.066	−0.009
	THO	0.056	0.038	0.153	0.055	−0.013	0.138		0.014	0.019	0.040	0.220	−0.022	0.052
Alcohol use	GEN							0.130	−0.055	0.155	−0.010	0.363	−0.361	0.246	0.163
	EXT	1.348	0.297	0.251	0.000	0.791	1.956		1.597	0.253	0.297	0.000	1.118	2.116
	INT	−0.419	0.406	−0.078	0.144	−1.092	0.567		−0.841	0.267	−0.156	0.008	−1.259	−0.413
	THO	−0.423	0.528	−0.079	0.185	−1.546	0.552		−0.356	0.280	−0.066	0.091	−0.900	0.179
Cannabis use	GEN							0.116	−0.118	0.287	−0.012	0.341	−0.678	0.450	0.138
	EXT	2.380	0.527	0.248	0.000	1.357	3.435		2.734	0.459	0.286	0.000	1.807	3.625
	INT	−1.946	1.040	−0.203	0.027	−3.259	2.266		−0.212	0.366	−0.022	0.278	−0.897	0.530
	THO	1.549	0.917	0.162	0.043	−0.225	3.400		0.726	0.478	0.076	0.057	−0.165	1.714
High body mass index	GEN							0.019	0.095	0.131	0.023	0.234	−0.163	0.349	0.029
	EXT	0.566	0.240	0.135	0.009	0.101	1.053		0.423	0.206	0.101	0.020	0.020	0.828
	INT	0.139	0.308	0.033	0.311	−0.444	0.788		0.428	0.181	0.102	0.012	0.085	0.757
	THO	−0.190	0.431	−0.045	0.313	−1.113	0.607		−0.147	0.222	−0.035	0.253	−0.580	0.298
(Low) subjective health	GEN							0.121	0.223	0.025	0.269	0.000	0.175	0.272	0.113
	EXT	0.059	0.047	0.072	0.099	−0.031	0.154		0.050	0.039	0.060	0.102	−0.027	0.124
	INT	0.182	0.080	0.220	0.027	−0.043	0.313		0.088	0.034	0.106	0.010	0.023	0.151
	THO	0.079	0.084	0.096	0.165	−0.096	0.239		0.055	0.042	0.066	0.095	−0.027	0.137

		Model ′B (revised‐bifactor)							Model C (higher‐order)
						95% CI							95% CI
		B	Posterior SD	β	p	Lower 2.5%	Upper 2.5%	R ²	B	Posterior SD	β	p	Lower 2.5%	Upper 2.5%	R ²
Temperament
Effortful control	GEN	−0.169	0.022	−0.233	0.000	−0.212	−0.128	0.162	−0.156	0.021	−0.212	0.000	−0.198	−0.115	0.126
	EXT	−0.109	0.034	−0.150	0.002	−0.172	−0.037
	INT	0.092	0.043	0.126	0.018	0.013	0.165
	THO
Frustration	GEN	0.201	0.021	0.287	0.000	0.161	0.241	0.136	0.201	0.020	0.288	0.000	0.160	0.240	0.132
	EXT	0.084	0.032	0.119	0.007	0.017	0.142
	INT	−0.021	0.035	−0.029	0.273	−0.091	0.048
	THO
Fear	GEN	0.228	0.020	0.335	0.000	0.189	0.266	0.108	0.224	0.019	0.328	0.000	0.186	0.261	0.084
	EXT	−0.021	0.029	−0.031	0.221	−0.082	0.034
	INT	0.035	0.034	0.052	0.149	−0.035	0.099
	THO
Affiliation	GEN	−0.019	0.018	−0.032	0.146	−0.053	0.016	0.053	−0.007	0.017	−0.013	0.332	−0.041	0.026	0.051
	EXT	−0.027	0.027	−0.047	0.158	−0.079	0.028
	INT	0.033	0.032	0.057	0.142	−0.029	0.095
	THO
Shyness	GEN	0.108	0.025	0.135	0.000	0.059	0.157	0.049	0.075	0.024	0.094	0.001	0.029	0.121	0.018
	EXT	−0.084	0.038	−0.104	0.013	−0.160	−0.011
	INT	0.065	0.046	0.082	0.083	−0.035	0.148
	THO
Economic outcomes
Attained educational level (low)	GEN	0.108	0.019	0.168	0.000	0.070	0.146	0.122	0.102	0.019	0.160	0.000	0.065	0.139	0.028
	EXT	0.019	0.035	0.030	0.290	−0.056	0.083
	INT	−0.182	0.058	−0.283	0.016	−0.258	−0.109
	THO
Frequent absenteeism work/study	GEN	0.777	0.177	0.132	0.000	0.431	1.121	0.025	0.716	0.175	0.122	0.000	0.372	1.061	0.016
	EXT	−0.178	0.274	−0.030	0.258	−0.727	0.350
	INT	−0.363	0.324	−0.062	0.120	−0.970	0.304
	THO
Social security benefits	GEN	0.056	0.009	0.199	0.000	0.039	0.073	0.056	0.053	0.008	0.190	0.000	0.037	0.070	0.038
	EXT	−0.006	0.014	−0.021	0.332	−0.035	0.020
	INT	−0.034	0.017	−0.120	0.024	−0.065	0.000
	THO
Serious financial difficulties	GEN	0.202	0.026	0.234	0.000	0.151	0.253	0.089	0.218	0.026	0.253	0.000	0.167	0.269	0.067
	EXT	0.134	0.040	0.155	0.001	0.053	0.212
	INT	−0.001	0.044	−0.001	0.487	−0.089	0.084
	THO
Social outcomes
Substantial antisocial behaviour	GEN	0.012	0.004	0.097	0.001	0.005	0.020	0.163	0.019	0.004	0.151	0.000	0.011	0.027	0.079
	EXT	0.048	0.006	0.381	0.000	0.037	0.061
	INT	0.006	0.007	0.052	0.161	−0.006	0.021
	THO
Teenage pregnancy	GEN	0.017	0.007	0.067	0.008	0.003	0.032	0.067	0.022	0.007	0.087	0.001	0.008	0.036	0.023
	EXT	0.031	0.011	0.118	0.005	0.008	0.052
	INT	−0.020	0.013	−0.077	0.064	−0.044	0.008
	THO
Recent let down experience	GEN	0.089	0.014	0.191	0.000	0.062	0.117	0.075	0.097	0.014	0.210	0.000	0.070	0.125	0.057
	EXT	0.049	0.021	0.105	0.014	0.006	0.090
	INT	−0.012	0.023	−0.027	0.292	−0.059	0.034
	THO
Recent interpersonal conflicts	GEN	0.084	0.013	0.199	0.000	0.060	0.109	0.075	0.093	0.013	0.220	0.000	0.068	0.118	0.054
	EXT	0.044	0.020	0.104	0.015	0.004	0.082
	INT	−0.025	0.023	−0.058	0.131	−0.067	0.022
	THO
Physical assault, including rape	GEN	0.016	0.006	0.086	0.002	0.005	0.028	0.020	0.018	0.006	0.093	0.001	0.007	0.029	0.012
	EXT	0.002	0.009	0.009	0.423	−0.017	0.019
	INT	−0.012	0.011	−0.064	0.117	−0.032	0.009
	THO
Psychological outcomes
Suicidal ideation	GEN	0.084	0.008	0.298	0.000	0.068	0.100	0.093	0.084	0.008	0.297	0.000	0.068	0.100	0.089
	EXT	0.005	0.013	0.018	0.348	−0.022	0.030
	INT	0.007	0.015	0.023	0.329	−0.024	0.035
	THO
Psychiatric hospitalization	GEN	0.033	0.006	0.171	0.000	0.022	0.044	0.035	0.032	0.005	0.167	0.000	0.021	0.043	0.029
	EXT	0.011	0.008	0.055	0.103	−0.006	0.027
	INT	0.007	0.010	0.038	0.219	−0.012	0.026
	THO
Specialist mental healthcare	GEN	0.228	0.022	0.302	0.000	0.186	0.271	0.101	0.219	0.022	0.290	0.000	0.177	0.261	0.085
	EXT	0.043	0.034	0.057	0.108	−0.029	0.105
	INT	−0.027	0.039	−0.036	0.232	−0.107	0.048
	THO
Unhappy and/or dissatisfied	GEN	0.508	0.045	0.343	0.000	0.421	0.597	0.134	0.492	0.043	0.335	0.000	0.407	0.577	0.115
	EXT	0.020	0.068	0.014	0.379	−0.116	0.151
	INT	0.124	0.080	0.084	0.066	−0.047	0.273
	THO
Sleep problems	GEN	0.275	0.033	0.255	0.000	0.210	0.339	0.075	0.262	0.032	0.243	0.000	0.199	0.325	0.061
	EXT	−0.048	0.054	−0.044	0.173	−0.163	0.050
	INT	−0.082	0.063	−0.076	0.078	−0.206	0.040
	THO
Loneliness	GEN	0.222	0.018	0.363	0.000	0.186	0.258	0.183	0.214	0.018	0.352	0.000	0.180	0.250	0.130
	EXT	−0.009	0.028	−0.014	0.372	−0.063	0.045
	INT	0.129	0.043	0.211	0.016	0.053	0.188
	THO
Health behaviours
Cigarette smoking (daily)	GEN	0.036	0.011	0.098	0.001	0.014	0.058	0.108	0.056	0.011	0.154	0.000	0.034	0.077	0.025
	EXT	0.112	0.017	0.309	0.000	0.080	0.147
	INT	0.004	0.019	0.010	0.425	−0.032	0.043
	THO
Alcohol use	GEN	−0.074	0.158	−0.014	0.318	−0.384	0.235	0.187	0.256	0.155	0.047	0.048	−0.045	0.558	0.119
	EXT	1.947	0.295	0.363	0.000	1.420	2.586
	INT	0.999	0.380	0.186	0.016	0.309	1.700
	THO
Cannabis use	GEN	0.932	0.296	0.098	0.001	0.354	1.522	0.148	1.371	0.291	0.142	0.000	0.804	1.947	0.077
	EXT	3.191	0.431	0.334	0.000	2.365	4.068
	INT	−0.061	0.503	−0.006	0.451	−0.962	1.042
	THO
High body mass index	GEN	0.346	0.129	0.083	0.003	0.089	0.595	0.018	0.333	0.128	0.080	0.005	0.083	0.583	0.007
	EXT	0.215	0.202	0.051	0.147	−0.194	0.597
	INT	−0.158	0.227	−0.038	0.235	−0.594	0.293
	THO
(Low) subjective health	GEN	0.263	0.025	0.317	0.000	0.214	0.311	0.111	0.264	0.024	0.319	0.000	0.216	0.312	0.106
	EXT	0.052	0.038	0.063	0.084	−0.024	0.125
	INT	0.056	0.043	0.068	0.096	−0.033	0.135
	THO

Open in a new tab

GEN, general psychopathology; EXT, externalizing problems; INT, internalizing problems; THO, thought problems.

Sex of adolescent (1 = boys, 0 = girls) was included as a covariate in all analyses. All p‐values are one‐tailed.

Parent‐reported psychopathology and life outcomes. Model fit statistics for both the structural and measurement models are shown in Table 1. Path diagrams, factor loadings and ancillary statistics for the measurement models, as well as the associations with outcomes and plots can be found in the supporting information. All models except the measurement model for model B converged adequately with no clear superior model. Most of the patterns found were similar to those indicated for adolescent‐reported psychopathology. However, associations between the parent‐reported psychopathology factors and the (also parent‐reported) temperamental outcomes were stronger in this batch of models compared with the models with adolescent‐reported psychopathology, suggesting common method variance effects.

Discussion

In the current study, we started with multiple measurement models (i.e. the traditional three‐correlated‐factors model (model A), the bifactor model (model B), the revised‐bifactor model (model ′B, as proposed by Caspi et al., 2013) and a higher‐order model (model C)) and then extended these models with the various life outcomes (i.e. structural models A, B, ′B and C). Bayesian fit indices (BIC and DIC) were slightly lower for the three‐correlated‐factors model, but differences were small and do not allow strong conclusions regarding the dominance of any of the models. To probe the usefulness of the different models and the GEN factor in particular, we tested the links between psychopathology and life outcomes across the various models.

First, the THO factor was related to many outcomes in model A. However, the correlations decreased in magnitude when a GEN factor was added to the model, suggesting that a substantial amount of variance (shared between the THO factor and the outcomes) shifted from the THO factor to the GEN factor. This seems to support our earlier findings14 when we found that a bifactor model in which a separate THO factor was not specified (thus forcing all variance into the GEN factor; model ′B) converged better than the bifactor model with a separate THO factor specified (model B). Moreover, this finding might bolster the idea that the GEN factor may primarily reflect disordered form and content of thoughts, which—as suggested by Caspi and colleagues (2013)—may capture the extreme of practically every disorder.5 Second, for INT, somewhat similar associations were found. Specifically, the INT factor showed substantial links with young adult fear, shyness, feelings of unhappiness and loneliness in the three‐correlated‐factors model (model A) that could partially be explained by the GEN factor. Notably, INT was related to a higher attained educational level both in model A and in the bifactor models (models B and ′B), suggesting that INT may have some advantage, which becomes most obvious when controlling for the underlying psychopathology. This findings is in line with recent findings demonstrating a positive link between internalizing problems and academic achievement28 and raises the possibility that individuals with internalizing tendencies but no severe psychopathology have more attentional control,53, 54, 55 which make them do better at school and work.56 Third, with regard to EXT, most of the associations revealed in model A remained in the bifactor models (models B and ′B) (e.g. low effortful control, frustration, antisocial behaviour, smoking, alcohol use, cannabis use and high BMI). Earlier research has found little evidence for an association between EXT and outcomes during adolescence,28 which may be explained by the limited selection of outcomes assessed (i.e. criminal, academic and affective outcomes). However, results from both studies seem to suggest that EXT, in addition to the link between EXT and outcomes, is rather independent of the GEN factor. Moreover, our findings suggest that we included some adolescent‐specific symptoms (e.g. delinquency) that may not be very indicative of underlying psychopathology and health outcomes that are not strongly related to general or severe psychopathology.57 Assessing more chronic or life‐course‐persistent externalizing problems (e.g. antisocial personality disorder) and measuring the health outcomes later in adulthood may shed light on this issue. Finally, with regard to the GEN factor, higher scores on the GEN factor were related to most adverse outcomes across life domains. This supports previous findings providing support for correlates of the GEN factor across different domains of functioning, including personality,5, 9, 17, 19 academic functioning5, 19, 28 and well‐being.28 Moreover, the GEN factor was related to some real‐life and severe adverse outcomes (i.e. BMI and psychiatric hospitalization) that only showed very small links with the psychopathology factors in model A. This is an important finding as it suggests that the GEN factor may not be entirely reflective of individual differences in response styles.5, 29, 58

To investigate the robustness of the associations between psychopathology and the life outcomes, and criterion validity of the GEN factor more specifically, the same links were examined using parent‐reported psychopathology. Patterns found were generally similar to those found for adolescent‐reported psychopathology. However, many of the associations with temperament were smaller in size for the adolescent‐reported psychopathology models, likely reflecting effects of common method variance. Relatedly, for more objective life outcomes such as psychiatric hospitalization and BMI, the strength of the effects was similar for adolescent‐reported and parent‐reported psychopathology. Some exceptions were found with regard to social outcomes (i.e. feeling let down by a close other and experiencing a recent interpersonal conflict), where links were weaker when externalizing problems were reported by the adolescent. It could be that parents are more perceptive to certain externalizing social problems of their child than adolescents are.

Taken together, our findings contribute to the understanding of the construct validity of a GEN factor as well as the meaning of the residual parts of INT and EXT that do not overlap with the GEN factor. Specifically, comparison of the associations with outcomes across the various models demonstrates that each higher‐order model has its own implications for the (interpretation of) both the factors59 and the associations found (e.g.5, 60). While this may not show from the small differences in model fit statistics between the measurement models, the associations with outcomes vary dependent on whether and how the GEN factor is taken into account. For example, whereas an association of β = 0.264 is found between INT and the temperamental trait fear in model A, the association drops to β = 0.063 after taking into account GEN. This suggests that associations traditionally assumed to be domain specific may instead be reflective of an individual's general propensity to develop psychopathology.

The current study is not without limitation. Alternative measurement models have been proposed, some of which we took into account (a higher‐order model), and some of which we did not take into account (removing intercorrelations between factors, as this would hinder shared variance by any two latent factors not shared with the third ) or using different assignment of problem domains to the broader factors.61, 62, 63, 64 Notably, Kotov and colleagues introduced a Hierarchical Taxonomy of Psychopathology (HiTOP11;). In addition to proposing a higher‐order model, HiTOP provides a classification system explicitly stating causal influences on psychopathology dimensions and capturing the full range of psychiatric problems (including personality disorders and eating pathology). As such, HiTOP bridges dimensional models including bifactor approaches with the broad yet categorical approaches to psychopathology as described in the DSM‐5.

Additionally, although all life outcomes were measured after the assessment of psychopathology, we could not control for all outcomes prior to the first measurement of psychopathology. As such, we cannot be conclusive regarding the direction of the effects, nor do we know whether the findings hold when individuals are older (which may point into the direction of a scarring effect30, 65, 66;). Relatedly, the longitudinal design was taken into account by modelling an assessment factor into each model to address age and wave‐related variance. Also, previous work in this field has rarely address the structure of psychopathology longitudinally.58, 67 Future research should take a more explicit developmental approach examining measurement invariance and age‐graded changes in the various models.

Furthermore, for most of the associations, both psychopathology and the outcomes were measured using reports from the same informant (i.e. the adolescent). As such, for these associations, links may partially be the result of correlated measurement error. However, all outcomes in the temperament domain were measured with parent report, and objective measures were used for psychiatric hospitalization (measured using the Psychiatric Case Register North Netherlands) in the psychological domain and BMI (measured during the physiological assessments) in the health domain. Also, outcome items were omitted that were very similarly worded or almost identical to the symptoms, and life events or behavioural outcomes were used whenever possible (i.e. limiting conceptual dependence). Nonetheless, more objective and real‐life outcomes would allow to provide a (more) robust test of the extent to which the GEN factor reflects individual differences in response style.5, 29, 58, 68 Relatedly, although a wide range of outcomes across several domains was taken into account, the current study was by no means exhaustive. For example, only five temperamental traits were taken into account including multifaceted constructs. For example, the negative associations between effortful control and psychopathology may be suppressed by some facets (i.e. activation control) that are positively associated with psychopathology.69 As such, an even broader range of outcomes and more specific individual characteristics might be important to consider.

Finally, it is important to keep in mind that virtually all associations represent weak to modest effects. Regarding the measurement models, differences found might mainly reflect the complexity of the models and are too small to draw strong conclusions in favour of a particular model. Also, the associations between psychopathology factors and outcomes were small in magnitude, as was the amount of variance in outcomes explained by the psychopathology factors. As such, the practical significance of our results is limited. Nonetheless, small effects may accumulate and as such should not be outright rejected,70 and several relative changes in associations across models might be interpreted as moderate (25–49%) to strong (>50%; based on standards used in intervention studies; e.g.71).

The current study sought to increase our understanding of the structure of psychopathology and the GEN factor in particular by providing relative comparisons between multiple measurement models and structural models examining the associations between psychopathology and life outcomes. While there was no clear superior model, modest associations between the psychopathology factors and life outcomes were found that suggest meaningful differences across the various models. Importantly, after taking into account the GEN factor, the associations with life outcomes decreased for the residual parts of THO (all domains) and INT (temperamental and psychological outcomes), but not for EXT, compared with the traditional three‐correlated‐factors model. These findings suggest that problems in the externalizing domain are not always pathological during adolescence and that the residual part of EXT adds explanatory power to the GEN factor in predicting life outcomes.

Supporting information

Table S1. List of study variable names and response categories

Table S2. Descriptive statistics for continuous life outcome variables

Table S3. Factor loadings and covariances for unconditional model A (three‐correlated‐factors) using adolescent report

Table S4. Factor loadings and covariances for unconditional model B (bifactor) using adolescent report

Table S5. Factor loadings and covariances for unconditional model ′B (revised‐bifactor) using adolescent report

Table S6. Factor loadings for unconditional model C (hierarchical) using adolescent report

Table S7. Factor loadings and covariances for unconditional model A (three‐correlated‐factors) using parent report

Table S8. Factor loadings and covariances for unconditional model B (bifactor) using parent report

Table S9. Factor loadings and covariances for unconditional model ′B (revised‐bifactor) using parent report

Table S10. Factor loadings for unconditional model C (hierarchical) using parent report

Table S10. Ancillary statistics for all unconditional models

Table S11. Correlations among mean plausible values for the general factor of psychopathology from unconditional models

Table S12. Associations between parent‐reported psychopathology and various life outcome variables in all four models

Click here for additional data file.^{(356.9KB, docx)}

Figure 1a. Associations between self‐reported psychopathology and temperament variables

Figure 1b. Associations between self‐reported psychopathology and economic outcomes

Figure 1c. Associations between self‐reported psychopathology and social outcomes

Figure 1d. Associations between self‐reported psychopathology and psychological outcomes

Figure 1e. Associations between self‐reported psychopathology and health behaviors

Figure 1f. Associations between parent‐reported psychopathology and temperament variables

Figure 1g. Associations between parent‐reported psychopathology and economic outcomes

Figure 1h. Associations between parent‐reported psychopathology and social outcomes

Figure 1i. Associations between parent‐reported psychopathology and psychological outcomes

Figure 1j. Associations between parent‐reported psychopathology and health behaviors

Click here for additional data file.^{(821.1KB, zip)}

Laceulle, O. M. , Chung, J. M. , Vollebergh, W. A. M. , and Ormel, J. (2020) The wide‐ranging life outcome correlates of a general psychopathology factor in adolescent psychopathology. Personality and Mental Health, 14: 9–29. 10.1002/pmh.1465.

Footnotes

Results from the models using Maximum Likelihood Robust estimation are available here: https://osf.io/8275s/.

Results for models specifying a bifactor structure

References

1. Achenbach TM, & Edelbrock CS. The classification of child psychopathology: a review and analysis of empirical efforts. Psychol Bull 1978; 85(6): 1275–1301. [PubMed] [Google Scholar]
2. Krueger RF, Caspi A, Moffitt TE, Silva PA. The structure and stability of common mental disorders (DSM‐III‐R): a longitudinal‐epidemiological study. J Abnorm Psychol 1998; 107(2): 216–227. [DOI] [PubMed] [Google Scholar]
3. Forbush KT, & Watson D. The structure of common and uncommon mental disorders. Psychol Med 2013; 43(01): 97–108. [DOI] [PubMed] [Google Scholar]
4. Vollebergh WAM, Iedema J, Bijl RV, de Graaf R, Smit F, Ormel J. The structure and stability of common mental disorders: the NEMESIS study. Arch Gen Psychiatry 2001; 58(6): 597–603. [DOI] [PubMed] [Google Scholar]
5. Caspi A, Houts RM, Belsky DW, Goldman‐Mellor SJ, Harrington H, Israel S et al. The p factor: one general psychopathology factor in the structure of psychiatric disorders? Clin Psychol Sci 2013; 2(2): 119–137. [DOI] [PMC free article] [PubMed] [Google Scholar]
6. Kim H, & Eaton NR. The hierarchical structure of common mental disorders: connecting multiple levels of comorbidity, bifactor models, and predictive validity. J Abnorm Psychol 2015; 124(4): 1064–1078. [DOI] [PubMed] [Google Scholar]
7. Lahey BB, Applegate B, Hakes JK, Zald DH, Hariri AR, Rathouz PJ. Is there a general factor of prevalent psychopathology during adulthood? J Abnorm Psychol 2012; 121(4): 971–977. [DOI] [PMC free article] [PubMed] [Google Scholar]
8. Lahey BB, Van Hulle CA, Singh AL, Waldman ID, Rathouz PJ. Higher‐order genetic and environmental structure of prevalent forms of child and adolescent psychopathology. Arch Gen Psychiatry 2011; 68(2): 181–189. [DOI] [PMC free article] [PubMed] [Google Scholar]
9. Tackett JL, Lahey BB, Van Hulle C, Waldman I, Krueger RF, Rathouz PJ. Common genetic influences on negative emotionality and a general psychopathology factor in childhood and adolescence. J Abnorm Psychol 2013; 122(4): 1142–1153. [DOI] [PMC free article] [PubMed] [Google Scholar]
10. Patalay P, Fonagy P, Deighton J, Belsky J, Vostanis P, Wolpert M. A general psychopathology factor in early adolescence. Br J Psychiatry 2015. Jul; 207(1): 15–22. [DOI] [PubMed] [Google Scholar]
11. Kotov R, Krueger RF, Watson D, Achenbach TM, Althoff RR, Bagby RM et al. The Hierarchical Taxonomy of Psychopathology (HiTOP): a dimensional alternative to traditional nosologies. J Abnorm Psychol 2017; 126(4): 454–477. [DOI] [PubMed] [Google Scholar]
12. Widiger TA, & Oltmanns JR. The general factor of psychopathology and personality. Clin Psychol Sci 2017; 5(1): 182–183. [DOI] [PMC free article] [PubMed] [Google Scholar]
13. Bonifay W, Lane SP, Reise SP. Three concerns with applying a bifactor model as a structure of psychopathology. Clin Psychol Sci 2017:2167702616657069; 5: 184–186. [Google Scholar]
14. Laceulle OM, Vollebergh WA, Ormel J. The structure of psychopathology in adolescence replication of a general psychopathology factor in the TRAILS study. Clin Psychol Sci 2015:2167702614560750; 3: 850–860. [Google Scholar]
15. Del Giudice M. The life history model of psychopathology explains the structure of psychiatric disorders and the emergence of the p factor: a simulation study. Clin Psychol Sci 2016; 4(2): 299–311. [Google Scholar]
16. Beauchaine TP, & Thayer JF. Heart rate variability as a transdiagnostic biomarker of psychopathology. Int J Psychophysiol 2015; 98(2): 338–350. [DOI] [PubMed] [Google Scholar]
17. Hankin BL, Davis EP, Snyder H, Young JF, Glynn LM, Sandman CA. Temperament factors and dimensional, latent bifactor models of child psychopathology: transdiagnostic and specific associations in two youth samples. Psychiatry Res 2017; 252: 139–146. [DOI] [PMC free article] [PubMed] [Google Scholar]
18. Carver CS, Johnson SL, Timpano KR. Toward a functional view of the p factor in psychopathology. Clin Psychol Sci 2017; 5(5): 880–889. [DOI] [PMC free article] [PubMed] [Google Scholar]
19. Castellanos‐Ryan N, Brière FN, O'Leary‐Barrett M, Banaschewski T, Bokde A, Bromberg U et al. The structure of psychopathology in adolescence and its common personality and cognitive correlates. J Abnorm Psychol 2016; 125(8): 1039–1052. [DOI] [PMC free article] [PubMed] [Google Scholar]
20. DeYoung CG, & Krueger RF. A cybernetic theory of psychopathology. Psychol Inq 2018; 29(3): 117–138. [Google Scholar]
21. Watts AL, Poore H, Waldman I. Riskier tests of the validity of the bifactor model of psychopathology. 2019. [Google Scholar]
22. van Bork R, Epskamp S, Rhemtulla M, Borsboom D, van der Maas HLJ. What is the p‐factor of psychopathology? Some risks of general factor modeling. Theory Psychol 2017; 27(6): 759–773. [Google Scholar]
23. Gignac GE. The higher‐order model imposes a proportionality constraint: that is why the bifactor model tends to fit better. Intelligence 2016; 55: 57–68. [Google Scholar]
24. Molenaar D. On the distortion of model fit in comparing the bifactor model and the higher‐order factor model. Int 2016; 57: 60–63. [Google Scholar]
25. Mansolf M, & Reise SP. When and why the second‐order and bifactor models are distinguishable. Int 2017; 61: 120–129. [Google Scholar]
26. Morgan GB, Hodge KJ, Wells KE, Watkins MW. Are fit indices biased in favor of bi‐factor models in cognitive ability research?: a comparison of fit in correlated factors, higher‐order, and bi‐factor models via Monte Carlo simulations. J Intelligence 2015; 3(1): 2–20. [Google Scholar]
27. Caspi A, & Moffitt TE. All for one and one for all: mental disorders in one dimension. Am J Psychiatry 2018; 175(9):831–844. [DOI] [PMC free article] [PubMed] [Google Scholar]
28. Sallis H, Szekely E, Neumann A, Jolicoeur‐Martineau A, van IJzendoorn M, Hillegers M et al. General psychopathology, internalising and externalising in children and functional outcomes in late adolescence. J Child Psychol Psychiatry 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
29. Lahey BB, Rathouz PJ, Keenan K, Stepp SD, Loeber R, Hipwell AE. Criterion validity of the general factor of psychopathology in a prospective study of girls. J Child Psychol Psychiatry 2015; 56(4): 415–422. [DOI] [PMC free article] [PubMed] [Google Scholar]
30. Ormel J, Oerlemans A, Raven D, Laceulle O, Hartman C, Veenstra R et al. Functional outcomes of child and adolescent mental disorders. Current disorder most important but psychiatric history matters as well. Psychol Med 2017; 47(7): 1271–1282. [DOI] [PubMed] [Google Scholar]
31. de Winter AF, Oldehinkel AJ, Veenstra R, Brunnekreef JA, Verhulst FC, Ormel J. Evaluation of non‐response bias in mental health determinants and outcomes in a large sample of pre‐adolescents. Eur J Epidemiol 2005; 20(2): 173–181. [DOI] [PubMed] [Google Scholar]
32. Nederhof E, Jörg F, Raven D, Veenstra R, Verhulst FC, Ormel J et al. Benefits of extensive recruitment effort persist during follow‐ups and are consistent across age group and survey method. The TRAILS study. BMC Med Res Methodol 2012; 12(1): 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
33. Oldehinkel AJ, Rosmalen JG, Buitelaar JK, Hoek HW, Ormel J, Raven D et al. Cohort profile update: the TRacking Adolescents' Individual Lives Survey (TRAILS). Int J Epidemiol 2015. Feb; 44(1): 76–76n. [DOI] [PMC free article] [PubMed] [Google Scholar]
34. Achenbach TM. Manual for the youth self‐report and 1991 profile. Burlington, VT: University of Vermont, 1991. [Google Scholar]
35. Verhulst FC, & Achenbach TM. Empirically based assessment and taxonomy of psychopathology: cross‐cultural applications. A review. Eur Child Adolesc Psychiatry 1995; 4(2): 61–76. [DOI] [PubMed] [Google Scholar]
36. Chorpita BF, Yim L, Moffitt C, Umemoto LA, Francis SE. Assessment of symptoms of DSM‐IV anxiety and depression in children: a Revised Child Anxiety and Depression Scale. Behav Res Ther 2000. 8/1; 38(8): 835–855. [DOI] [PubMed] [Google Scholar]
37. Mathyssek CM, Olino TM, Hartman CA, Ormel J, Verhulst FC, Van Oort FVA. Does the Revised Child Anxiety and Depression Scale (RCADS) measure anxiety symptoms consistently across adolescence? The TRAILS study. Int J Methods Psychiatr Res 2013; 22(1): 27–35. [DOI] [PMC free article] [PubMed] [Google Scholar]
38. Wigman JT, Vollebergh WAM, Raaijmakers QA, Iedema J, van Dorsselaer S, Ormel J et al. The Structure of The Extended Psychosis Phenotype in Early Adolescence‐‐A Cross‐sample Replication. Schizophr Bull 2011; 37(4): 850–860. [DOI] [PMC free article] [PubMed] [Google Scholar]
39. Achenbach TM. Manual for the Child Behavior Checklist/4‐18 and 1991 profile. Burlington, VT: University of Vermont, 1991. [Google Scholar]
40. Rothbart MK, Ahadi SA, Evans DE. Temperament and personality: origins and outcomes. J Pers Soc Psychol 2000; 78(1): 122–135. [DOI] [PubMed] [Google Scholar]
41. Sondeijker FEPL, Ferdinand RF, Oldehinkel AJ, Tiemeier H, Ormel J, Verhulst FC. HPA‐axis activity as a predictor of future disruptive behaviors in young adolescents. Psychophysiology 2008; 45(3): 398–404. [DOI] [PubMed] [Google Scholar]
42. Caspi A, Moffitt TE, Thornton A, Freedman D, Amell JW, Harrington H et al. The life history calendar: a research and clinical assessment method for collecting retrospective event‐history data. Int J Methods Psychiatr Res 1996; 6(2): 101–114. [Google Scholar]
43. Kessler RC, & Ustün TB. The World Mental Health (WMH) Survey Initiative Version of the World Health Organization (WHO) Composite International Diagnostic Interview (CIDI). Int J Methods Psychiatr Res 2004; 13(2): 93–121. [DOI] [PMC free article] [PubMed] [Google Scholar]
44. Hunt SM, McKenna S, McEwen J, Williams J, Papp E. The Nottingham Health Profile: subjective health status and medical consultations. Soc Sci Med A 1981; 15(3): 221–229. [DOI] [PubMed] [Google Scholar]
45. Muthén L, Muthén B. Mplus. The comprehensive modelling program for applied researchers: user's guide 2019;5.
46. Muthén B. Bayesian analysis in Mplus: a brief introduction 2010.
47. Helm JL, Castro‐Schilo L, Zavala‐Rojas D, DeCastellarnau A, Oravecz Z. Bayesian estimation of the true score multitrait–multimethod model with a split‐ballot design. Struct Equ Model Multidiscip J 2018; 25(1): 71–85. [Google Scholar]
48. Helm JL, Castro‐Schilo L, Oravecz Z. Bayesian versus maximum likelihood estimation of multitrait–multimethod confirmatory factor models. Struct Equ Model Multidiscip J 2017; 24(1): 17–30. [Google Scholar]
49. Spiegelhalter DJ, Best NG, Carlin BP, Van Der Linde A. Bayesian measures of model complexity and fit. J R Stat Soc Series B Stat Methodology 2002; 64(4): 583–639. [Google Scholar]
50. Gelman A, Carlin JB, Stern HS, Rubin DB. Bayesian data analysis (vol. 2). Chapman: Boca Raton, FL, 2014. [Google Scholar]
51. Team RC. R: A language and environment for statistical computing. 2013.
52. Wickham H. Ggplot2: elegant graphics for data analysis. Springer, 2016. [Google Scholar]
53. Eisenberg N, Sadovsky A, Spinrad TL, Fabes RA, Losoya SH, Valiente C et al. The relations of problem behavior status to children's negative emotionality, effortful control, and impulsivity: current relations and prediction of change. Dev Psychol 2005; 41: 193–211. [DOI] [PMC free article] [PubMed] [Google Scholar]
54. Eisenberg N, Valiente C, Spinrad TL, Cumberland A, Liew J, Reiser M et al. Longitudinal relations of children's effortful control, impulsivity, and negative emotionality to their externalizing, internalizing and co‐occuring behavior problems. Dev Psychol 2009; 45: 988–1008. [DOI] [PMC free article] [PubMed] [Google Scholar]
55. Eisenberg N, Cumberland A, Spinrad TL, Fabes RA, Shepard SA, Reiser M et al. The relations of regulation and emotionality to children's externalizing and internalizing problem behavior. Child Dev 2001; 72: 1112–1134. [DOI] [PubMed] [Google Scholar]
56. Rueda MR, Checa P, Rothbart MK. Contributions of attentional control to socioemotional and academic development. Early Educ Dev 2010; 21(5): 744–764. [Google Scholar]
57. Edwards A, Gardner C, Hickman M, Kendler K. A prospective longitudinal model predicting early adult alcohol problems: evidence for a robust externalizing pathway. Psychol Med 2016; 46(05): 957–968. [DOI] [PMC free article] [PubMed] [Google Scholar]
58. Snyder HR, Young JF, Hankin BL. Strong homotypic continuity in common psychopathology‐, internalizing‐, and externalizing‐specific factors over time in adolescents. Clin Psychol Sci 2017; 5(1): 98–110. [DOI] [PMC free article] [PubMed] [Google Scholar]
59. Kotov R. The general factor of psychopathology: artifact or real? Presented at: International Convention on Psychological Science 2015. 2015.
60. Conway CC, Starr LR, Espejo EP, Brennan PA, Hammen C. Stress responsivity and the structure of common mental disorders: transdiagnostic internalizing and externalizing dimensions are associated with contrasting stress appraisal biases. J Abnorm Psychol 2016; 125(8): 1079–1089. [DOI] [PMC free article] [PubMed] [Google Scholar]
61. Geeraerts SB, Deutz MHF, Deković M, Bunte T, Schoemaker K, Espy KA et al. The Child Behavior Checklist dysregulation profile in preschool children: a broad dysregulation syndrome. J Am Acad Child Adolesc Psychiatry 2015; 54(7): 595–602. e2. [DOI] [PubMed] [Google Scholar]
62. Deutz MH, Geeraerts SB, van Baar AL, Deković M, Prinzie P. The dysregulation profile in middle childhood and adolescence across reporters: factor structure, measurement invariance, and links with self‐harm and suicidal ideation. Eur Child Adolesc Psychiatry 2016; 25(4): 431–442. [DOI] [PMC free article] [PubMed] [Google Scholar]
63. Haltigan JD, Aitken M, Skilling T, Henderson J, Hawke L, Battaglia M et al. “P” and “DP:” examining symptom‐level bifactor models of psychopathology and dysregulation in clinically referred children and adolescents. J Am Acad Child Adolesc Psychiatry 2018; 57(6): 384–396. [DOI] [PubMed] [Google Scholar]
64. McElroy E, Belsky J, Carragher N, Fearon P, Patalay P. Developmental stability of general and specific factors of psychopathology from early childhood to adolescence: dynamic mutualism or p‐differentiation? J Child Psychol Psychiatry 2018; 59(6): 667–675. [DOI] [PMC free article] [PubMed] [Google Scholar]
65. Tackett JL. Evaluating models of the personality–psychopathology relationship in children and adolescents. Clin Psychol Rev 2006; 26(5): 584–599. [DOI] [PubMed] [Google Scholar]
66. Watson D, Kotov R, Gamez W. Basic dimensions of temperament in relation to personality and psychopathology. Pers Psychopathol 2006: 7–38. [Google Scholar]
67. Murray AL, Eisner M, Ribeaud D. The development of the general factor of psychopathology ‘p factor' through childhood and adolescence. J Abnorm Child Psychol 2016; 44(8): 1573–1586. [DOI] [PubMed] [Google Scholar]
68. Borsboom D, & Cramer AO. Network analysis: an integrative approach to the structure of psychopathology. Annu Rev Clin Psychol 2013; 9: 91–121. [DOI] [PubMed] [Google Scholar]
69. Snyder HR, Gulley LD, Bijttebier P, Hartman CA, Oldehinkel AJ, Mezulis A et al. Adolescent emotionality and effortful control: core latent constructs and links to psychopathology and functioning. J Pers Soc Psychol 2015; 109(6): 1132–1149. [DOI] [PMC free article] [PubMed] [Google Scholar]
70. Funder DC, & Ozer DJ. Evaluating effect size in psychological research: sense and nonsense. Adv Methods Pract Psychol Sci 2019:2515245919847202; 2: 156–168. [Google Scholar]
71. Fava M. Diagnosis and definition of treatment‐resistant depression. Biol Psychiatry 2003; 53(8): 649–659. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials