Abstract
Objective:
The current study investigates the correlational structure of psychopathology in a large sample of Canadian adolescents and highlights the association between the psychopathological dimensions and gender.
Method:
Data came from 3826 Canadian adolescents aged 12.8 ± 0.4 y. Five alternative dimensional models were tested using confirmatory factor analysis, and the association between gender, language, and the mean level of psychopathological dimensions was examined using a multiple-indicators multiple-causes model.
Results:
A bifactor model with 1 general psychopathology factor and 3 specific dimensions (internalizing, externalizing, thought disorder) provided the best fit to the data. Results indicated metric invariance of the bifactor structure with respect to language. Females reported higher mean levels of internalizing, and males reported higher mean levels of externalizing. No significant sex differences emerged in liability to thought disorder or general psychopathology. The presence of a general psychopathology factor increased the association between gender and specific dimensions.
Conclusions:
The current study is the first to highlight the bifactor structure including a specific thought disorder factor in a Canadian sample of adolescents. The findings further highlight the importance of transdiagnostic approaches to prevention and intervention among young adolescents.
Keywords: adolescence, bifactor model, externalizing, internalizing, thought disorder, general psychopathology
Abstract
Objectif:
La présente étude recherche la structure corrélationnelle de la psychopathologie dans un vaste échantillon d’adolescents canadiens et présente l’association entre les dimensions psychopathologiques et le sexe.
Méthode:
Les données proviennent de 3 826 adolescents canadiens âgés de 12,8±0,4 ans. Cinq modèles dimensionnels alternatifs ont été testés à l’aide d’une analyse factorielle confirmatoire et de l’association entre le sexe, la langue, et le niveau moyen des dimensions psychopathologiques, qui a été examiné à l’aide d’un modèle à indicateurs et causes multiples.
Résultats:
Un modèle bifactoriel comportant un facteur de psychopathologie générale et trois dimensions spécifiques (internalisation, externalisation, trouble de la pensée) offrait le meilleur ajustement aux données. Les résultats indiquaient une invariance métrique de la structure bifactorielle en ce qui concerne la langue. Les filles déclaraient des niveaux moyens d’internalisation plus élevés et les garçons déclaraient des niveaux moyens d’externalisation plus élevés. Aucune différence significative n’est apparue entre les sexes eu égard au trouble de la pensée ou à la psychopathologie générale. La présence d’un facteur de psychopathologie générale augmentait l’association entre les dimensions sexuelles et spécifiques.
Conclusions:
La présente étude est la première à présenter une structure bifactorielle incluant un facteur spécifique de trouble de la pensée dans un échantillon d’adolescents canadiens. Les résultats soulignent en outre l’importance des approches transdiagnostiques de la prévention et des interventions auprès de jeunes adolescents.
The traditional view that considers mental disorders as distinct categories of syndromes has been challenged by high comorbidity rates among psychiatric disorders.1 Considerable overlap between some disorders might indicate that a dimensional approach may present a more parsimonious view of the structure of psychopathology.2 Although emerging evidence points toward the categorical nature of a number of disorders (e.g., schizotypy, autism), there has been substantial research suggesting that underlying transdiagnostic dimensions may explain patterns of comorbidity.3 The seminal study by Krueger et al.4 suggested a transdiagnostic model of psychopathology with 2 dimensions representing internalizing (mood and anxiety disorders) and externalizing (antisocial and impulsivity-related disorders). This bidimensional structure has received robust support from studies based on community and clinical samples regardless of gender or ethnicity.5 A small number of studies have recently incorporated psychosis-related symptoms into their models, pointing toward the presence of a distinct thought disorder spectrum.6–8
There is a substantial correlation between transdiagnostic dimensions, which motivated a number of recent studies showing that a bifactor model, with an orthogonal general factor along with specific dimensions, provides better model fit than the correlated factor models. The bifactor model includes a general dimension, also called general p factor,7 capturing the common variance shared across all indicators (i.e., underlying liability to experience all forms of psychopathology), as well as specific dimensions reflecting the residual shared variance among specific symptom clusters (i.e., internalizing, externalizing, and thought disorder). Throughout this article, we are going to use the terms general p factor and specific dimensions to distinguish between these constructs. The bifactor structure of psychopathology has been replicated across different developmental periods with a range of different psychopathological symptoms/disorders.6,9–13 Indeed, literature has demonstrated that the p factor remained stable from age 7 to 16 y.2,7 However, the literature still faces several inconsistencies regarding the structure of psychopathology, specifically in earlier periods of life. Therefore, the current study focuses on the comparison between alternative models of psychopathology in a large sample of Canadian adolescents.
Recent literature in the field has focused on examining the efficiency of bifactor models, whereby all dimensions are constrained to be orthogonal to verify that the general factor accounts for associations among specific dimensions. Some studies indicated that the p factor accounts for covariation between specific dimensions,9,14 whereas others suggest that the p factor does not completely account for covariation between specific dimensions and common variance remains for specific dimensions to be correlated, albeit with attenuated loadings.6 It is noteworthy that the latter study focused on the symptom-level assessment of psychopathology while 2 former studies focused on the disorder-level assessment. This might highlight the limitations of disorders as building blocks for the assessment of dimensional structure of psychopathology (e.g., considerable overlap between diagnostic criteria of some disorders, arbitrary thresholds used to establish different diagnoses).15 Recent network analysis studies underscored the role of micro-level associations between overlapping/nonoverlapping symptoms that would have been masked at the disorder level.16,17 The symptom-level analysis of the structure of psychopathology can thus facilitate the emergence of new dimensions or interdimensional correlations by unpacking low-prevalence disorders and by using more precise measurement units.18
Most of the structural studies to date have been confined to adult samples, and few studies examine alternative models of adolescent psychopathology. Adolescence is a critical period during which mental disorders such as depression, substance use, and psychotic disorders emerge.1,19 Existing adolescent studies provide evidence indicating a general psychopathology factor with high levels of temporal stability.14 However, results concerning the number and the structure of specific dimensions are inconsistent. For instance, a study of Dutch adolescents found support for general psychopathology and orthogonal internalizing and externalizing dimensions. However, they failed to find a thought disorder dimension, perhaps because of the use of disorder-level units of measurement in their models.20 In the same vein, 2 recent studies identified a bifactor best-fitting structure with a general psychopathology factor and orthogonal internalizing and externalizing dimensions.9,21 However, both studies did not include measures of psychotic symptoms, and one did not examine if the non–orthogonal-specific dimensions could improve the fit of the bifactor model. In contrast, a study of Australian adolescents6 reported that a modified bifactor model, with 3 non–orthogonal-specific dimensions (internalizing, externalizing, thought disorder), provided the best fit.
Another significant issue concerns the role of gender in the structure of psychopathology among adolescents with theoretical implications in terms of etiology and clinical implications in terms of targeted intervention. Only a few studies based on adolescent samples have focused on the role of gender. A recent study indicated that the general and externalizing factors differed significantly across gender, but distress and fear dimensions (internalizing subdimensions) did not differ across gender.14 These results are in part consistent with the results of other studies that reported a gender difference in internalizing and externalizing dimensions and no gender differences in the general psychopathology factor dimension.6,21 One study also reported that after controlling for the variance associated with general vulnerability, associations between gender and the internalizing and externalizing dimensions increased.21 Concerning the thought disorder dimension, one aforementioned study6 reported absence of gender difference, which is in contrast to an epidemiological study of Spanish adolescents indicating gender disparities in thought disorder symptoms, with higher rates of ideas of reference and paranoid ideation among males.22
Given the relative dearth of literature examining the symptom-level structure of psychopathology among adolescents and gender differences, this study investigated the structure of psychopathology among a large sample of Canadian adolescents. The main goal was to test alternative dimensional models based on the previous literature and theory using symptom-level indicators to evaluate the best-fitting structure of psychopathology. This will provide the opportunity to compare the indicator characteristics with previously published studies in a cross-cultural context and will eventually contribute to future meta-analytic studies of cross-cultural differences in the structure of psychopathology. Subsequently, focusing on the bilingual nature of the sample, we examined the structural invariance across French-speaking and English-speaking adolescents. Finally, we examined to what extent the dimensions in the best-fitting model are related to gender.
Method
Participants and Procedure
This study is a part of the Coventure study,23 an ongoing cluster randomized controlled trial evaluating the effectiveness of school-based personality-targeted interventions on psychopathological outcomes. In total, 3826 grade 7 adolescents (12.8 ± 0.4 years old, 49.2% girls) from 31 secondary schools in the greater Montreal area were invited to complete a confidential web-based survey during class time to assess psychopathological symptoms.
Measures
The instruments used to measure internalizing, externalizing, and psychotic symptoms are described below. Indicators of internalizing and externalizing dimensions were partly measured using the Strengths and Difficulties Questionnaire (SDQ), a brief, 25-item instrument to assess emotional and behavioral problems in children and adolescents.24,25 The SDQ has been validated in clinical practice, community, and epidemiological settings across different countries.26,27 Eight items from the SDQ emotional and peer scales were used as internalizing indicators. Likewise, 12 items from the depression and anxiety scales of the Brief Symptom Inventory were used as internalizing indicators. This scale has demonstrated high test-retest reliability and validity.28 The externalizing dimension was assessed using 7 items from the SDQ behavioral problems and hyperactivity scales. Finally, 9 items of Adolescent Psychotic-Like Symptoms Screener was used to assess hallucinatory experiences and delusional beliefs. This measure has been validated using item response theory.29 In line with the literature,6 and considering the limited range of severity in our community-based sample, Likert-type items were recoded into dichotomous variables. This procedure improves statistical power and yields stable estimates.
Statistical Analysis
Analyses involved 2 steps. First, alternative dimensional models studied in the literature,6,7,9 were estimated using confirmatory factor analysis. Model A estimates a 1-factor model. Model B comprises 3 correlated dimensions (internalizing, externalizing, thought disorder). Model C tests a modified bifactor model with 3 correlated dimensions and a general psychopathology factor. Model D tests a classic bifactor model with 3 uncorrelated dimensions and a general psychopathology factor. Model E estimates a higher-order model in which one overarching factor explains 3 specific dimensions. In the second step, we examined the structural invariance of the best-fitting model between French-speaking and English-speaking adolescents. Finally, differences in the psychopathological dimensions based on gender and language were examined using a multiple-indicators multiple-causes (MIMIC) model.
All models were estimated in Mplus v.7.3, using robust weighted least squares estimation. Analyses were adjusted for school-level clustering. Model fit was evaluated using the root mean square error of approximation (RMSEA; <0.05), comparative fit index (CFI; >0.95), and Tucker-Lewis index (TLI; >0.95).30 Models were rerun with robust maximum likelihood estimation to generate the Bayesian information criterion (BIC) and Akaike’s information criterion. A 6- to 10-point difference in BIC values indicates better performance of the model showing the lower BIC value. In addition, interdimension correlations >0.80 evidence multicollinearity and lack of discriminant validity between dimensions.30 Finally, recent literature brought up issues concerning the bifactor model such as difficulty of the interpretation of the general factor, possibility of overfitting, and criterion and predictive validity of the model.31 To rule out the overfitting and to ensure the consistency of a bifactor model, the literature strongly suggests the evaluation of reliability using a range the omega indices.32,33 These indices were calculated for the model with lowest BIC value.
Results
The Structure of Adolescent Psychopathology
Goodness-of-fit indices and factor loadings/factor correlations are presented in Tables 1 and 2, respectively. Model A did not fit the data well (CFI and TLI <0.90). Model B provided a good fit to the data (CFI = 0.95, TLI = 0.95, RMSEA = 0.021), and interfactor correlations were significant and moderate in size, ruling out multicollinearity. Model C provided an excellent fit to the data (CFI = 0.96, TLI = 0.96, RMSEA = 0.018), and interfactor correlations were small in size ruling out multicollinearity. Model D provided an excellent fit to the data, albeit the fit indices deteriorated marginally compared with model C (CFI = 0.96, TLI = 0.95, RMSEA = 0.018). Finally, model E provided good fit to the data (CFI = 0.95, TLI = 0.95, RMSEA = 0.021), and loadings on the higher-order factor were all significant and moderate in size. The BIC for model C displayed a 10-point difference from other models. Focusing on the omega measures,32,33 the whole structure in model C presented an excellent reliability (ω = 0.96), the general factor presented a moderate reliability (ωH = 0.70), and specific dimensions had moderate to high reliability (internalizing, ωs = 0.85; externalizing, ωs = 0.71; thought disorder, ωs = 0.77). In this model, anger, being unhappy, and being bullied had the highest loadings for general psychopathology; sadness, loneliness, and nervousness had the highest loadings for internalizing; anger, restlessness, and fidgeting had the highest loadings for externalizing; and auditory and visual hallucinations had the highest loadings for thought disorder. However, it is noteworthy that the interfactor correlations between the specific dimensions are low, and the factor loadings of model C and model D are similar. Therefore, it is hard to claim that model C is distinctively better than model D.
Table 1.
Model | No. of Parameters | χ2 | df | CFI | TLI | RMSEA (90% CI) | AIC | BIC |
---|---|---|---|---|---|---|---|---|
Model A: 1 factor | 72 | 3545.342 | 594 | 0.850 | 0.840 | 0.037 (0.036, 0.038) | 131381.712 | 131829.212 |
Model B: correlated 3 factors | 75 | 1538.911 | 591 | 0.951 | 0.948 | 0.021 (0.020, 0.022) | 127337.061 | 127803.207 |
Model C: modified bifactor | 111 | 1244.031 | 555 | 0.964 | 0.960 | 0.018 (0.017, 0.020) | 126307.639 | 126997.535 |
Model D: classic bifactor | 108 | 1261.427 | 558 | 0.960 | 0.954 | 0.018 (0.017, 0.020) | 126348.275 | 127019.525 |
Model E: higher order | 75 | 1538.911 | 591 | 0.951 | 0.948 | 0.021 (0.020,0.022) | 131654.894 | 132114.825 |
χ2, chi-square statistic; df, degree of freedom; CFI, comparative fit index; TLI, Tucker-Lewis index; RMSEA, root mean square error of approximation; CI, confidence interval; AIC, Akaike’s information criterion; BIC, Bayesian information criterion.
aBolded values indicate best fitting model.
Table 2.
Model A | Model B | Model C | Model D | Model E | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Symptom | P | INT | EXT | TD | P | INT | EXT | TD | P | INT | EXT | TD | INT | EXT | TD |
Somatization | 0.46 | 0.47 | 0.48 | 0.17 | 0.51 | 0.07 | 0.47 | ||||||||
Solidarity | 0.41 | 0.43 | 0.37 | 0.22 | 0.39 | 0.17 | 0.43 | ||||||||
Worries | 0.55 | 0.57 | 0.47 | 0.34 | 0.52 | 0.24 | 0.57 | ||||||||
Unhappy | 0.72 | 0.74 | 0.60 | 0.44 | 0.65 | 0.35 | 0.74 | ||||||||
Nervous | 0.59 | 0.61 | 0.52 | 0.34 | 0.57 | 0.24 | 0.61 | ||||||||
Bullied | 0.52 | 0.53 | 0.62 | 0.09 | 0.62 | -0.01 | 0.53 | ||||||||
Better with adults | 0.29 | 0.30 | 0.34 | 0.05 | 0.35 | -0.02 | 0.30 | ||||||||
Fearful | 0.54 | 0.55 | 0.41 | 0.37 | 0.51 | 0.23 | 0.55 | ||||||||
Ending life | 0.69 | 0.71 | 0.53 | 0.47 | 0.60 | 0.37 | 0.71 | ||||||||
Lonely | 0.74 | 0.77 | 0.40 | 0.70 | 0.51 | 0.64 | 0.77 | ||||||||
Sad | 0.79 | 0.80 | 0.36 | 0.78 | 0.48 | 0.73 | 0.80 | ||||||||
No interest | 0.71 | 0.74 | 0.46 | 0.58 | 0.54 | 0.51 | 0.74 | ||||||||
Hopelessness | 0.72 | 0.75 | 0.48 | 0.58 | 0.56 | 0.51 | 0.75 | ||||||||
Worthlessness | 0.75 | 0.79 | 0.48 | 0.64 | 0.58 | 0.56 | 0.79 | ||||||||
Feelings hurt | 0.75 | 0.77 | 0.49 | 0.59 | 0.59 | 0.50 | 0.77 | ||||||||
Feeling tense | 0.65 | 0.69 | 0.49 | 0.49 | 0.57 | 0.38 | 0.69 | ||||||||
Scared | 0.75 | 0.78 | 0.53 | 0.57 | 0.67 | 0.39 | 0.78 | ||||||||
Tearful | 0.76 | 0.79 | 0.44 | 0.68 | 0.59 | 0.54 | 0.79 | ||||||||
Nervous | 0.72 | 0.76 | 0.44 | 0.64 | 0.58 | 0.50 | 0.76 | ||||||||
Panic | 0.74 | 0.77 | 0.51 | 0.57 | 0.65 | 0.40 | 0.77 | ||||||||
Restless | 0.40 | 0.59 | 0.33 | 0.79 | 0.34 | 0.75 | 0.58 | ||||||||
Anger | 0.49 | 0.68 | 0.65 | 0.51 | 0.57 | 0.11 | 0.68 | ||||||||
Fidgeting | 0.45 | 0.63 | 0.43 | 0.68 | 0.43 | 0.68 | 0.63 | ||||||||
Fighting | 0.34 | 0.49 | 0.42 | 0.17 | 0.36 | 0.27 | 0.48 | ||||||||
Distracted | 0.47 | 0.65 | 0.54 | 0.31 | 0.50 | 0.35 | 0.65 | ||||||||
Lying | 0.38 | 0.53 | 0.48 | 0.14 | 0.42 | 0.22 | 0.53 | ||||||||
Stealing | 0.35 | 0.47 | 0.43 | 0.10 | 0.38 | 0.19 | 0.47 | ||||||||
Thoughts read | 0.47 | 0.66 | 0.31 | 0.60 | 0.37 | 0.56 | 0.66 | ||||||||
Special messages | 0.47 | 0.67 | 0.31 | 0.61 | 0.37 | 0.58 | 0.67 | ||||||||
Spied upon | 0.49 | 0.70 | 0.43 | 0.50 | 0.47 | 0.44 | 0.70 | ||||||||
Heard voices | 0.60 | 0.83 | 0.45 | 0.67 | 0.51 | 0.63 | 0.83 | ||||||||
Controlled | 0.55 | 0.80 | 0.38 | 0.72 | 0.44 | 0.70 | 0.80 | ||||||||
Reads minds | 0.43 | 0.64 | 0.28 | 0.61 | 0.36 | 0.54 | 0.64 | ||||||||
Body changed | 0.59 | 0.76 | 0.45 | 0.59 | 0.50 | 0.55 | 0.76 | ||||||||
Special power | 0.53 | 0.75 | 0.32 | 0.71 | 0.38 | 0.70 | 0.75 | ||||||||
Visual Hallucinations | 0.56 | 0.81 | 0.38 | 0.73 | 0.45 | 0.69 | 0.81 | ||||||||
Factor correlations | |||||||||||||||
Internalizing | — | — | 0.59 | 0.47 | — | 0.11 | 0.21 | — | — | — | — | — | — | — | |
Externalizing | — | — | — | 0.44 | — | — | 0.06 | — | — | — | — | — | — | — | |
General psychopathology | — | — | — | — | — | — | — | — | — | — | — | 0.80 | 0.74 | 0.59 |
INT, internalizing; EXT, externalizing; TD, thought disorder; P, general psychopathology.
aFactor loadings and correlations with a P value <0.05 are displayed in boldface.
To investigate whether alternative models yield a better fit, and in line with the literature,6,7 we conducted exploratory factor analysis (EFA) specifying the different number of factors and different structures going from a unidimensional structure to a bifactor structure with 2 specific dimensions. None of these exploratory models fit better than the modified bifactor model (BIC range for EFA models = 131883-127419; BIC value of model C = 126997, BIC value of model D = 127019). In summary, after testing a series of EFA and confirmatory factor analysis models, those fit statistics suggests that, apart from the unidimensional, models with three specific dimensions provide excellent fit and could be considered as plausible. Moreover, comparison of the BIC measures indicated that the bifactor structure (model C and D) provided a meaningful alternative to simple factor models, enabling simultaneous estimation of general and specific factors. Subsequently, a series of sensitivity analysis models were estimated to ensure the robustness of the findings. In the first step, to rule out the sampling bias, the aforementioned models were estimated on several random splits of our data. Similar patterns of results indicated plausibility of the models with 3 specific dimensions and superior fit of the bifactor structure (model C and D). In the second step, all of the models were estimated excluding the items that were not significantly related to any specific dimensions in the bifactor structure (model C and D). In the same vein, results indicated plausibility of the models with 3 specific dimensions and superior fit of the bifactor structure. Finally, we examined the structural invariance across French-speaking and English-speaking adolescents. Results indicated metric invariance of the bifactor structure (model C) between French-speaking and English-speaking adolescents (−2 ▵LL[68] = 82.57, P = 0.11) pointing toward equality of the factor loadings, and lack of scalar invariance (−2 ▵LL[36] = 53.55, P = 0.03) pointing to the difference in the base rate of the symptoms between French-speaking and English-speaking adolescents.
Differences in Psychopathological Liabilities
MIMIC modeling was used to regress the latent factors in model C on gender and language (i.e., indirect effects only model). The modification indices did not support the inclusion of direct paths between individual indicators gender and language permitting direct comparisons of factor means. This model provided an excellent fit to the data (CFI = 0.96, TLI = 0.96, RMSEA = 0.018). Females reported higher mean levels of latent internalizing (β = 0.161, P > 0.001) and thought disorder (β = 0.218, P > 0.001) than males, whereas males reported higher mean levels of externalizing than females (β = –0.142, P > 0.05). No significant gender differences were observed in terms of the general psychopathology dimension. As mentioned above, to compare the effect of general p factor on the associations between gender and specific dimensions, a similar MIMIC model was performed on model B (3 correlated specific dimensions). Results showed that the magnitude of the associations between gender and the specific dimensions was lower compared with the results of model D. Likewise, after controlling for effect of gender and socioeconomic status, there was no significant difference in terms of psychopathological dimensions between French-speaking and English-speaking adolescents.
Discussion
To our knowledge, the current study is the first to examine the structure of psychopathology in a community sample of Canadian adolescents and to establish the association between gender and resulting latent dimensions in Canada. The current study contributes to recent literature using dimensional models to examine underlying structure of adolescent psychopathology.6,7,9,20,21 Aligned with the prior studies, the findings herein demonstrated excellent fit of bifactor models (i.e., 3 specific dimensions and a general p factor). We addressed shortcomings associated with previous adolescent research of covariance structure by using symptom-level analysis and examining additional exploratory models to facilitate identification of new factors as well as including well-established indicators of thought disorder.
Our findings are in line with research from Australia,6 New Zealand,7 the United Kingdom,21,34 the Netherlands,20 and a multinational European study9 identifying a general psychopathology factor and specific internalizing and externalizing dimensions. Consistent with the adult literature, while the mean level of some specific dimensions were related to gender, the overall structure of adolescent psychopathology was invariant with respect to gender and language.6 Aiming toward comparison of item characteristics between the samples, our study used the exact same indicators used in a similar study on an Australian sample of adolescents.6 Both studies highlighted feelings of loneliness and sadness among the most discriminant indicators of the internalizing dimension, grandiosity delusions and auditory/visual hallucinations among the most discriminant indicators of the thought disorders dimension. However, in contrast to the Australian sample, the most discriminant indicators of the externalizing dimension in our sample are attention-deficit–hyperactivity disorder (ADHD) indicators (e.g., restlessness) and not conduct disorder indicators (e.g., stealing). Likewise, our results are in contrast with prior studies that encountered problems estimating a specific thought disorder factor7,20 and non–substance-related externalizing specific dimension that is measured by ADHD, conduct disorder, and oppositional defiant disorder indicators.9 This could be due to symptom-level analysis along with a greater number of externalizing and thought disorder indicators, which conferred greater capacity to model these dimensions.
Consistent with prior studies, gender was not associated with general psychopathology. Females reported higher mean levels of latent internalizing, and males reported higher mean levels of externalizing.6,21 Moreover, our results highlight that the associations between gender and the specific dimensions increased after accounting for the general psychopathology factor. A range of factors, such as hormones, might explain differential tendencies to specific psychopathological dimensions. However, our findings suggest that gender is not a major component of the etiology and development of the general liability to psychopathology.
The current study has a number of potential implications in terms of evaluation, intervention, and nosology research. From an evaluation perspective, the estimated scores for the p factor represent the general liability to psychopathology (related to reactivity to emotions and emotion regulation difficulties35,36), and the estimated scores for specific dimensions represent the liability to each psychopathological profile. Building on the results presented in the current study, a computerized adaptive test of adolescence psychopathology can provide the possibility of a time-efficient and accurate estimation of factor scores that are applicable in clinical and school settings.37,38 From an intervention perspective, our findings suggest that intervention/prevention strategies should be tailored according to general or specific risk profiles identified by the aforementioned evaluation. For instance, given the lack of correlation between the p factor and specific dimensions, it is possible for an individual to score high on the p factor and moderate on the specific dimensions, suggesting the individual may benefit from transdiagnostic prevention/intervention programs more than dimension-specific programs.39 Finally, in terms of implications for nosology research in psychopathology, the classical categorical versus dimensional debate disregards the general p factor that might be informative in terms of common liability to experience all types of psychopathology related to reactivity to emotions and emotion regulation difficulties.35,36 Literature suggests that the p factor may account for the nonspecific consequences of psychopathological conditions such as risk behaviors as well as difficulties in identifying unique neurological markers and distinctive treatments for individual disorders.6,7,40
The above findings should be considered in light of the following limitations. First, although our study uses the exact same indicators (symptoms) and dichotomization strategy used in the literature,6 it is noteworthy that the indicators come from different instruments and were administered separately. Future work needs to build on our results by examining a larger item bank that targets all dimensions administered in a randomized order. Therefore, the current study will set on track the development and calibration of a bilingual large item pool that provides the possibility of fine-tuning the first Canadian computerized adaptive test of adolescence psychopathology. Second, our study does not represent the full range of conditions experienced in childhood and adolescence. Particularly, symptoms of early-onset disorders such as autism and separation anxiety disorder were not assessed in our sample. A more comprehensive set of indicators would provide more detailed insights into the structure of adolescent psychopathology. Likewise, our analysis of the externalizing dimension focused on conduct disorder/ADHD symptoms due to low endorsement of drug and alcohol misuse in our sample at 13 y old. However, the increasing role of substance use symptoms in the structure of psychopathology will be studied in the subsequent waves of the Coventure study. Third, the current study used a large school-based sample. The use of community-sample data overcomes some inherent problems with clinical samples. Clinical samples typically display a restricted range of symptomatology and motivation toward treatment, which hampers generalizability (i.e., Berkson’s bias). However, the resulting bifactor structure should be examined and validated in large samples with a relatively high base rate of psychopathological symptoms (e.g., samples from juvenile justice programs). Accordingly, caution should be exercised in extrapolating the findings. Fourth, based on the fit indices and omega reliability measures, we presented here a sound interpretation of the bifactor model informed by the literature regarding shared variance associated with multiple disorders, particularly in childhood and adolescents. However, the utility of the bifactor structure has yet to be examined, and future studies should establish the predictive validity of the general p factor based on the association between early-adolescence p factor scores and subsequent late-adolescence risk behaviors (e.g., substance use, suicide attempts). Finally, similar to most studies on the structure of adolescent psychopathology, the indicators used in this study are school-based self-report data. Although all of the models controlled for the clustering effects of school environment, future studies should investigate whether the assessment method affects the structure of adolescent psychopathology using multimethod and multiple informant data.
Notwithstanding the above limitations, the strengths of this study include the use of a large sample of adolescents and comparison of a range of dimensional models. The current study provides the first examination of the underlying structure of psychopathology among Canadian adolescents and adds to recent literature examining the bifactor model of psychopathology, including a thought disorder dimension. In closing, the study of the shared commonalities of psychopathological symptoms has important implications in terms of evaluation, prevention, and intervention through the development of transdiagnostic approaches to address multiple problems in a single framework.41 Capitalizing on these findings, promising future directions involve the validation studies of the bifactor structure in relation to subsequent clinical, cognitive, and biological outcomes, as well as study of longitudinal invariance of adolescent psychopathology structure.
Acknowledgements
We would like to thank the participants of the Coventure project for making this study happen.
Footnotes
Declaration of Conflicting Interests: The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: MHA, MS, NC, and PC report no biomedical financial interests or potential conflicts of interest.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for this study was provided by the Canadian Institutes of Health Research (grant FRN114887). MHA was supported by a postdoctoral fellowship from the Canadian Institutes of Health Research. MS and NC were supported by Australian National Health and Medical Research Council Early Career Fellowship, and PC was supported by a senior investigator award from the Fonds de la Recherche du Québec en Santé.
References
- 1. Kessler RC, Chiu WT, Demler O, et al. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):617–627. doi:10.1001/archpsyc.62.6.617. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Murray AL, Eisner M, Ribeaud D. The development of the general factor of psychopathology “p Factor” through childhood and adolescence. J Abnorm Child Psychol. 2016;44(8):1573–1586. doi:10.1007/s10802-016-0132-1. [DOI] [PubMed] [Google Scholar]
- 3. Coghill D, Sonuga-Barke EJS. Annual research review: categories versus dimensions in the classification and conceptualisation of child and adolescent mental disorders—implications of recent empirical study. J Child Psychol Psychiatry. 2012;53(5):469–489. doi:10.1111/j.1469-7610.2011.02511.x. [DOI] [PubMed] [Google Scholar]
- 4. Krueger RF, Caspi A, Moffitt TE, et al. The structure and stability of common mental disorders (DSM-III-R): a longitudinal-epidemiological study. J Abnorm Psychol. 1998;107(2):216–227. [DOI] [PubMed] [Google Scholar]
- 5. Krueger RF, Eaton NR. Transdiagnostic factors of mental disorders. World Psychiatry. 2015;14(1):27–29. doi:10.1002/wps.20175. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Carragher N, Teesson M, Sunderland M, et al. The structure of adolescent psychopathology: a symptom-level analysis. Psychol Med. 2016;46(5):981–994. doi:10.1017/S0033291715002470. [DOI] [PubMed] [Google Scholar]
- 7. Caspi A, Houts RM, Belsky DW, et al. The p factor: one general psychopathology factor in the structure of psychiatric disorders? Clin Psychol Sci. 2014;2(2):119–137. doi:10.1177/2167702613497473. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Keyes KM, Eaton NR, Krueger RF, et al. Thought disorder in the meta-structure of psychopathology. Psychol Med. 2013;43(8):1673–1683. doi:10.1017/S0033291712002292. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Castellanos-Ryan N, Brière FN, O’Leary-Barrett M, et al. The structure of psychopathology in adolescence and its common personality and cognitive correlates. J Abnorm Psychol. 2016;125(8):1039–1052. doi:10.1037/abn0000193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Lahey BB, Applegate B, Hakes JK, et al. Is there a general factor of prevalent psychopathology during adulthood? J Abnorm Psychol. 2012;121(4):971–977. doi:10.1037/a0028355. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Lahey BB, Rathouz PJ, Keenan K, et al. Criterion validity of the general factor of psychopathology in a prospective study of girls. J Child Psychol Psychiatry. 2015;56(4):415–422. doi:10.1111/jcpp.12300. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Tackett JL, Lahey BB, van Hulle C, et al. Common genetic influences on negative emotionality and a general psychopathology factor in childhood and adolescence. J Abnorm Psychol. 2013;122(4):1142–1153. doi:10.1037/a0034151. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Blanco C, Wall MM, He JP, et al. The space of common psychiatric disorders in adolescents: comorbidity structure and individual latent liabilities. J Am Acad Child Adolesc Psychiatry. 2015;54(1):45–52. doi:10.1016/j.jaac.2014.10.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Greene AL, Eaton NR. The temporal stability of the bifactor model of comorbidity: an examination of moderated continuity pathways. Compr Psychiatry. 2017;72:74–82. doi:10.1016/j.comppsych.2016.09.010. [DOI] [PubMed] [Google Scholar]
- 15. Carragher N, Krueger RF, Eaton NR, et al. Disorders without borders: current and future directions in the meta-structure of mental disorders. Soc Psychiatry Psychiatr Epidemiol. 2015;50(3):339–350. doi:10.1007/s00127-014-1004-z. [DOI] [PubMed] [Google Scholar]
- 16. Borsboom D, Cramer AOJ. Network analysis: an integrative approach to the structure of psychopathology. Annu Rev Clin Psychol. 2013;9:91–121. doi:10.1146/annurev-clinpsy-050212-185608. [DOI] [PubMed] [Google Scholar]
- 17. Afzali MH, Sunderland M, Teesson M, et al. A network approach to the comorbidity between posttraumatic stress disorder and major depressive disorder: the role of overlapping symptoms. J Affect Disord. 2017;208:490–496. doi:10.1016/j.jad.2016.10.037. [DOI] [PubMed] [Google Scholar]
- 18. Markon KE. Modeling psychopathology structure: a symptom-level analysis of Axis I and II disorders. Psychol Med. 2010;40(2):273 doi:10.1017/S0033291709990183. [DOI] [PubMed] [Google Scholar]
- 19. Beesdo K, Knappe S, Pine DS. Anxiety and anxiety disorders in children and adolescents: developmental issues and implications for DSM-V. Psychiatr Clin North Am. 2009;32(3):483–524. doi:10.1016/j.psc.2009.06.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Laceulle OM, Vollebergh WAM, Ormel J. The structure of psychopathology in adolescence: replication of a general psychopathology factor in the TRAILS study. Clin Psychol Sci. 2015;3(6):1–11. doi:10.1177/2167702614560750. [Google Scholar]
- 21. Patalay P, Fonagy P, Deighton J, et al. A general psychopathology factor in early adolescence. Br J Psychiatry. 2015;207(1):15–22. doi:10.1192/bjp.bp.114.149591. [DOI] [PubMed] [Google Scholar]
- 22. Lemos-Giráldez S, Paino M, Sierra-Baigrie S, et al. Psychotic-like experiences in the adolescent general population. Actas Esp Psiquiatr. 2011;39(3):155–162. [PubMed] [Google Scholar]
- 23. Conrod P, O’Leary-Barrett M, Massé B, et al. A cluster-randomised controlled trial evaluating the effects of delaying onset of adolescent substance abuse on cognitive development and addiction: the Co-Venture trial. Addiction. 2017;112(10):1871–1881. [DOI] [PubMed] [Google Scholar]
- 24. Goodman A. Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatry. 2001;40(11):1337–1345. doi:10.1097/00004583-200111000-00015. [DOI] [PubMed] [Google Scholar]
- 25. Goodman A, Lamping DL, Ploubidis GB. When to use broader internalising and externalising subscales instead of the hypothesised five subscales on the Strengths and Difficulties Questionnaire (SDQ): data from British Parents, Teachers and Children. J Abnorm Child Psychol. 2010;38(8):1179–1191. doi:10.1007/s10802-010-9434-x. [DOI] [PubMed] [Google Scholar]
- 26. Giannakopoulos G, Tzavara C, Dimitrakaki C, et al. The factor structure of the Strengths and Difficulties Questionnaire (SDQ) in Greek adolescents. Ann Gen Psychiatry. 2009;8(1):20 doi:10.1186/1744-859X-8-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Rothenberger A, Becker A, Erhart M, et al. Psychometric properties of the parent strengths and difficulties questionnaire in the general population of German children and adolescents: results of the BELLA study. Eur Child Adolesc Psychiatry. 2008;17(suppl 1):99–105. doi:10.1007/s00787-008-1011-2. [DOI] [PubMed] [Google Scholar]
- 28. Derogatis LR, Melisaratos N. The brief symptom inventory: an introductory report. Psychol Med. 1983;13(3):595 doi:10.1017/S0033291700048017. [PubMed] [Google Scholar]
- 29. Laurens KR, Hobbs MJ, Sunderland M, et al. Psychotic-like experiences in a community sample of 8000 children aged 9 to 11 years: an item response theory analysis. Psychol Med. 2012;42(7):1495–1506. doi:10.1017/S0033291711002108. [DOI] [PubMed] [Google Scholar]
- 30. Brown TA. Confirmatory factor analysis for applied research London (UK): Guilford; 2006. [Google Scholar]
- 31. Bonifay W, Lane SP, Reise SP. Three concerns with applying a bifactor model as a structure of psychopathology. Clin Psychol Sci. Forthcoming.
- 32. Rodriguez A, Reise SP, Haviland MG, et al. Statistical indices evaluating bifactor models: calculating and interpreting statistical indices. Psychol Methods. 2015;21(2):137–150. doi:10.1037/met0000045. [DOI] [PubMed] [Google Scholar]
- 33. Rodriguez A, Reise SP, Haviland MG. Applying bifactor statistical indices in the evaluation of psychological measures. J Pers Assess. 2016;98(3):223–237. doi:10.1080/00223891.2015.1089249. [DOI] [PubMed] [Google Scholar]
- 34. St Clair MC, Neufeld S, Jones PB, et al. Characterising the latent structure and organisation of self-reported thoughts, feelings and behaviours in adolescents and young adults. PLoS One. 2017;12(4):e01753 81. doi:10.1371/journal.pone.0175381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Carver CS, Johnson SL, Timpano KR. Toward a functional view of the p factor in psychopathology. Clin Psychol Sci. Forthcoming doi:10.1177/2167702617710037. [DOI] [PMC free article] [PubMed]
- 36. Aldao A, Gee DG, De Los Reyes A, et al. Emotion regulation as a transdiagnostic factor in the development of internalizing and externalizing psychopathology: current and future directions. Dev Psychopathol. 2016;28(4 pt 1):927–946. doi:10.1017/S0954579416000638. [DOI] [PubMed] [Google Scholar]
- 37. Gibbons RD, Weiss DJ, Kupfer DJ, et al. Using computerized adaptive testing to reduce the burden of mental health assessment. Psychiatr Serv. 2008;59(4):361–368. doi:10.1176/ps.2008.59.4.361. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Gibbons RD, Weiss DJ, Frank E, et al. Computerized adaptive diagnosis and testing of mental health disorders. Ann Rev Clin Psychol. Forthcoming doi:10.1146/annurev-clinpsy-021815-093634. [DOI] [PubMed]
- 39. Newby JM, McKinnon A, Kuyken W, et al. Systematic review and meta-analysis of transdiagnostic psychological treatments for anxiety and depressive disorders in adulthood. Clin Psychol Rev. 2015;40:91–110. doi:10.1016/j.cpr.2015.06.002 [DOI] [PubMed] [Google Scholar]
- 40. Snyder HR, Hankin BL, Sandman CA, et al. Distinct patterns of reduced prefrontal and limbic gray matter volume in childhood general and internalizing psychopathology. Clin Psychol Sci. Forthcoming doi:10.1177/2167702617714563. [DOI] [PMC free article] [PubMed]
- 41. Carragher N, Krueger RF, Eaton NR, et al. Disorders without borders: current and future directions in the meta-structure of mental disorders. Soc Psychiatry Psychiatr Epidemiol. 2015;50(3):339–350. doi:10.1007/s00127-014-1004-z. [DOI] [PubMed] [Google Scholar]