The cross-national structure of mental disorders: results from the World Mental Health Surveys

Abstract

Background

The patterns of comorbidity among mental disorders have led researchers to model the underlying structure of psychopathology. While studies have suggested a structure including internalizing and externalizing disorders, less is known with regard to the cross-national stability of this model. Moreover, little data is available on the placement of eating disorders, bipolar disorder and psychotic experiences in this structure.

Methods

We evaluated the structure of mental disorders with data from the World Health Organization Composite International Diagnostic Interview, including 15 lifetime mental disorders and six psychotic experiences. Respondents (n=5,478–15,499) were included from 10 high, middle and lower-middle income countries across the world aged 18 years or older. Confirmatory Factor Analyses (CFA) were used to evaluate and compare the fit of different factor structures to the lifetime disorder data. measurement invariance was evaluated with multigroup CFA (MG-CFA).

Results

A second-order model with internalizing and externalizing factors and fear and distress subfactors best described the structure of common mental disorders. MG-CFA showed that this model was stable across countries. Of the uncommon disorders, bipolar disorder and eating disorder were best grouped with the internalizing factor, and psychotic experiences with a separate factor.

Conclusions

These results indicate that cross-national patterns of lifetime common mental-disorder comorbidity can be explained with a second-order underlying structure that is stable across countries and can be extended to also cover less common mental disorders.

Introduction

Co-morbidity among mental disorders is common (e.g., Kessler et al., 1994; Bijl et al., 1998; Hasin & Kilcoyne, 2012; Teesson et al., 2009) and has been hypothesized to reflect the latent general structure of psychopathology (Sher & Trull, 1996; Mineka et al., 1998; Widiger & Clark, 2000; Carragher et al., 2015). Much scientific work has focused on gaining a better understanding of this ‘meta structure’ of mental disorders (e.g., Mineka et al., 1998; Krueger & Markon, 2006; Carragher et al., 2015; Eaton et al., 2015; Kotov et al., 2017) and has shown that the structure of common mental disorders can be explained by two broad underlying domains: internalizing and externalizing (Krueger et al., 1998; Krueger 1999; Krueger & Markon, 2006). Depressive disorders and anxiety disorders load on the internalizing factor and conduct disorder (CD), substance-related disorders and antisocial problems load on the externalizing factor.

The existence of separate, but correlated, internalizing and externalizing domains has been confirmed repeatedly in empirical studies. However, the structure is likely to be more complex and multiple underlying sub-domains are likely to exist. For instance, the internalizing domain has been consistently shown to have at least two lower-order subfactors that explain the more specific clustering of, respectively, ‘fear’ (panic disorder, agoraphobia, specific phobia, social phobia, and obsessive compulsive disorder [OCD]) and ‘distress’ disorders (major depressive episode [MDE], dysthymia, generalized anxiety disorder [GAD] and post-traumatic stress disorder [PTSD] e.g., Krueger et al., 1998; Krueger 1999; Vollebergh et al., 2001; Watson, 2005; Slade & Watson, 2006; Krueger & Markon, 2006; Miller et al., 2011; Eaton et al., 2013a; Kotov et al., 2017). For externalizing disorders, subfactors have been found that explain additional clustering of disorders that are either characterized by ‘norm-violations’ (CD, substance-use disorders) or by ‘oppositional behavior’ (oppositional defiant disorder [ODD], attention deficit hyperactivity disorder [ADHD]; Farmer et al., 2009). It has also been suggested that there may be aggression-related and substance-related subfactors (Krueger et al., 2007). Overall, the findings on the subfactor structure of the externalizing domain have so far been less consistent than for the internalizing domain (e.g., Krueger et al., 2005; Markon & Krueger, 2005; Vrieze et al., 2012).

Despite ongoing discussions about the best lower-order structure, the higher-order, two-factor model has been found to be very robust and is considered to reflect the natural structure of common mental disorders (Krueger, 1999; Watson, 2005; Krueger & Markon, 2006; Kotov et al., 2011; Carragher et al., 2015; Kotov et al., 2017). Various studies have supported this idea. For instance, variations on the internalizing and externalizing domains have been found to be linked to variations on distinct genetic risk factors (Kendler et al., 2011; Lahey et al., 2017). In addition, the internalizing and externalizing domains have been shown to account for a large part of comorbidity patterns that are observed over a patients’ lifetime: the association of disorders with subsequent onset was found to be stronger within each domain than between domains (Kessler et al., 2011a; 2011b). However, disorders from each of the separate domains have also been shown to predict each other over time (Lahey et al., 2017). Structural studies that investigated psychopathology together with personality disorders have shown that the joint latent structure can be described with a finite number of dimensions, including internalizing and externalizing dimensions (Markon et al., 2010; Kotov et al., 2011; Røysamb et al., 2011; Kendler et al., 2011; South & Jarnecke, 2017). The two-factor structure has been shown to be structurally invariant across ethnic groups in the U.S. (Eaton et al., 2013b), gender (Hicks et al., 2007; Kramer et al., 2007; Eaton et al., 2012) and over time (e.g., Vollebergh et al., 2001; Eaton et al., 2011).

Despite the many insights that have been gained from the above described research, important research questions still remain unsatisfactorily answered. The current study aims to address the following of these questions: (1) how stable is the cross-national structure of common mental disorders and (2) where do uncommon mental disorders fit into the structure?

The first question deals with what has been referred to as ‘structural validity’. Establishing the structural validity of the model of mental disorders is very important as this stability across countries determines to what extent factors and (mean) factor-scores can be validly compared across countries when conducting mental health research. Therefore, more insight must be gained into cross-national measurement invariance of the latent structure of common mental disorders. Currently, there is little available research on this. Although the same general latent structure has been replicated in different countries (e.g., Australia: Slade & Watson, 2006; the Netherlands: Vollebergh et al., 2001; Norway: Røysamb et al., 2010), most of this work is from Western countries. Krueger et al. (2003) conducted the only cross-national study of the latent structure of mental disorders with data from the WHO Collaborative Study of Psychological Problems in General Health Care, which were collected in 15 countries around the world (Brazil, Chile, China, France, Germany, Greece, India, Italy, Japan, the Netherlands, New Zealand, Nigeria, Turkey, the U.K. and the U.S.). They showed that 2 factors (internalizing vs. alcohol problems) described the structure best in the cross-national and most of the country datasets. Formal evaluations of measurement invariance showed that the model had configural invariance and metric invariance (invariant factor loadings across countries) across countries, indicating that the factor-loading configuration was invariant but that factor means and residual variances could vary across countries. Unfortunately, this study included only one externalizing disorder (‘hazardous use of alcohol’), prohibiting a thorough investigation of the structural stability of the full externalizing domain. In the current cross-national study, the stability of a model based on a broader range of disorders could be investigated.

The optimal placement of many less common mental disorders into the 2-factor model has remained unclear, partly due to the limited availability of datasets that include these less common disorders. Fortunately, more recent, large epidemiological datasets have enabled researchers to extend the original two-factor model by including additional disorders. These studies have shown that psychotic experiences have been found to load on a separate factor (‘thought disorders’ or ‘psychosis’; Kotov et al., 2011; Keyes et al., 2013; Markon, 2010; Wright et al., 2013; Kotov et al., 2017). Bipolar disorders have been found to load on this ‘thought disorders’ factor as well (Kotov et al., 2011), but others have found a ‘bipolar/mania’ sub-factor of the internalizing domain (Forbush & Watson, 2013; Kotov et al., 2015) or have found bipolar disorder to cross-load on the fear and distress subfactors (Eaton et al., 2013a), making it unclear whether bipolar disorder belongs with the internalizing disorders, the thought disorders or both (Kotov et al., 2017). Eating disorders have been found to group with the internalizing disorders, more specifically, as a subfactor of the internalizing domain (Forbush et al., 2010; Forbush & Watson, 2013). Taken together, research on uncommon disorder placement has been comparatively scarce and the results rather inconsistent. This could reflect the multifactorial nature of the studied uncommon disorders, but could also be explained by methodological differences (e.g., the kind and number of included symptoms/disorders) which brings us to a more nuanced notion of common and uncommon disorders: some of the common disorders have rather low prevalence (e.g. panic disorder) while some of the uncommon disorders are in fact more prevalent (e.g. eating disorder). This has a lot to do with methodological differences in their grouping but also in- exclusion of subthreshold variants of a disorder (e.g. psychotic experiences versus schizophrenia). Although not perfect, we will use the term uncommon disorders in the present study to describe eating disorders, bipolar disorder and psychotic experiences. Both from an etiological and clinical perspective, it would be very useful to find out much more about the placement of these disorders into the latent structure, as this could provide very interesting clues about the nature and the extent of overlap between vulnerabilities for common and uncommon, but very severe, disorders (Kotov et al., 2017). To gain a more systematic and complete insight into this, more research is needed using samples, in which all common but also (several) uncommon disorders were systematically assessed. Such datasets were available for the current study.

Given the above described need for further research, the current study aimed to investigate (1) the latent structure of common mental disorders and its measurement invariance across countries, and (2) the placement of uncommon disorders into the latent structure. Data came from the World Mental Health (WMH) surveys, which were conducted in several countries around the world and included a comprehensive assessment of both common internalizing and externalizing disorders, (e.g., ADHD, CD,ODD and Intermittent Explosive Disorder [IED], substance abuse and dependence) and less common disorders. More specifically, a range of models, based on the previous literature was tested to evaluate the optimal placement of (1) eating disorders, (2) bipolar disorder and (3) psychotic experiences in the model.

Methods

World Mental Health Surveys

Data came from the World Mental Health Surveys (Kessler et al, 2004). The WHO Composite International Diagnostic Interview (CIDI) version 3.0 was administered in 29 WMH surveys across the world. Most surveys used stratified multistage clustered area probability household sampling with no substitution for non-participants. Data collection took place between 2001 and 2012, and response rates ranged from 45.9 to 97.2%, with an average of 69.5%. Classification of country income categories was based on the World Bank criteria at the time of each survey (The World Bank, 2009). All WMH surveys were conducted face-to-face by lay interviewers who had received standardized training. Standardized translation, back-translation, harmonization and quality control procedures were applied in all of the participating survey sites (Pennell et al, 2008). Informed consent was obtained according to protocols endorsed by local Institutional Review Boards.

The CIDI was divided into two parts, with Part I assessing core mental disorders and Part II additional disorders and correlates. Part I was completed by all subjects and Part II was administered to all subjects meeting criteria for any of the Part I disorders and a probability subsample of the other subjects. To adjust for differential sampling, all responses in the Part II subsample were weighted by the inverse of their probability of selection into the Part II sample.

Mental Disorders

Analyses were conducted with lifetime CIDI/DSM-IV diagnoses (present/absent), without hierarchy rules. The following common mental disorders were included: Major depressive episode (MDE), Dysthymia, Panic Disorder, Agoraphobia, Social phobia, Specific phobia, GAD, PTSD, ADHD, ODD, Alcohol abuse, Drug abuse, CD and IED. The prevalence rates of these disorders in the different countries have been described previously (see Supplement 1 for disorder-specific references). Alcohol and drug abuse were combined into a single substance-abuse variable (with or without substance dependence). Bipolar-i, bipolar-ii and subthreshold bipolar disorder were assessed with the CIDI and were combined into a single bipolar disorder variable (present/absent). In the eating disorders dataset, bulimia and binge-eating disorder were assessed in all part II subjects in Romania, Brazil, and Poland and in a random part of part-II subjects in the other countries. combined into a single eating-disorder variable (Anorexia Nervosa was not included due to very low prevalence). The assessed psychotic experiences included hallucinations (visual and auditory) and delusions (insertion/withdrawal of thoughts, mind control by some strange force, ideas of reference [e.g., telepathy], plot to harm you/people following you). In this dataset, psychotic experiences were assessed in all part II subjects in Brazil and Romania, and in a random part of the part II sample in the other countries. The six assessed psychotic experiences were used as individual input variables in the CFAs.

Samples

To enable inclusion of as much disorders as possible, all current analyses were run in subjects in the Part II sample within the age-range of 18–44 years because most externalizing disorders (e.g., ADHD) were only assessed in these subjects to limit recall-bias. Bipolar disorder, eating disorders and psychotic experiences were only assessed in subsamples. Therefore, models including these respective disorders were estimated in the subsets of countries, in which they were assessed.

The subsample that was used to investigate the structure of all common mental disorders included 10 samples from 9 countries (Brazil, Colombia, Colombia-Medellin, Mexico, Murcia, North-Ireland, Peru, Poland, Romania, U.S.; n=15,499). This dataset was also used to investigate the optimal placement of bipolar disorder. The additional placement of IED into the model was investigated in a subsample where IED was assessed (Brazil, Colombia, North-Ireland, Peru, Poland, Romania, U.S.; n=12,162). The placement of eating disorders was investigated in a subsample (n=10,585) that included all subjects that were assessed for eating disorders with the CIDI. In Romania, Brazil, and Poland, eating disorders were assessed in all part II subjects. In the other countries, eating disorders were assessed in a random part of part-II subjects. The placement of psychotic experiences was investigated in a subsample of 6 countries, in which they were assessed in addition to common mental disorders and bipolar disorder (Brazil, Colombia, Mexico, Peru, Romania, U.S.; n= 5,478). In each of these countries, psychotic experiences were assessed in a random subsample and only those who had completed the psychosis section were included.

Analyses

Hypothesized models from the literature were fit to the WMH surveys data using confirmatory factor analyses (CFA). Multigroup-CFA (MG-CFA) was used to investigate structural invariance across countries. All CFAs were conducted with Mplus 7 (Muthén & Muthén, 1998–2012), using a mean and variance adjusted Weighted Least Squares estimator (WLSMV). All CFAs were run using the Mplus procedures for complex designs and included WMH surveys design variables (clusters and strata) to account for effects of the complex design (Asparouhov, 2005) and weights to adjust for differential probabilities of selection and discrepancies with census data (see Heeringa et al., 2008 for more info on sampling weights and design variables in WMH). The Comparative Fit Index (CFI) and the Root Mean Square Error of Approximation (RMSEA) were used to evaluate model fit, with a CFI≥0.95 and an RMSEA≤0.06 indicating good fit to the data (Hu & Bentler, 1999).

In the MG-CFAs, the model with the best cross-national fit was estimated with increasing levels of invariance across countries: configural invariance (similar patterns of factor loadings and item-thresholds; loadings and thresholds may differ across countries), partial invariance (constrained factor loadings across countries, freely estimated item-thresholds in each country) and scalar invariance, (factor loadings and item-thresholds constrained across countries). The difference in CFI between models with different levels of invariance (ΔCFI) was used to compare models with different levels of measurement invariance. A difference of ≥0.01 has previously been suggested to indicate a meaningful difference when comparing two groups (Cheung & Rensvold, 2002).

Results

Bivariate associations

Table 2 shows the tetrachoric correlations between the lifetime disorders. The highest correlations among common disorders were observed between MDE and dysthymia (0.82), ODD and CD (0.71), MDE and GAD (0.65), GAD and PTSD (0.65), ADHD and ODD (0.65), and dysthymia and GAD (0.61). In addition, high correlations were observed between several of the psychotic experiences. The lowest correlations among common disorders were observed for Agoraphobia with CD (0.24) and agoraphobia with substance abuse (0.19), eating disorder with CD (0.21), and specific phobia with substance abuse. In addition, lower correlations were observed between many of the psychotic experiences and other disorders.

Table 2.

Tetrachoric correlations of lifetime common mental disorders, bipolar disorder, eating disorder and psychotic experiences

	MDE	Dysth.	GAD	PTSD	PD	Ago	SAD	Sp. Phob.	ADHD	ODD	CD	Sub- stance	IED	Eat	Bipol.	PE1: Vis. Hal.	PE2: Aud. Hal.	PE3: Thought	PE4: Mind cont.	PE5: Tele-pathy
MDE	-
Dysth	0.82 a
GAD	0.65 a	0.61 a
PTSD	0.56 a	0.46 a	0.51 a
PD	0.47 a	0.46 a	0.49 a	0.46 a
Ago.	0.46 a	0.44 a	0.44 a	0.38 a	0.55 a
SAD	0.50 a	0.49 a	0.50 a	0.44 a	0.47 a	0.65 a
Sp. Phob.	0.42 a	0.38 a	0.38 a	0.38 a	0.45 a	0.58 a	0.52 a
ADHD	0.39 a	0.42 a	0.36 a	0.42 a	0.37 a	0.34 a	0.45 a	0.31 a
ODD	0.36 a	0.36 a	0.34 a	0.41 a	0.34 a	0.28 a	0.43 a	0.31 a	0.65 a
CD	0.34 a	0.35 a	0.32 a	0.40 a	0.38 a	0.24 a	0.38 a	0.28 a	0.57 a	0.71 a
Substance	0.29 a	0.27 a	0.28 a	0.32 a	0.30 a	0.19 a	0.29 a	0.19 a	0.38 a	0.48 a	0.55 a
IED	0.40 b	0.34 b	0.40 b	0.31 b	0.38 b	0.33 b	0.39 b	0.31 b	0.43 b	0.44 b	0.48 b	0.35 b	-
Eat.	0.36 c	0.32 c	0.33 c	0.31 c	0.25 c	0.39 c	0.35 c	0.31 c	0.39 c	0.27 c	0.21 c	0.28 c	-e	-
Bipol.	0.52 a	0.43 a	0.45 a	0.38 a	0.44 a	0.48 a	0.44 a	0.39 a	0.46 a	0.46 a	0.43 a	0.42 a	-e	-e	-
PE 1: Vis. Hal.	0.27 d	0.28 d	0.34 d	0.29 d	0.29 d	0.23 d	0.19 d	0.26 d	0.19 d	0.26 d	0.18 d	0.12 d	-e	-e	0.24 d	-
PE 2: Aud. Hal.	0.39 d	0.38 d	0.22 d	0.37 d	0.35 d	0.20 d	0.22 d	0.26 d	0.21 d	0.23 d	0.14 d	0.17 d	-e	-e	0.28 d	0.72 d	-
PE 3: Thought	0.48 d	0.35 d	0.44 d	0.41 d	0.39 d	0.35 d	0.34 d	0.31 d	0.32 d	0.25 d	0.21 d	0.27 d	-e	-e	0.37 d	0.64 d	0.62 d	-
PE 4: Mind cont.	0.45 d	0.37 d	0.39 d	0.20 d	0.29 d	0.23 d	0.05 d	0.24 d	0.17 d	0.16 d	0.17 d	0.12 d	-e	-e	0.25 d	0.64 d	0.69 d	0.84 d	-
PE 5: Telepathy	0.41 d	0.41 d	0.33 d	0.21 d	0.24 d	0.16 d	0.15 d	0.14 d	0.33 d	0.30 d	0.25 d	0.25 d	-e	-e	0.29 d	0.76 d	0.65 d	0.73 d	0.82 d	-
PE 6: Delusion	0.44 d	0.36 d	0.30 d	0.51 d	0.43 d	0.46 d	0.42 d	0.37 d	0.35 d	0.17 d	0.33 d	0.34 d	-e	-e	0.39 d	0.46 d	0.55 d	0.77 d	0.65 d	0.58 d

All correlation coefficients estimated with Mplus 7.0.

MDE=Major Depressive Episode; Dysth=Dysthymia; PTSD=Posttraumatic stress disorder; GAD=Generalized Anxiety Disorder; PD=panic disorder; Ago=Agoraphobia; SAD=Social Anxiety disorder; Sp.Phob=Specific phobia; ADHD=Attention Deficit Hyperactivity Disorder; CD=Conduct Disorder; ODD=Oppositional Defiant Disorder; Substance=Substance abuse disorder (with/without dependence); PE=Psychotic experience; Vis. Hal. =having a visual hallucination; Aud.Hal=having an auditory hallucination; Thought=believing that thoughts are extracted from or inserted into your head ; Mind cont.=feeling that your mind was taken over by a strange force; Telepathy= believing that a strange force tries to communicate with you (e.g., through special signs from the radio or TV); Delusion: believing that there is a plot going on against you or that people follow you.

^aEstimated in sample including Colombia. Peru. Romania, Brazil, Northern Ireland, Poland, Murcia, Medellin, Mexico and the United States (n=15499).

^bEstimated in subsample including Colombia, Peru, Romania, Brazil, Northern Ireland, Poland and the United States (n=12,162).

^cEstimated in sample including the complete data or a random subsample that got the eating disorders assessment from Colombia, Peru, Romania, Brazil, Northern Ireland, Poland, Murcia, Medellin, Mexico and the United States (n=10,585).

^dEstimated in subsample including complete data or a random subsample that got the psychotic experiences assessment from Colombia, Peru, Romania, Brazil, Mexico and the United States (n=5,478).

^eIED was not investigated together with eating disorders bipolar disorder and PEs; Eating disorders were not investigated together with IED, bipolar disorders and PEs

Structural model of the common mental disorders

A simple 2-factor model and, a higher-order model with distress and fear sub-factors were estimated (Figure 1a; Table 3). Of these models, the higher-order model showed the best fit to the data (CFI=0.985; RMSEA=0.017), in line with a large body of previous work (Krueger et al., 1998; Krueger 1999; Vollebergh et al., 2001; Watson, 2005; Slade & Watson, 2006; Krueger & Markon, 2006; Miller et al., 2011; Eaton et al., 2013a; Kotov et al., 2017). In this model, all disorders showed considerable standardized loadings on their respective factors, the fear and distress factors showed considerable loadings on the second-order internalizing factor and the correlation between the second-order internalizing and externalizing factors was 0.61. Similar results were found when the analyses were run in a subset of countries that also all assessed IED.

Figure 1.

Factor loadings (unstandardized) of different structural models in the WMH surveys. One loading per factor was fixed to 1 for model identification. INT=internalizing; EXT=externalizing,; PSY=psychotic experiences; PD=panic disorder; Ago=Agoraphobia; SAD=Social Anxiety disorder; Sp.Phob=Specific phobia; MDE=Major Depressive Episode; Dysth=Dysthymia; PTSD=Posttraumatic stress disorder; GAD=Generalized Anxiety Disorder; ADHD=Attention Deficit Hyperactivity Disorder; CD=Conduct Disorder; ODD=Oppositional Defiant Disorder; Sub=Substance abuse disorder (with/without dependence); Eat=Eating disorder; Bipol=Bipolar disorder; Visual=Having visual hallucination; Audit=Having auditory hallucination; Thought=Having ideas of thought extraction or thought insertion; Mind=Feeling that your mind was taken over by an external force; Reference=Feeling that a strange force tries to communicate with you; Plot=Bbelief in an unjust plot that is going on to harm you.

Table 3.

Confirmatory factor analyses of the structure of common and uncommon mental disorders

Included disorders	Sample (#countries)	Factor model		Fit indices
			#FP	CFI	RMSEA (90% CI)
MDE, Dysthymia, Panic disorder, PTSD, Social phobia, Specific phobia, Agoraphobia, GAD, ADHD, ODD, Conduct disorder, Substance abusee	15,499 (10)	2-factor modelc	25	0.968	0.024 (0.022–0.026)
		Higher-order modeld with fear and distress subfactors	27	0.985	0.017 (0.015–0.019)
MDE, Dysthymia, Panic disorder, PTSD, Social phobia, Specific phobia, Agoraphobia, GAD, ADHD, ODD, Conduct disorder, Substance abusee +IED	12,162 (7)	2-factor modelc	27	0.971	0.022 (0.020–0.024)
		Higher-order modeld with fear and distress subfactors and IED loading on the externalizing factor	29	0.983	0.017 (0.015–0.019)
MDE, Dysthymia, Panic disorder, PTSD, Social phobia, Specific phobia, Agoraphobia, GAD, ADHD, ODD, Conduct disorder, Substance abusee +Bipolar disorders	15,499 (10)	Higher-order modeld with bipolar disorder loading directly on the second-order internalizing factor.	28	0.983	0.017 (0.015–0.018)
		Higher-orderd with bipolar disorder loading directly on the externalizing factor.	28	0.974	0.021 (0.019–0.022)
		Higher-orderd model with bipolar disorder loading on the fear subfactor	29	0.979	0.019 (0.017–0.020)
		Higher-orderd model with bipolar disorder loading on the distress subfactor	29	0.977	0.020 (0.018–0.021)
		Higher-orderd model with bipolar disorder loading on the fear and distress subfactor	30	0.980	0.018 (0.017–0.020)
MDE, Dysthymia, Panic disorder, PTSD, Social phobia, Specific phobia, Agoraphobia, GAD, ADHD, ODD, Conduct disorder, Substance abusee +Eating disorders	10,585 (10)a	Higher-order: eating disorders loading directly on internalizing	28	0.977	0.018 (0.016–0.021)
		Higher-orderd: eating disorders loading on fear	29	0.986	0.015 (0.013–0.017)
		Higher-orderd: eating disorders loading on distress	29	0.984	0.015 (0.013–0.018)
		Higher-orderd: eating disorders loading on fear & distress	30	0.986	0.015 (0.013–0.017)
MDE, Dysthymia, Panic disorder, PTSD, Social phobia, Specific phobia, Agoraphobia, GAD, ADHD, ODD, Conduct disorder, Substance abusee +Psychotic experiences	5,478 (6)b	3-factorc: PEs load on their own factor	39	0.972	0.015 (0.012–0.017)
		Higher-order with PEs loading on their own factor	41	0.984	0.011 (0.008–0.014)
MDE, Dysthymia, Panic disorder, PTSD, Social phobia, Specific phobia, Agoraphobia, GAD, ADHD, ODD, Conduct disorder, Substance abusee +Bipolar disorder and Psychotic experiences	5,478 (6)b	Higher-order with bipolar loading on the internalizing factor	43	0.983	0.011 (0.009–0.013)
		Higher-order with bipolar loading on the PE factor	43	0.959	0.017 (0.015–0.019)
		Higher-order with bipolar loading on the internalizing and PE factors	43	0.984	0.011 (0.009–0.013)

^aEating disorders were assessed in random subsamples in 7 of the 10 included countries.

^bPsychotic experiences were assessed in random subsamples in 4 of the 6 samples.

^cModel without cross-loadings and with freely estimated correlation(s) between factors

^dModel with freely estimated correlation between the higher-order factors

^eIncluding abuse and dependence

Structural validity

The invariance of the model of common mental disorders (minus IED) across countries was tested with MG-CFA using country as the group variable. When fitted to the individual samples, good fit of the model was observed in Brazil (CFI=0.969; RMSEA=0.017), Colombia (CFI=0.975; RMSEA=0.015), Colombia-Medellin (CFA=0.976; RMSEA=0.022), Mexico (CFI=0.982; RMSEA=0.016) and the U.S. (CFI=0.98; RMSEA=0.022). In the remaining countries (North Ireland, Murcia, Poland, Romania and Peru), model fitting was complicated by correlations close to 0 (or negative) in the disorder correlation matrix (mostly due to low frequencies of one or more disorders), which led to parameter estimates that were hard to interpret. Subsequent measurement-invariance analyses were therefore conducted with the first 5 samples from 4 countries. The results (Table 4) showed that the model with configural invariance across the 5 samples fit the data well (CFI=0.978; RMSEA=0.019). Constraining the first-order and second-order factor loadings led to a small decrease in fit (CFI=0.976; RMSEA=0.018). Constraining all first and second order factor loadings and all item-thresholds to be the same across samples (scalar invariance) led to a larger decrease in fit (CFI=0.962; RMSEA=0.021; ΔCFI=0.016). The decrease in CFI from the configural model to the model with constrained factor loadings (ΔCFI=0.002) indicated that partial invariance did hold across the 5 samples. The decrease in CFI from the configural model to the scalar model (ΔCFI=0.016) indicated that the model did not have scalar invariance across the 5 samples.

Table 4.

analyses of measurement invariance across five samples of the higher-order structural model of common mental disorders.

Model	Model parameters	Fit indices
		CFI	RMSEA (90%CI)	ΔCFI
1. Configural invariance	-1st and 2nd order factor loadings, item-thresholds and factor correlations unconstrained;	0.978	0.019 (0.016–0.022)	-
	Fixed parameters: Factor means and 1st order factor intercepts=0 across countries for model identification. Factor variances=1 across countries. Item residual variances =1 across countries.
2. Partial invariance	-1st and 2nd order factor loadings constrained; -Item-thresholds and correlations unconstrained.	0.976	0.018 (0.015–0.021)	0.002
	Fixed parameters: Factor means and 1st order factor intercepts=0 across countries for model identification. Factor variances=1 across countries. Item residual variances =1 across countries.
3. Scalar invariance	1st and 2nd order factor loadings and item-thresholds constrained. -Factor means, factor correlations unconstrained.	0.962	0.021 (0.019–0.024)	0.016
	Fixed parameters: 1st order factor intercepts=0 across countries for model identification. Factor variances=1 across countries. Item residual variances =1 across countries.

CFI=comparative fit index; RMSEA=root mean squared error of approximation.

All models testing invariance across 10 samples.

All models estimated with the theta parameterization in Mplus 7 (Muthén & Muthén, 1998–2012).

Structural model of common disorders and bipolar disorders

The placement of bipolar disorder in the higher-order model was investigated next (Table 3). Different ways of grouping bipolar disorder with the internalizing factor all led to better fit than grouping it with the externalizing domain. Models with bipolar disorder loading separately on the second-order internalizing factor alongside the distress and fear sub-factors (in line with Forbush & Watson [2013] and Kotov et al. [2015]) showed slightly better fit (CFI=0.982; RMSEA=0.017) than a configuration with bipolar disorder loading on either the fear or distress subfactors, or bipolar disorder cross-loading on the fear and distress sub-factors (as found by Eaton et al. [2013]; CFI=0.980; RMSEA=0.018), but the difference in model-fit was very small, suggesting that bipolar disorder can be grouped with the internalizing domain.

Structural model of common disorders and eating disorders

The placement of eating disorders in the higher-order model was investigated next. In line with previous work (Forbush et al., 2010; Forbush & Watson, 2013), placing eating disorders as a subdomain under the internalizing domain led to better fit than grouping eating disorders with the externalizing domain. However, unlike previously found, a model with eating disorders loading directly on the internalizing factor (i.e., as a separate internalizing subdomain) alongside the distress and fear subfactors was not the best-fitting solution. A model with eating disorders loading on the distress and/or on the fear subfactor(s) showed better fit. However, fit was very similar across the latter models (all RMSEA=0.015) making it is hard to draw definite conclusions about the best subfactor-grouping of eating disorders (see Figure 1b for an illustration of the model with eating disorders loading on the distress subfactor)

Structural model of common disorders and psychotic experiences

A model with all psychotic experiences loading on a separate factor alongside the internalizing and externalizing factors fit the data well (Table 3; CFI=0.972; RMSEA=0.015) and a higher-order model with a psychotic experiences factor, and distress and fear subfactors for the internalizing factor showed even better fit (CFI=0.984; RMSEA=0.011).

Structural model of common disorders, bipolar disorder and PEs

A model with all common disorders, bipolar disorder and psychotic experiences was estimated next. IED and eating disorders were not included in these analyses because they were only assessed in smaller subsamples. Several higher-order models were compared, evaluating whether bipolar disorder could be better grouped with the internalizing disorders, with the psychotic experiences, or with both (see Table 3). The results showed that a model with bipolar disorders set to load directly on the internalizing factor (see Figure 1c) fit the data better than a model with bipolar disorder loading on the same factor as the psychotic experiences. Setting bipolar disorder to cross-load on both factors did not improve fit much (same RMSEA) compared to the first model. In this model, bipolar disorders showed a very small standardized loading on the psychotic experiences factor (−0.19), which differed strongly in terms of direction and magnitude from the other psychotic experiences loadings (0.76–0.91).

Discussion

We addressed several unresolved issues regarding the structure of mental disorders using the largest cross-national dataset to date encompassing a relatively wide selection of both internalizing and externalizing disorders, and both common and uncommon disorders. Overall, we found that a higher-order model with an internalizing and externalizing factor and fear and distress subfactors, described the structure of common mental disorders well. Investigations of measurement invariance of this structure across 5 selected countries showed that the factor loadings can be assumed to be stable. Investigations of the placement of uncommon disorders into the model showed that eating disorders and bipolar disorder could be grouped under the internalizing factor and that psychotic experiences were best modeled as a separate domain within the model. When bipolar disorder and psychotic experiences were included together in a single model, bipolar disorder was still best grouped with the internalizing domain (but not with a specific subfactor).

The results confirmed the higher-order structure of common mental disorders that has been observed in many previous studies (i.e. Krueger et al., 1998; Krueger 1999; Vollebergh et al., 2001; Watson, 2005; Slade & Watson, 2006; Krueger & Markon, 2006; Miller et al., 2008; Eaton et al., 2013b). Importantly, the current findings about measurement invariance showed that the structure was relatively stable across 5 different countries. These findings align with the only previous cross-national study that also found evidence for metric cross-national measurement invariance (Krueger, 2003). The current results extend on these previous findings by showing that a higher-order model that encompasses a full range of externalizing disorders, instead of only substance-related problems, fits the data well and shows metric invariance across countries.

When investigated together with the common mental disorders, bipolar disorder was found to be best grouped with the internalizing factor, in line with previous research showing bipolar/mania to be an internalizing subfactor (e.g., Forbush & Watson, 2013; Kotov et al., 2015). Other authors found bipolar disorder to cross-load on the distress and fear subfactors when investigated in the context of other internalizing disorders (Eaton et al., 2013), but such model-configurations were not found to fit better in the current study. These results indicate that, in the context of the included common mental disorders, bipolar disorder shows a clear general association with the internalizing domain. Previously, bipolar disorders have also been shown to be partly related to a ‘thought problems’ dimension together with e.g., psychosis (Kotov et al., 2017). Therefore, this was also evaluated by investigating a model that included both bipolar disorder and psychotic experiences. These analyses showed that grouping bipolar disorder strictly with the psychotic experiences led to a decrease in fit and letting bipolar disorder cross-load on the internalizing and psychotic experiences factor did hardly improve fit. This could be explained by the fact that the internalizing and psychotic experiences factors were already strongly correlated, making the added cross-loading redundant. In addition, the fact that six psychotic experiences from a single interview module (see below for further discussion) were used and just a single bipolar-disorder indicator, could have made the setup of the analysis suboptimal to evaluate the existence of a broad thought-disorder factor. Finally, an alternative explanation may be that there is significant heterogeneity in our measure of bipolar disorder. As it consists of different subtypes and represents different episodes, it is possible that some subtypes or episodes may be more strongly related to either internalizing disorders or thought problems.

As previously observed (Forbush et al., 2010; Forbush & Watson, 2013) models grouping eating disorders with the internalizing disorders fit best to the data. Setting eating disorders to load on the fear, distress or both subfactors led to similar model-fit, but led to better fit than setting eating disorders to load directly on the internalizing domain. These findings suggest that, when investigated in the context of common mental disorders, eating disorders show a general association with the internalizing domain and its subdomains, although its exact placement remained somewhat unclear. These somewhat ambiguous findings could partly be related to the fact that only one pooled eating-disorder indicator was used in the current model. It could be that a separate eating disorder (sub)factor could be identified if more individual indicators were used, also including Anorexia Nervosa.

In line with previous findings that psychotic experiences reflect a separate mental-health domain (i.e. ‘thought problems’) alongside the internalizing and externalizing domains (e.g. Markon, 2010; Wright et al., 2013), the current results showed that a model with psychotic experiences loading on their own separate factor fit the data well. Interestingly, this factor showed considerable correlation with the internalizing factor, indicating that psychotic experiences are related to other mental disorders, in line with previous work showing the cross-diagnostic importance of psychotic experiences (Rössler et al., 2011; Fusar-Poli et al., 2012; Werberloff et al., 2012).

The current results align with previous work in the WMH surveys. Kessler et al. (2011a, 2011b) investigated the role of the internalizing and externalizing domains in the development of comorbidity patterns within persons over time and showed that these were explained to a strong degree by within-domain clustering. The current results confirmed that the 2-factor structure of mental disorders and expanded on these results by providing more insight into the finer-grained structure and placement of mental disorders into a higher-order factor model.

An important implication of the higher-order model is that variations in mental health/disorders could be explained by causal influences that occur at different levels with different degrees of disorder specificity. Some influences can be disorder-specific, some occur at the level of the subfactor and are shared with a limited set of other disorders (i.e. other fear or distress disorders), and some occur at the level of the higher-order domains (i.e. internalizing or externalizing) and are shared with all other disorders within the domain.

It is important to note that there are also influences that are very non-specific and influence all disorders, irrespective of their grouping with first- or second order factors (Lahey et al., 2017). Indeed, the internalizing and externalizing domains have been repeatedly shown to be robustly correlated, which could indicate the existence of an overarching general factor of psychopathology that accounts for the shared variance of all disorders (‘p-factor’; Caspi et al., 2017; Lahey et al., 2017). To investigate this, bifactor models can be used that incorporate both a general factor and domain-specific group factors (e.g., Lahey et al., 2017). Although it has been pointed out that bifactor models can become complex and very hard to interpret (Eid et al., in press; Koch et al., in press), their use in structural research of mental disorders is an interesting topic for further investigation.

There are some limitations that should be kept in mind when interpreting the findings of this study. First, the analyses were limited to those WMH surveys with data on all disorders and to subjects in the age-range of 18 to 45 years, which may limit the generalizability of the findings to other populations. Second, the models were fitted on cross-national datasets but prevalence rates could differ across surveys. Still, the investigations of measurement invariance indicated that at least partial invariance across countries could be assumed, although only part of the countries could be included in these analyses as some countries showed very low disorder frequencies. Investigations of measurement invariance across larger sets of countries will probably only be possible with smaller models that include a very limited range of disorders. This could be done in future research. Third, psychotic experiences, rather than psychotic disorders and schizophrenia were assessed and included in the analyses. On the one hand, this could seem somewhat inconsistent, as all other modelled mental problems were included as formal disorders. On the other hand, psychotic experiences are more common than full-blown psychotic disorders, which provides more possibilities to model the full psychotic spectrum, irrespective of diagnosis. However, it is possible that some of the observed clustering of psychotic experiences in the analyses can be explained by the fact that they were assessed differently from the other diagnoses. Finally, the general indices of fit indicated good fit for all tested models. As such all model-comparisons were among models that (objectively) fit the data well and were often based on very small differences in fit.

The current results show that the structure of mental disorders is best represented by a higher-order factor model with some degree of cross-national stability. In addition, eating disorders were shown to group with the internalizing domain, bipolar disorder was shown to group most strongly with the internalizing domain and psychotic experiences were shown to group with their own domain that was correlated with the other domains.

Table 1.

WMH sample characteristics by World Bank income categories^a

					Sample size
Country by income category	Surveyb	Sample characteristicsc	Field dates	Age range	Part I	Part II	Part II aged 18–44	Response rated
Low and lower middle income countries
Colombia	NSMH	All urban areas of the country (approximately 73% of the total national population).	2003	18–65	4,426	2,381	1,731	87.7
Peru	EMSMP	Five urban areas of the country (approximately 38% of the total national population).	2004–5	18–65	3,930	1,801	1,287	90.2
Total					(8,356)	(4,182)	(3,018)	88.9
Upper-middle income countries
Brazil - São Paulo	São Paulo Megacity	São Paulo metropolitan area.	2005–8	18–93	5,037	2,942	1,824	81.3
Colombia – Medelline	MMHHS	Medellin metropolitan area	2011–12	19–65	3,261	1,673	970	97.2
Mexico	M-NCS	All urban areas of the country (approximately 75% of the total national population).	2001–2	18–65	5,782	2,362	1,736	76.6
Romania	RMHS	Nationally representative.	2005–6	18–96	2,357	2,357	940	70.9
Total					(16,437)	(9,334)	(5,470)	80.5
High-income countries
N. Ireland	NISHS	Nationally representative.	2005–8	18–97	4,340	1,986	907	68.4
Poland	EZOP	Nationally representative	2010–11	18–65	10,081	4,000	2,276	50.4
Spain - Murcia	PEGASUS- Murcia	Murcia region.	2010–12	18–96	2,621	1,459	631	67.4
United States	NCS-R	Nationally representative.	2001–3	18–99	9,282	5,692	3,197	70.9
Total					(26,324)	(13,137)	(7,011)	60.8
TOTAL N					(51,117)	(26,653)	(15,499)	69.9

^aThe World Bank (2012) Data. Accessed May 12, 2012 at: http://data.worldbank.org/country. Some of the WMH countries have moved into new income categories since the surveys were conducted. The income groupings above reflect the status of each country at the time of data collection. The current income category of each country is available at the preceding URL.

^bNSMH (The Colombian National Study of Mental Health); EMSMP (La Encuesta Mundial de Salud Mental en el Peru); MMHHS (Medellín Mental Health Household Study); M-NCS (The Mexico National Comorbidity Survey); RMHS (Romania Mental Health Survey); NISHS (Northern Ireland Study of Health and Stress); EZOP (Epidemiology of Mental Disorders and Access to Care Survey); PEGASUS-Murcia (Psychiatric Enquiry to General Population in Southeast Spain-Murcia);NCS-R (The US National Comorbidity Survey Replication).

^cMost WMH surveys are based on stratified multistage clustered area probability household samples in which samples of areas equivalent to counties or municipalities in the US were selected in the first stage followed by one or more subsequent stages of geographic sampling (e.g., towns within counties, blocks within towns, households within blocks) to arrive at a sample of households, in each of which a listing of household members was created and one or two people were selected from this listing to be interviewed. No substitution was allowed when the originally sampled household resident could not be interviewed. Several WMH surveys (Poland, Spain-Murcia) used country resident or universal health-care registries to select respondents without listing households. 4 of the 10 surveys are based on nationally representative samples.

^dThe response rate is calculated as the ratio of the number of households in which an interview was completed to the number of households originally sampled, excluding from the denominator households known not to be eligible either because of being vacant at the time of initial contact or because the residents were unable to speak the designated languages of the survey. The weighted average response rate is 69.9%.

^eColombia moved from the “lower and lower-middle income” to the “upper-middle income” category between 2003 (when the Colombian National Study of Mental Health was conducted) and 2010 (when the Medellin Mental Health Household Study was conducted), hence Colombia’s appearance in both income categories. For more information, please see footnote a.