Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2025 Apr 1.
Published in final edited form as: J Affect Disord. 2024 Jan 13;350:255–263. doi: 10.1016/j.jad.2024.01.058

Internalizing Psychopathology and its Links to Suicidal Ideation, Dysfunctional Attitudes, and Help-Seeking Readiness in a National Sample of College Students

Candice Basterfield 1,1, Ellen E Fitzsimmons-Craft 2,1, C Barr Taylor 3, Daniel Eisenberg 4, Denise E Wilfley 5, Michelle G Newman 6
PMCID: PMC11057016  NIHMSID: NIHMS1960395  PMID: 38224742

Abstract

Background:

Recent evidence suggests that multiple emotional disorders may be better assessed using dimensional models of psychopathology. The current study utilized a cross-sectional population survey of college students (N = 8,613 participants) to examine the extent to which broad psychopathology factors account for specific associations between emotional problems and clinical and behavioral validators: suicidality, dysfunctional attitudes, and treatment seeking.

Methods:

Confirmatory factor models were estimated to identify the best structure of psychopathology. Models were then estimated to examine the broad and specific associations between psychopathology indicators and the clinical and behavioral validators.

Results:

The hierarchical model of psychopathology with internalizing problems at the top, fear, and distress at the second level, and five specific symptom dimensions at the third level evidenced the best fit. The associations between symptom indicators of psychopathology and clinical and behavioral validators were relatively small and inconsistent. Instead, much of the association between clinical and behavioral validators and emotional problems operated at a higher-order level.

Limitations:

The cross-sectional nature of the survey precludes the ability to draw conclusions regarding causality.

Conclusions:

Researchers should focus on investigating the shared or common components across emotional disorders, particularly concerning individuals presenting with higher rates of suicidal ideation dysfunctional attitudes, and help-seeking behavior. Using higher-order dimensions of psychopathology could simplify the complex presentation of multiple co-occurring disorders and suggest valid constructs for future investigations.

Keywords: Hierarchal model of psychopathology, Suicidal ideation, Dysfunctional attitudes, Treatment seeking, Comorbidity

Introduction

Student mental health has become an increasingly prominent issue on college campuses (Gallagher, 2015; Jao et al., 2019). Studies indicate that the prevalence of mental disorders on college campuses is widespread and has increased in the past few years, with one in three first-year college students meeting criteria for a clinically significant mental health problem such as depression, anxiety disorders, or suicidality (Auerbach et al., 2018; Eisenberg et al., 2013). Thus, identifying risk factors and diagnostic markers that place college students at increased risk for developing emotional problems can facilitate the development of prevention and intervention and have lifelong benefits.

Epidemiological data on population samples have consistently identified a high rate of comorbidity (or co-occurring disorders) between mood, anxiety, psychotic, and substance use disorders with the presence of any disorder associated with increased odds of experiencing another disorder (Slade et al., 2015). In the National Comorbidity Survey, 59.2% of lifetime and 57.5% of 12-month cases of DSM-IV major depressive disorder (MDD) were comorbid with DSM-IV anxiety disorders (Kessler et al., 2005). Not surprisingly, higher rates of comorbidity have been linked to increased levels of distress, higher disorder severity, service use, and poor treatment response (Teesson, 2009). In adolescence, comorbidity is the rule rather than the exception, and many disorders initially present as shared symptoms or signs later develop into more defined disorders (McElroy, 2018). The within-diagnosis heterogeneity and high rates of comorbidity between disorders is particularly problematic, as different individuals with the same disorder may have different etiologies and presentations, and the high rates of comorbidity suggest that current diagnostic categories may be too narrow in scope (Kotov et al., 2021). This has prompted some researchers to hypothesize that there may be shared or transdiagnostic factors at play that can better explain the co-occurrence of what has been traditionally thought of as multiple disorders as well as understanding their possible links with other clinically significant outcomes, such as treatment response or suicidal ideation (Kotov et al., 2021). For example, Krueger (1999) fitted a series of exploratory and confirmatory factor models using large-scale epidemiological data to explain the relationship between multiple disorders using a series of shared latent dimensions. Results suggested that DSM-IV mood and anxiety disorders loaded onto a single higher-order latent dimension referred to as the internalizing liability and has demonstrated excellent fit using data from different countries and research settings (Krueger & Markon, 2006; Wright et al., 2013).

The Hierarchal Taxonomy of Psychopathology (HiTOP) consortium suggested that a more efficient way to capture and organize psychopathology is through a system of continuous latent factors structured within a larger hierarchy based on empirical data. This hierarchy ranges from specific symptom clusters at the lowest level to a broad psychopathology factor at the highest level. Research has identified broad superspectra (e.g., internalizing) encompassing narrower subdimensions, including the distress and fear subfactors. The distress subfactor includes disorders that involve pervasive emotionality, such as MDD, generalized anxiety disorder (GAD), and posttraumatic stress disorder (PTSD). The fear subfactor is defined by disorders involving context-delimited forms of distress, including behavioral avoidance, such as panic disorder, social anxiety disorder, and specific phobias (Kotov et al., 2021; Watson, 2005).

Nevertheless, research suggests that some psychopathology indicators work differently from sample to sample as to whether these load on the fear or distress subfactor. For example, individuals diagnosed with PTSD are more likely also to meet the criteria for MDD, GAD, and social phobia. Thus, models characterize PTSD as reflecting internalizing distress (Kotov et al., 2017; Kotov et al., 2021; Watson, 2005) and possibly fear (Forbes et al., 2012; Forbes et al., 2010; Lockwood & Forbes, 2014; Watson, 2005). Relatedly, GAD is generally made to load on distress; however, some have argued that worry should be grouped with both distress and fear disorders (Mennin et al., 2008). Social anxiety disorder has typically served as an indicator of fear-based disorders. However, Prenoveau et al. (2010) reported that it loaded with fear and distress, consistent with findings that social anxiety shares features with GAD and depression (Naragon-Gainey & Watson, 2011).

Studies have not only identified the latent structure of psychopathology but have also demonstrated the varying levels of validity of latent models in capturing the correlation between multiple individual mental disorders (i.e., comorbidity) and other important indicators of interest that have been hypothesized to be linked with multiple disorders (Conway et al., 2019). When broad latent factors capture the overall observed variance among individual indicators of psychopathology and other critical clinical and behavioral validators, it can be concluded that the correlation is primarily influenced by shared characteristics across multiple disorders and/or symptoms rather than information unique to individual disorders and/or symptoms. The shared and specific associations between psychopathology and numerous additional clinical validators have been investigated in adult samples (Eaton et al., 2013; Starr et al., 2014; Sunderland & Slade, 2015).

In an earlier study, Eaton et al. (2013) investigated the ability of an internalizing factor to predict future suicide attempts in the general population. Internalizing liability was strongly and significantly associated with suicide attempts, and it was a better predictor of this relationship than any diagnosis specific to a particular disorder. This research suggested that a more concise and informative approach to assessing multiple comorbid disorders could be useful and provide additional support for the importance of the internalizing dimension in a wide variety of clinical issues. Sunderland and Slade (2015) investigated the relationship between internalizing psychopathology and suicidality, treatment seeking, and disability in the Australian population using latent factor analysis. The researchers found that the latent factor of internalizing psychopathology was a better predictor of suicidality, treatment seeking, and disability than individual diagnoses of depression and anxiety. The findings showed that broad latent factors, rather than individual diagnoses, could effectively capture multiple associations between clinical and behavioral validators and specific disorders encompassing latent factors. These results offer proof of validity using an empirically based understanding of psychopathology. Additionally, these studies strengthen the usefulness of dimensional models as a structure for organizing research in psychopathology (Conway et al., 2019).

To our current knowledge, much of this structural work has focused on adolescent and adult samples. Emerging adulthood involves significant changes and challenges in various aspects of life, such as living arrangements, relationships, education, and employment. These transitions can lead to stress and psychological difficulties for the individual going through this phase of life (Matud et al., 2020). In college students, several key clinical validators including suicidal ideation and dysfunctional attitudes that have been previously associated with emotional disorders (Baek et al., 2015; Yapan et al., 2020). Research suggests that about one-third of college students reported lifetime suicidal ideation, with a median age of 14 years and persistence in the range of 40% to 50% (Mortier et al., 2018). Given that many students experience suicidal ideation and do not seek treatment, it is critical to identify contributing factors and engage these students in treatment (Drum et al., 2009). The presence of dysfunctional attitudes likewise has the potential to compound the severity of a disorder. Research suggests maladaptive beliefs contribute to the maintenance and recurrence of psychiatric and behavioral problems (Hankin & Abramson, 2001). Therefore, it is essential to understand whether a risk factor is specific to a lower-order dimension or associated with emotional problems more broadly. For example, if broad latent factors effectively encompass the overall observed variability among indicators of psychopathology and clinical and behavioral validators, we can deduce that the connection is largely influenced by shared features across symptoms and disorders, as opposed to information specific to individual disorders or symptoms. Nonetheless, pinpointing specific associations would reveal distinctive aspects of the relationship between a symptom or disorder and a clinical or behavioral validator, carrying implications for the assessment of emotional disorders (Forbes et al., 2019). Research has shown that conditions like MDD encompass a blend of both broad (distress) and specific (anhedonia) individual differences. While any of these may be proximally linked to suicidal ideation and dysfunctional attitudes, the size of these influences is impossible to know without examining them directly. Finally, different internalizing disorders are related to varying levels of treatment-seeking behavior; for instance, individuals with social phobia or agoraphobia may be less likely to seek support due to the nature of the disorder (Griffiths, 2013). The present study (a) employed a measurement framework using confirmatory factor analysis to assess the validity and utility of a hierarchal internalizing liability model within a national sample of college students, and (b) investigated the criterion validity of higher-order dimensions, specifically examining whether a hierarchal model of psychopathology effectively encapsulated the overall association among specific indicators of psychopathology (MDD) and clinical and behavioral validators. To disentangle these associations, path analysis was utilized looking at total and direct effects, providing a nuanced exploration of the relationships.

Method

Participants

First- or second-year undergraduate students at 4-year colleges and universities completed a mental health screening survey between October 2019 and May 2020. Students were recruited from schools in the United States. The screening survey was sent to all first- and second-year students at participating institutions and was conducted to determine eligibility for later enrollment in a randomized controlled trial (ClinicalTrials.gov—Harnessing Mobile Technology to Reduce Mental Health NCT04162847).

All data for the present study were collected before selecting for the randomized controlled trial or intervention delivery. A total of 11,007 students accessed the survey. Participants were excluded if they did not provide informed consent (n = 1568), were under 18 years old (n = 168), were not first- or second-year undergraduates (n = 656), exited the survey before reporting on any psychological disorders (n = 216), or failed an attention check that asked participants to choose a specific response (n = 13). The resulting sample included 8, 389 students who participated. Each student participated once. The sample comprised 2,467 males, 5,922 females, and 207 participants who reported trans, non-confirming, or self-identify. There were 6,047 White individuals, 567 Black, 1,027 Asian, 47 American Indians or Alaskan Natives, 24 Native Hawaiian or Pacific Islanders, and 504 multiracial.

Measures

Indicators of psychopathology

Generalized anxiety disorder (GAD) was assessed using the Generalized Anxiety Disorder Questionnaire-IV (GAD-Q-IV). The GAD-Q-IV is a 9-item self-report diagnostic measure of GAD that assesses all the DSM-5 criteria of GAD. The measure has demonstrated good sensitivity (89%) and specificity (82%; Newman et al., 2002). Participants screened positive for GAD if they endorsed all diagnostic criteria. Internal consistency in the present sample was 0.74.

Social phobia was assessed using the Social Phobia Diagnostic Questionnaire (SPDQ). The SPDQ is a 29-item self-report measure of symptoms of social anxiety disorder. The SPDQ has demonstrated good specificity (82%) and sensitivity (85%; Newman et al., 2003). Participants screened positive for a social phobia if they endorsed all diagnostic criteria. Internal consistency in the present sample was 0.75.

Panic disorder was assessed with the Panic Disorder Self-Report (PDSR). The PDSR is a 24-item self-report measure designed to diagnose panic disorder based on DSM-5 criteria. The PDSR has demonstrated good specificity (1.00) and sensitivity (0.89) (Newman et al., 2006). Participants screened positive for panic disorder if they endorsed all diagnostic criteria. Internal consistency in the present sample was 0.77.

Major depressive disorder (MDD) was assessed using the Patient Health Questionnaire-9 (PHQ-9; Kroenke & Spitzer, 2002). The PHQ-9 assesses symptoms of major depression based on diagnostic criteria of the DSM-IV. It has been shown to have good construct and criterion validity (Kroenke et al., 2010), and a sensitivity of 0.88 and specificity of 0.85 (Manea, 2012). Participants screened positive for probable MDD if they scored 10 or higher. Internal consistency in the present sample was 0.73.

Posttraumatic stress disorder (PTSD) was assessed using the Primary Care PTSD Screen (Prins et al., 2003). Participants screened positive for probable PTSD if they scored three or higher on the screen, demonstrating sensitivity of 0.78 and specificity of 0.89 (Prins et al., 2003). Internal consistency in the present sample was 0.77.

Clinical and behavioral validators

Suicidal ideation was assessed using item 9 of the PHQ-9, which asked how much respondents thought about hurting themselves or that they would be better off dead in the past two weeks. Responses of 1 (several days) or higher screened positive for suicidal ideation.

Dysfunctional Attitudes Scale-Short Form (DAS-SF) is a self-report questionnaire containing nine items designed to assess the dysfunctional attitudes of individuals. Subjects rated their agreement to each of the 9 statements on a Likert scale of agreement from 1 to 5. Scores could range from 9 to 45, with higher scores reflecting more dysfunctional attitudes. The DAS-SF demonstrated high levels (.91 to .93) of concurrent validity with the original 40-item Dysfunctional Attitudes scale, and has shown good reliability and validity in both student and patient samples (Beevers et al., 2007). Internal consistency in the present sample was 0.79.

Readiness to seek help was measured through the inclusion of a question that assessed the readiness of the participant to seek help: “On a scale from 0 to 10, where 0 is definitely not ready, and 10 is definitely ready, how ready are you at the present time to seek help for your mental and emotional health?” Questions were rated on a 10-point Likert scale from 0= ‘definitely not ready’ to 10=‘definitely ready’. This measure was part of the Healthy Minds study and was adapted by these researchers from the Healthcare for Communities Survey, a national survey that assessed substance use and mental health problems (Wells et al., 2012). Internal consistency in the present sample was 0.82.

Data Analysis

The first stage involved testing measurement models and constructing the broad internalizing latent dimensions using confirmatory factor analysis (CFA). Based on previous studies (Krueger et al., 2018; Krueger & Markon, 2006), two separate CFA models were tested and compared. Two variants of the two-factor model involved fear and distress: panic and social phobia loading on fear, and one with GAD, MDD, and PTSD loading on distress (Kotov et al., 2021; Watson, 2005). The second model was a two-factor model, where panic disorder and post-traumatic stress disorder (PTSD) were coupled together on one latent factor labeled fear (Forbes et al., 2012; Forbes et al., 2010; Lockwood & Forbes, 2014; Watson, 2005), and MDD, GAD, and social phobia on the other latent factor labeled distress (Prenoveau et al., 2010).

The CFA models were estimated using the maximum likelihood estimator with robust standard errors, and the observed indicators followed a continuous and multivariate normal distribution (Li, 2016). Model fit was determined using a range of absolute statistical fit indices, including the root mean square error of approximation (RMSEA), which assessed the approximate fit of the model, where values close to 0.06 or below indicated good fit (Hu & Bentler, 1999). The second index was the comparative fit index (CFI), which used a hypothetical baseline model with unrelated observed variables to compare the model. Values close to 0.95 or higher indicated a good fit (Hu & Bentler, 1999). Finally, the Tucker–Lewis index (TLI) measured the relative reduction in misfit per degree of freedom. Generally, a TLI greater than 0.95 is a commonly used cut-off criterion (Hu & Bentler, 1999).

The best-fitting model was used to examine the association between broad latent variables of psychopathology and clinical and behavioral validators of interest. Specifically, to answer questions about multiple levels of the dimensional hierarchy, total, direct, and indirect effects were estimated using clinical and behavioral validators on psychopathology outcomes one at a time in path analysis. This approach was useful to evaluate the extent to which each construct in the hierarchical model was associated with the outcome of interest and then to disentangle the portion of that relationship that was unique to the construct of interest (i.e., a direct effect) as opposed to being accounted for by higher-order factors that captured variance shared with other constructs in the model (i.e., an indirect effect).

Specifically, the total effect represented the zero-order association between a clinical and behavioral validator and a psychopathology dimension. It did not account for shared variance between emotional problems (e.g., MDD) and other variables in the model. The direct effect represented the portion of the association unique to the psychopathology dimension of interest (i.e., the proportion of the total effect not mediated by higher-order emotional dimensions). The indirect effects reflected the proportion of the association between clinical and behavioral validators and psychopathology mediated by higher-order constructs above and beyond lower-order dimensions. There were no indirect effects of clinical and behavioral validators on internalizing because internalizing was at the apex of the hierarchy (see Figure 2). For example, Figure. 1 illustrates the direct and indirect effects of panic disorder (predictor) on clinical and behavioral validators (outcomes). A p-value of 0.05 was used to determine significant direct, indirect, and total effects for the main analyses; as well as 95% confidence intervals. To determine whether the association between a clinical and behavioral validator and a psychopathology problem was specific to that dimension or if the association reflected a broader association between clinical and behavioral validators and a higher-order factor, the percentage of variance explained via indirect effects (i.e., the percentage of variance accounted for by higher-order factors) was evaluated using Cohen (1992) rough guide for interpreting standardized path coefficients, with coefficients of less than .20 small effect, .50 medium effect, and .80 or greater large effect. Missing data were dealt with multiple imputations by chained equations (MICE). Multiple imputation utilized the MICE package (van Buuren & Groothuis-Oudshoorn, 2011). Data were imputed with 10 iterations for each analysis and used predictive mean matching to replace the missing values with plausible estimates.

Figure 2.

Figure 2.

Open in a new tab

Direct and indirect effects of clinical validators on panic disorder. Path a represents the direct effect of clinical validator on panic disorder, whereas path b represents the indirect effect of clinical validator on panic disorder.

Figure 1.

Figure 1.

Open in a new tab

Measurement model for internalizing liability.

Results

Descriptive statistics

The sample comprised 8,613 participants (5,922 female, 2,467 male, and 207 participants reported trans, non-confirming, or self-identify) with ages ranging from 18 to 76 (M: 20.24; SD: 4.09). Prevalence rates were the highest for probable MDD (59.02%), GAD (47.88%), PTSD (27.65%), social phobia (21.43%), and panic disorder (12.57%). Pearson correlations for all psychological disorders and clinical and behavioral validators are provided in Table 1, with results indicating moderate to high correlations among psychopathology indicators.

Table 1.

Correlation Matrix for Indicators of Psychopathology and Clinical and Behavioral Validators

Social phobia Panic disorder Probable MDD GAD PTSD Suicidal ideation DAS-SF Readiness to seek help
Social phobia 1
Panic disorder 0.36** 1
Probable MDD 0.56** 0.41** 1
GAD 0.62** 0.46** 0.65** 1
PTSD 0.40** 0.37** 0.45** 0.48** 1
Suicidal ideation 0.36** 0.27** 0.52** 0.35** 0.29** 1
DASSF 0.36** 0.19** 0.28** 0.30** 0.18** 0.26** 1
Readiness to seek help 0.09** 0.18** 0.16** 0.21** 0.10** 0.07** 0.03 1
Open in a new tab

Note: MDD=major depressive disorder, GAD= generalized anxiety disorder, PTSD= posttraumatic stress disorder; DAS-SF= Dysfunctional Attitudes Scale-Short Form

*

p < 0.05

**

p < 0.01

Measurement model

The second hypothesized model fits the data (see Figure 1). Indices of model fit suggested that the internalizing, distress, and fear factors adequately represented the correlations among symptom scales: χ2(4) = 106.28, p = <.001; comparative fit index (CFI) = .99; root mean square error of approximation (RMSEA) = 0.028; standardized root mean square residual (SRMR) = .0082. The standardized factor loadings on the first-order factors ranged from 0.59 to 0.95, and the loadings of fear and distress on the internalizing factor were both 0.95. It was necessary to constrain the two (unstandardized) second-order loadings to be equivalent for model identification purposes. The path diagram for the best-fitting model and standardized factor loadings are provided in Figure 1.

Broad associations

To determine the extent to which the associations between clinical and behavioral validators and dimensions of psychopathology problems were unique to those dimensions, total, direct, and indirect effects were estimated (see Table 2 to 4 for total effects). Clinical and behavioral indicators have no indirect effect on internalizing because internalizing is at the top of the hierarchy in the model (see Figure 1). As described above, the total effect represents the bivariate association between clinical validators and a psychopathology dimension. This effect does not adjust for shared variance between the psychopathology variable (e.g., MDD symptoms) and other variables in the model.

Table 2.

Suicidal Ideation

Psychopathology domain Total effect (95% CI) Direct effect (95% CI) Percentage accounted by higher order factors
Panic disorder Estimate 0.271 0.001 100%
(0.257, 0.279) (−0.017, 0.008)
p-value p<0.001 p=0.409
PTSD Estimate 0.344 −0.007 0%
(0.340, 0.372) (−0.014,0.019)
p-value p<0.001 p= 0.413
Major depression Estimate 0.517 0.223 57%
(0.511, 0.527) (0.216, 0.235)
p-value p<0.001 p<0.001
GAD Estimate 0.400 −0.234 0%
(0.393, 0.416) (−0.252, −0.225)
p-value p<0.001 p<0.001
Social anxiety Estimate 0.380 −0.043 0%
disorder (0.370, 0.391) (−0.061, −0.037)
p-value p<0.001 p<0.001
Fear Estimate 0.457 −0.101 0%
(0.442, 0.470) (−0.129, −0.095)
p-value p<0.001 p<0.001
Distress Estimate 0.553 0.096 83%
(0.551, 0.573) (0.090, 0.122)
p-value p<0.001 p<0.001
Internalizing Estimate 0.535 0.535 --
(0.533, 0.552) (0.552, 0.533)
p-value p<0.001 p<0.001
Open in a new tab

Note: All effects are fully standardized estimates from R. Total effects represent the association between suicidal ideation and a psychopathology dimension and is the sum of direct and indirect effects. The direct effect represents the portion of the total effect that is unique to the psychopathology dimension of interest (i.e., the portion of the total effect that is not mediated by higher-order factors). When direct effects had opposite signs to the relevant total effects, the percentage of the total effect accounted for by higher order factors (final column in the table) was 0%.3

The first five rows of Table 2 present the total effects of suicidal ideation on symptom measures. Suicidal ideation had a significant positive total effect on panic disorder (β = 0.27, p < 0.001; 95% CI: 0.26, 0.28), PTSD (β = 0.34, p < 0.001), social phobia (β = 0.38, p < 0.001, 95% CI: 0.37, 0.39), depression (β = 0.52, p < 0.001, 95% CI: 0.51, 0.53), and GAD (β = 0.40, p < 0.001, 95% CI: 0.39, 0.42). The total effects for fear (β = 0.46, p < 0.001, 95% CI: 0.44, 0.47), distress (β = 0.55, p < 0.001, 95% CI: 0.55, 0.57), and internalizing (β = 0.535, p < 0.001, 95% CI: 0.53, 0.55) were positive (see Table 2).

Dysfunctional attitudes had significant total effects on symptom measures for panic disorder (β = 17, p < 0.001, 95% CI: 0.15, 0.20), PTSD (β = 0.26, p < 0.001, 95% CI: 0.23, 0.29), social phobia (β = 0.38, p < 0.001, 95% CI: 0.36, 0.40), depression (β = 0.31, p < 0.001, 95% CI: 0.26, 0.280.29, 0.34), and GAD (β = 0.34, p < 0.001, 95% CI: 0.32, 0.37), The total effects on higher-order factors were also positive: fear (β = 0.32, p < 0.001, 95% CI: 0.28, 0.35), distress (β = 0.47, p < 0.001, 95% CI: 0.44, 0.49), and internalizing (β = 0.43, p < 0.001, 95% CI: 0.42, 0.46; see Table 3).

Table 3.

Dysfunctional Attitudes

Psychopathology domain Total effect (95%CI) Direct effect (95% CI) Percentage accounted by higher order factors
Panic disorder Estimate 0.174 −0.114 100%
(0.148, 0.195) (−0.204, 0.147)
p-value p<0.001 p=0.373
PTSD Estimate 0.263 0.014 95%
(0.231, 0.293) (−0.016, 0.046)
p-value p<0.001 p=0.367
Major depression Estimate 0.314 −0.016 100%
(0.293, 0.338) (−0.043, 0.009)
p-value p<0.001 p=0.198
GAD Estimate 0.341 −0.134 0%
(0.317, 0.365) (−0.163, −0.104)
p-value p<0.001 p<0.001
Social anxiety Estimate 0.383 0.144 62%
disorder (0.361, 0.403) (0.121,0.168)
p-value p<0.001 p<0.001
Fear Estimate 0.321 −0.156 0%
(0.283, 0.351) (−0.199, −0.123)
p-value p<0.001 p<0.001
Distress Estimate 0.468 0.147 69%
(0.443, 0.494) (0.118, 0.186)
p-value p<0.001 p<0.001
Internalizing Estimate 0.434 0.434 --
(0.416, 0.463) (0.416, 0.463)
p-value p<0.001 p<0.001
Open in a new tab

Note: All effects are fully standardized estimates from R. Total effects represent the association between suicidal ideation and a psychopathology dimension and is the sum of direct and indirect effects. The direct effect represents the portion of the total effect that is unique to the psychopathology dimension of interest (i.e., the portion of the total effect that is not mediated by higher-order factors). When direct effects had opposite signs to the relevant total effects, the percentage of the total effect accounted for by higher order factors (final column in the table) was 0%.

Readiness to seek help had a significant total positive effect on panic disorder (β = 17, p < 0.001; 95% CI: 0.15, 0.19), PTSD (β = 0.05, p = 0.001; 95% CI: 0.01, 0.09), social phobia (β = 0.09, p < 0.001; 95% CI: 0.07, 0.11), depression (β = 0.16, p < 0.001; 95% CI: 0.13, 0.18), GAD (β = 0.20, p < 0.001; 95% CI: 0.18, 0.23), fear (β = 0.25, p < 0.001; 95% CI: 0.21, 0.28), distress (β = 0.22, p < 0.001; 95% CI: 0.20, 0.25), and internalizing (β = 0.25, p < 0.001; 95% CI: 0.22, 0.28; see table 4).

Table 4.

Readiness to Seek Help

Psychopathology domain Total effect (95% CI) Direct effect (95% CI) Percentage accounted by higher order factors
Panic disorder Estimate 0.171 0.100 59%
(0.146, 0.193) (0.078, 0.123)
p-value p<0.001 p<0.001
PTSD Estimate 0.053 −0.069 0%
(0.017, 0.087) (−0.100, −0.038)
p-value p<0.01 p<0.001
Major depression Estimate 0.155 0.003 98%
(0.132, 0.180) (−0.100, −0.038)
p-value p<0.001 p=0.773
GAD Estimate 0.202 0.051 75%
(0.178, 0.225) (0.027, 0.072)
p-value p<0.001 p<0.001
Social anxiety Estimate 0.090 −0.068 0%
disorder (0.065, 0.114) (−0.089, −0.044)
p-value p<0.001 p<0.001
Fear Estimate 0.250 0.026 90%
(0.212, 0.283) (−0.012, 0.057)
p-value p<0.001 p=0.124
Distress Estimate 0.224 −0.026 100%
(0.198, 0.252) (−0.057, 0.012)
p-value p<0.001 p=0.124
Internalizing Estimate 0.245 0.245 --
(0.220, 0.276) (0.220, 0.276)
p-value p<0.001 p<0.001
Open in a new tab

Note: All effects are fully standardized estimates from R. Total effects represent the association between suicidal ideation and a psychopathology dimension and is the sum of direct and indirect effects. The direct effect represents the portion of the total effect that is unique to the psychopathology dimension of interest (i.e., the portion of the total effect that is not mediated by higher-order factors). When direct effects had opposite signs to the relevant total effects, the percentage of the total effect accounted for by higher order factors (final column in the table) was 0%.

Specific associations

The direct effect represented the extent to which clinical validators and a psychopathology dimension were related above and beyond any higher-order constructs (e.g., after partialling out associations with distress, fear, and internalizing). The direct effects of suicidal ideation on symptom measures had a significant, positive effect on depression (β = 0.22, p < 0.001; 95% CI: 0.22, 0.24), a positive effect on panic disorder (β = 0.01, p = 0.40; 95% CI: −0.02, 0.01), and negative effects on PTSD (β = −0.01, p = 0.41; 95% CI: −0.01, 0.02), social phobia (β = −0.04, p < 0.001; 95% CI: −0.06, −0.04), and GAD (β = −0.23, p < 0.001; 95% CI: −0.25, −0.23), after partialling its effects on the higher-order psychopathology dimensions. Suicidal ideation had significant negative direct effects on fear (β = −0.10, p < 0.001; 95% CI: − 0.13, −0.01), and positive total effects on distress (β = 0.10, p < 0.001; 95% CI: 0.09, 0.12), and internalizing (β = 0.54, p < 0.001; 95% CI: 0.55, 0.53).

Dysfunctional attitudes had negative direct effects on panic disorder (β = −0.01, p = 0.37; 95% CI: −0.20, 0.15), depression (β = −0.02, p = 0.19; 95% CI: −0.04, 0.01), and a significant negative effect on GAD (β = −0.13, p < 0.001; 95% CI: −0.16, −0.10). Dysfunctional attitudes had a positive effect on PTSD (β = 0.01, p =0.36; 95% CI: −0.02, 0.05), and significant positive direct effects on social phobia (β = 0.14, p < 0.001; 95% CI: 0.12, 0.17). Dysfunctional attitudes had significant negative direct effects on fear (β = −0.16, p < 0.001); 95% CI: −0.20, −0.12, and significant positive direct effects on distress (β = 0.15, p < 0.001; 95% CI: 0.12, 0.19) and internalizing (β = 0.43, p < 0.001; 95% CI: 0.42, 0.46).

Readiness to seek help had a significant positive direct effect on panic disorder (β = 0.07, p < 0.001; 95% CI: 0.08, 0.12), depression (β = 0.94, p < 0.001; 95% CI: −0.10, −0.04), and GAD (β = 0.05, p < 0.001; 95% CI: 0.03, 0.07), and significant direct negative effects on PTSD (β = −0.07, p = 0.001; 95% CI: −0.10, −0.04), and social phobia (β = −0.06, p < 0.001; 95% CI: −0.09, −0.04). The direct effects for fear (β = 0.03, p = 0.12; 95% CI: −0.01, 0.06), distress (β = − 0.03, p =0.12; 95% CI: −0.06, 0.01), and internalizing (β = 0.25, p < 0.001; 95% CI: 0.22, 0.28) were positive.

Discussion

The current study utilized a large population-based sample of college students to (a) compare alternative and theoretically plausible models of psychopathology; and (b) investigate the shared and specific associations among multiple indicators of psychopathology (e.g., MDD, distress fear, etc.) and other clinical and behavioral validators: suicidal ideation, dysfunctional attitudes, and readiness to seek help.

A substantial prevalence of probable MDD was observed at 59.02%, followed by GAD at 47.88%, PTSD at 27.65%, social phobia at 21.43%, and panic disorder at 12.57%. These figures align with similar studies, offering a comparative perspective. For instance, López-Castro et al. (2021) reported anxiety symptoms in 52% and depressive symptoms as high as 69% among American college students in New York City during the COVID-19 pandemic.

Modeling was employed to enable the size and significance of the specific associations to be evaluated after accounting for the shared association (via a reduced set of latent factors). The study found evidence that the best-fitting model among this sample comprised of two first-order factors representing distress and fear. The correlations between these two latent factors were represented by a shared, second-order factor representing internalizing liability. These results suggest that broad latent factors, such as internalizing, fear, and distress, can adequately capture multiple associations between additional clinical and behavioral indicators and individual disorders that comprise the latent factors. Indeed, these results provide additional evidence of criterion validity associated with a more parsimonious and empirically informed conceptualization of psychopathology, potentially leading to more efficient assessments (Ruggero et al., 2019). Consistent with past studies (Naragon-Gainey & Watson, 2011; Sunderland et al., 2021), the results suggest that the associations between the clinical and behavioral validators and emotional problems generally operated at a higher order level. For example, higher-order dimensions (internalizing, fear, distress) were significantly and strongly associated with clinical and behavioral validators, whilst the unique contribution of each disorder provided smaller and sometimes inconsistent predictions of this relationship.

Looking at the associations between first-order latent factors (distress and fear) and clinical and behavioral indicators, the internalizing factor could not sufficiently explain the total association between distress and suicidal ideation, dysfunctional attitudes, and readiness to seek help, as well as fear and the readiness to seek help. These direct effects suggest that there are additional factors unique to first-order factors (fear and distress) that increase the association with clinical and behavioral indicators than would be expected if estimated via the second-order internalizing factor alone. This provides evidence of incremental validity of the first-order factors over and above the second-order internalizing factor with respect to clinical and behavioral indicators and suggest that first-order factors may be differentiated from internalizing psychopathology more broadly. For example, the direct associations between distress and suicidal ideation and dysfunctional attitudes indicates that knowledge of an individual’s propensity to experience internalizing psychopathology in general may not provide enough detail regarding their likelihood of also experiencing suicidal ideation and dysfunctional attitudes. Additionally, the first-order factor, fear, and direct association with readiness to seek help, also suggests that there is something unique about the fear factor in explaining one’s propensity to seek out help.

Although broad dimensions, such as internalizing, fear, and distress, have been shown to have strong criterion validity across most published investigations and account for most of the association for panic and PTSD symptoms (Kotov et al., 2017; Kotov et al., 2021), the current study also demonstrated the importance of symptom scales. For suicidal ideation, higher-order dimensions (i.e., fear, distress, internalizing) were strongly and significantly associated, whilst the unique contribution of each disorder, besides depression, provided little prediction of this relationship. Nevertheless, there was a positive, direct effect on probable MDD, which indicated that suicidal ideation was positively associated with the part of probable MDD that remained after adjusting for the pathology depression shares with other emotional problems. Therefore, there appears to be some modest connection between suicidal ideation and depression, even after considering what is shared between depression and internalizing more broadly. In addition, the zero-order association between GAD, social anxiety, fear and suicidal ideation were positive (total effects), whereas the partial association (i.e., partialing out variance shared with other symptoms; direct effect) were negative, suggesting that suicidal ideation was positively related to GAD, social anxiety, and fear on a bivariate level, but inversely related to the part of GAD, social anxiety, and fear that did not overlap with other dimensions. These results support research that having a depressive disorder or having both a depressive and anxiety disorder (when compared to having just an anxiety disorder) had a strong influence on suicidal ideation as shown in many studies (Eikelenboom et al., 2012; Wiebenga et al., 2021).

Relatedly, there was a small association between dysfunctional attitudes and social phobia, even after considering what is shared between social phobia and internalizing more broadly. Concerning readiness to seek help, the current study demonstrated that the internalizing liability was strongly related to readiness to seek help. Importantly, it appears that whether an individual is ready to seek treatment provides little information on the probability of any specific psychological disorder. There was a small, negative relationship between readiness to seek help and PTSD and social phobia, suggesting that those with PTSD and social anxiety disorder were less inclined to seek help. This is in line with previous research suggesting that individuals with social phobia and PTSD may be less likely to seek support due to the nature of the disorder (Griffiths, 2013; Smith, 2020). Nevertheless, this finding does not augment the overall positive effect showing that the collective burden of all internalizing disorders contributed to the inclination to seek help.

These results support the idea that multiple levels of breadth are essential to consider in psychopathology research. In particular, the results have important implications for classifying and evaluating emotional disorders. Previous research has shown that latent models help describe connections between multiple emotional disorders at the syndrome and symptom levels (Wright et al., 2013). Additionally, research suggests that dimensional assessments for psychopathology can increase inter-diagnostic reliability by 15%, and validity by 37% compared to traditional categorical approaches (Markon et al., 2011). Using higher-order dimensions of psychopathology can also simplify the complex presentation of multiple co-occurring disorders and suggest valid constructs for future investigations. Utilizing dimensional measurements allows for a more detailed and sophisticated exploration of the circular nature between broader rates of internalizing symptoms and clinical variables of interest, in contrast to traditional categorical diagnoses (Forbes et al., 2019). Our results suggest that suicidal ideation was broadly associated with internalizing problems, and at a more fine-grained level, with depression-specific problems. Relatedly, dysfunctional attitudes also appear to be broadly associated with internalizing problems, particularly distress, and with specific social anxiety problems too. These findings contribute to the utility of specifying multiple levels of the psychopathology hierarchy simultaneously rather than focusing on aspects of psychopathology that may be too narrow or broad. The occurrence of mental disorders has remained constant over the past few decades (James et al., 2017). This emphasizes the challenges associated with treating psychopathology once it has emerged and underscores the significance of primary prevention efforts (McDaid et al., 2019). Cost-effective preventive measures are most impactful when directed at high-risk groups rather than the entire population (Arango et al., 2018). However, existing diagnostic manuals are primarily designed for diagnosing fully developed disorders and offer limited guidance for identifying individuals with early-stage psychopathology that have not yet reached clinical levels. Dimensional models of psychopathology are better able to offer a comprehensive understanding of subthreshold psychopathology, presenting a nuanced and multidimensional view of vulnerabilities, which can serve as a valuable resource for preventive strategies (Forbes et al., 2019). Prevention programs may find dimensional approaches that concentrate on common mechanisms particularly effective. These approaches have the distinct advantage of addressing temperamental risk factors like neuroticism, rather than exclusively concentrating on reducing symptoms associated with a specific diagnostic status (Zalta & Shankman, 2016). The results should be interpreted in the context of limitations. One limitation of the findings is that the cross-sectional nature of the survey precludes the ability to make conclusions regarding causality or temporality between psychopathology and suicidal ideation, dysfunctional attitudes, and readiness to seek help. Furthermore, the diagnostic categorization was based on self-reports.

In conclusion, among a large population-based sample of college students, psychopathology might be best conceptualized and organized using a hierarchical structure of latent factors that differ in their specificity. These higher-order latent factors can adequately and more efficiently account for most of the multiple pairwise associations evident between indicators of psychopathology and clinical and behavioral validators. However, there were significant, direct effects identified between symptom validators and clinical and behavioral variables factors, which provide further evidence for the utility of estimating multiple levels of the psychopathology hierarchy simultaneously.

Highlights.

  • Clinical and behavioral validators are strongly linked to higher-order dimensions like internalizing, rather than specific emotional disorders, emphasizing shared associations.

  • First-order factors (distress and fear) add to our understanding of associations with clinical validators beyond the second-order internalizing factor.

  • Using a dimensional approach to assess psychopathology considers both the specific symptoms and broader dimensions in clinical practice and can lead to more precise assessments.

Funding:

The current study was funded by the National Institutes of Health (Grant No. 5R01MH115128-03).

Footnotes

2

An exploratory factor analysis was conducted on half of the data, followed by a confirmatory factor analysis (CFA) on the other half. Parallel analysis was used to extract two factors, as suggested by the scree plot. The standardized factor loadings on the first factor were as follows: social phobia (0.74), generalized anxiety disorder (GAD) (0.67), and probable major depressive disorder (MDD) (0.61). On the second factor, the loadings were panic disorder (0.56) and post-traumatic stress disorder (PTSD) (0.50). The CFA, using the same data set, yielded satisfactory fit indices (RMSEA = 0.03, CFI = 0.98, TLI = 0.99), supporting the specified model.

3

The final column in Table 2 presents the percentage of the total effect for each outcome that was accounted for by factors higher in the hierarchy. For symptom scales, this column indicates the total effect accounted for by distress, fear, and internalizing factors. For distress and fear factors, this column indicates the total effect accounted for by internalizing. This percentage represents the magnitude of the indirect effect, and it is computed by subtracting the direct effect from the total effect and dividing the difference by the total effect.

Declarations of interest: none

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Contributor Information

Candice Basterfield, The Pennsylvania State University, University Park.

Ellen E. Fitzsimmons-Craft, Washington University, St. Louis.

C. Barr Taylor, Stanford University and Palo Alto University.

Daniel Eisenberg, University of California, Los Angeles.

Denise E. Wilfley, Washington University, St. Louis

Michelle G. Newman, The Pennsylvania State University, University Park

References

  1. Arango C, Díaz-Caneja CM, McGorry PD, Rapoport J, Sommer IE, Vorstman JA, McDaid D, Marín O, Serrano-Drozdowskyj E, Freedman R, & Carpenter W. (2018). Preventive strategies for mental health. The Lancet Psychiatry, 5(7), 591–604. 10.1016/S2215-0366(18)30057-9 [DOI] [PubMed] [Google Scholar]
  2. Auerbach RP, Mortier P, Bruffaerts R, Alonso J, Benjet C, Cuijpers P, Demyttenaere K, Ebert DD, Green JG, Hasking P, Murray E, Nock MK, Pinder-Amaker S, Sampson NA, Stein DJ, Vilagut G, Zaslavsky AM, & Kessler RC (2018). WHO World Mental Health Surveys International College Student Project: Prevalence and distribution of mental disorders. Journal of Abnormal Psychology, 127(7), 623–638. 10.1037/abn0000362 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Baek JH, Heo JY, Fava M, Mischoulon D, Nierenberg A, Hong JP, Roh S, & Jeon HJ (2015). Anxiety symptoms are linked to new-onset suicidal ideation after six months of follow-up in outpatients with major depressive disorder. J Affect Disord, 187, 183–187. 10.1016/j.jad.2015.08.006 [DOI] [PubMed] [Google Scholar]
  4. Beevers CG, Strong DR, Meyer B, Pilkonis PA, & Miller IR (2007). Efficiently assessing negative cognition in depression: an item response theory analysis of the Dysfunctional Attitude Scale. Psychol Assess, 19(2), 199–209. 10.1037/1040-3590.19.2.199 [DOI] [PubMed] [Google Scholar]
  5. Cohen J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159. 10.1037/0033-2909.112.1.155 [DOI] [PubMed] [Google Scholar]
  6. Conway CC, Forbes MK, Forbush KT, Fried EI, Hallquist MN, Kotov R, Mullins-Sweatt SN, Shackman AJ, Skodol AE, South SC, Sunderland M, Waszczuk MA, Zald DH, Afzali MH, Bornovalova MA, Carragher N, Docherty AR, Jonas KG, Krueger RF, . . . Eaton NR (2019). A Hierarchical Taxonomy of Psychopathology Can Transform Mental Health Research. Perspect Psychol Sci, 14(3), 419–436. 10.1177/1745691618810696 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Drum DJ, Brownson C, Burton Denmark A, & Smith SE (2009). New data on the nature of suicidal crises in college students: Shifting the paradigm. Professional Psychology: Research and Practice, 40(3), 213–222. 10.1037/a0014465 [DOI] [Google Scholar]
  8. Eaton NR, Krueger RF, Markon KE, Keyes KM, Skodol AE, Wall M, Hasin DS, & Grant BF (2013). The structure and predictive validity of the internalizing disorders. J Abnorm Psychol, 122(1), 86–92. 10.1037/a0029598 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Eikelenboom M, Smit JH, Beekman AT, & Penninx BW (2012). Do depression and anxiety converge or diverge in their association with suicidality? J Psychiatr Res, 46(5), 608–615. 10.1016/j.jpsychires.2012.01.025 [DOI] [PubMed] [Google Scholar]
  10. Eisenberg D, Hunt J, & Speer N. (2013). Mental health in American colleges and universities: variation across student subgroups and across campuses. J Nerv Ment Dis, 201(1), 60–67. 10.1097/NMD.0b013e31827ab077 [DOI] [PubMed] [Google Scholar]
  11. Forbes D, Elhai JD, Lockwood E, Creamer M, Frueh BC, & Magruder KM (2012). The structure of posttraumatic psychopathology in veterans attending primary care. J Anxiety Disord, 26(1), 95–101. 10.1016/j.janxdis.2011.09.004 [DOI] [PubMed] [Google Scholar]
  12. Forbes D, Parslow R, Creamer M, O’Donnell M, Bryant R, McFarlane A, Silove D, & Shalev A. (2010). A longitudinal analysis of posttraumatic stress disorder symptoms and their relationship with Fear and Anxious-Misery disorders: implications for DSM-V. J Affect Disord, 127(1–3), 147–152. 10.1016/j.jad.2010.05.005 [DOI] [PubMed] [Google Scholar]
  13. Forbes MK, Rapee RM, & Krueger RF (2019). Opportunities for the prevention of mental disorders by reducing general psychopathology in early childhood. Behavior Research and Theory, 119, 1–9. 10.1016/j.brat.2019.103411 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Gallagher R. (2015). National Survey of College Counseling Centers 2014. The International Association of Counseling Services IACS [Online], Project Report 2015. http://d-scholarship.pitt.edu/ [Google Scholar]
  15. Griffiths KM (2013). Towards a framework for increasing help-seeking for social anxiety disorder. Australian & New Zealand Journal of Psychiatry, 47(10), 899–903. 10.1177/0004867413493335 [DOI] [PubMed] [Google Scholar]
  16. Hankin BL, & Abramson LY (2001). Development of gender differences in depression: An elaborated cognitive vulnerability-transactional stress. Psychological Bulletin, 6(6), 773–796. 10.1037/0033-2909.127.6.773 [DOI] [PubMed] [Google Scholar]
  17. Hu L. t., & Bentler PM (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. 10.1080/10705519909540118 [DOI] [Google Scholar]
  18. James SL, Abate D, Abate KH, Abay SM, Abbafati C, Abbasi N, Abbastabar H, Abd-Allah F, Abdela J, Abdelalim A, & Abdollahpour I. (2017). Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: A systematic analysis for the global burden of disease study. The Lancet, 392(10159), 1789–1858. 10.1016/S0140-6736(18)32279-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Jao NC, Robinson LD, Kelly PJ, Ciecierski CC, & Hitsman B. (2019). Unhealthy behavior clustering and mental health status in United States college students. Journal of American College Health, 67(8), 790–800. 10.1080/07448481.2018.1515744 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, & Walters EE (2005). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62(6), 593–602. 10.1001/archpsyc.62.6.593 [DOI] [PubMed] [Google Scholar]
  21. Kotov R, Krueger RF, Watson D, Achenbach TM, Althoff RR, Bagby RM, Brown TA, Carpenter WT, Caspi A, Clark LA, Eaton NR, Forbes MK, Forbush KT, Goldberg D, Hasin D, Hyman SE, Ivanova MY, Lynam DR, Markon K, . . . Zimmerman M. (2017). The Hierarchical Taxonomy of Psychopathology (HiTOP): A dimensional alternative to traditional nosologies. Journal of Abnormal Psychology, 126(4), 454–477. 10.1037/abn0000258 [DOI] [PubMed] [Google Scholar]
  22. Kotov R, Krueger RF, Watson D, Cicero DC, Conway CC, DeYoung CG, Eaton NR, Forbes MK, Hallquist MN, Latzman RD, Mullins-Sweatt SN, Ruggero CJ, Simms LJ, Waldman ID, Waszczuk MA, & Wright AGC (2021). The Hierarchical Taxonomy of Psychopathology (HiTOP): A Quantitative Nosology Based on Consensus of Evidence. Annu Rev Clin Psychol, 17, 83–108. 10.1146/annurev-clinpsy-081219-093304 [DOI] [PubMed] [Google Scholar]
  23. Kroenke K, & Spitzer RL (2002). The PHQ-9: A new depression diagnostic and severity measure. Psychiatric Annals, 32(9), 509–515. 10.3928/0048-5713-20020901-06 [DOI] [Google Scholar]
  24. Kroenke K, Spitzer RL, Williams JBW, & Löwe B. (2010). The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: A systematic review. General Hospital Psychiatry, 32(4), 345–359. 10.1016/j.genhosppsych.2010.03.006 [DOI] [PubMed] [Google Scholar]
  25. Krueger RF (1999). The structure of common mental disorders [doi: 10.1001/archpsyc.56.10.921]. Archives of General Psychiatry, 56(10), 921–926. [DOI] [PubMed] [Google Scholar]
  26. Krueger RF, Kotov R, Watson D, Forbes MK, Eaton NR, Ruggero CJ, Simms LJ, Widiger TA, Achenbach TM, & Bach B. (2018). Progress in achieving quantitative classification of psychopathology. World psychiatry, 17(3), 282–293. 10.1002/wps.20566 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Krueger RF, & Markon KE (2006). Reinterpreting comorbidity: a model-based approach to understanding and classifying psychopathology. Annu Rev Clin Psychol, 2, 111–133. 10.1146/annurev.clinpsy.2.022305.095213 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Li CH (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behav Res Methods, 48(3), 936–949. 10.3758/s13428-015-0619-7 [DOI] [PubMed] [Google Scholar]
  29. Lockwood E, & Forbes D. (2014). Posttraumatic Stress Disorder and Comorbidity: Untangling the Gordian Knot. Psychological Injury and Law, 7(2), 108–121. 10.1007/s12207-014-9189-8 [DOI] [Google Scholar]
  30. López-Castro T, Brandt L, Anthonipillai NJ, Espinosa A, & Melara R. (2021). Experiences, impacts and mental health functioning during a COVID-19 outbreak and lockdown: Data from a diverse New York City sample of college students. PloS one, 16(4), 1–17. 10.1371/journal.pone.0249768 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Manea L, Gilbody S, & McMillan D. (2012). Optimal cut-off score for diagnosing depression with the Patient Health Questionnaire (PHQ-9): A meta-analysis. Canadian Medical Association Journal, 184(3), E191–E196. 10.1503/cmaj.110829 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Markon KE, Chmielewski M, & Miller CJ (2011). The reliability and validity of discrete and continuous measures of psychopathology: a quantitative review. Psychological Bulletin, 137(5), 856–879. 10.1037/a0023678 [DOI] [PubMed] [Google Scholar]
  33. Matud MP, Diaz A, Bethencourt JM, & Ibanez I. (2020). Stress and Psychological Distress in Emerging Adulthood: A Gender Analysis. J Clin Med, 9(9). 10.3390/jcm9092859 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. McDaid D, Park AL, & Wahlbeck K. (2019). The economic case for the prevention of mental illness. Annual Review of Public Health 40, 373–389. 10.1146/annurev-publhealth-040617-013629 [DOI] [PubMed] [Google Scholar]
  35. McElroy E, Belsky J, Carragher N, Fearon P, & Patalay P. (2018). Developmental stability of general and specific factors of psychopathology from early childhood to adolescence: Dynamic mutualism or p-differentiation? Journal of Child Psychology and Psychiatry, 59(6), 667–675. 10.1111/jcpp.12849 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Mennin DS, Heimberg RG, Fresco DM, & Ritter MR (2008). Is generalized anxiety disorder an anxiety or mood disorder? Considering multiple factors as we ponder the fate of GAD. Depression and Anxiety, 25(4), 289–299. 10.1002/da.20493 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Mortier P, Auerbach RP, Alonso J, Bantjes J, Benjet C, Cuijpers P, Ebert DD, Green JG, Hasking P, Nock MK, O’Neill S, Pinder-Amaker S, Sampson NA, Vilagut G, Zaslavsky AM, Bruffaerts R, Kessler RC, & Collaborators WW-I (2018). Suicidal Thoughts and Behaviors Among First-Year College Students: Results From the WMH-ICS Project. J Am Acad Child Adolesc Psychiatry, 57(4), 263–273 e261. 10.1016/j.jaac.2018.01.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Naragon-Gainey K, & Watson D. (2011). Clarifying the dispositional basis of social anxiety: A hierarchical perspective. Personality and Individual Differences, 50(7), 926–934. 10.1016/j.paid.2010.07.012 [DOI] [Google Scholar]
  39. Newman MG, Holmes M, Zuellig AR, Kachin KE, & Behar E. (2006). The reliability and validity of the Panic Disorder Self-Report: A new diagnostic screening measure of panic disorder [Journal; Peer Reviewed Journal; Original Journal Article]. Psychological Assessment, 18(1), 49–61. 10.1037/1040-3590.18.1.49 [DOI] [PubMed] [Google Scholar]
  40. Newman MG, Kachin KE, Zuellig AR, Constantino MJ, & Cashman-McGrath L. (2003). The Social Phobia Diagnostic Questionnaire: Preliminary validation of a new self-report diagnostic measure of social phobia. Psychological Medicine, 33(4), 623–635. 10.1017/S0033291703007669 [DOI] [PubMed] [Google Scholar]
  41. Newman MG, Zuellig AR, Kachin KE, Constantino MJ, Przeworski A, Erickson T, & Cashman-McGrath L. (2002). Preliminary reliability and validity of the Generalized Anxiety Disorder Questionnaire-IV: A revised self-report diagnostic measure of generalized anxiety disorder. Behavior Therapy, 33(2), 215–233. 10.1016/S0005-7894(02)80026-0 [DOI] [Google Scholar]
  42. Prenoveau JM, Zinbarg RE, Craske MG, Mineka S, Griffith JW, & Epstein AM (2010). Testing a hierarchical model of anxiety and depression in adolescents: a tri-level model. J Anxiety Disord, 24(3), 334–344. 10.1016/j.janxdis.2010.01.006 [DOI] [PubMed] [Google Scholar]
  43. Prins A, Ouimette P, Kimerling R, Cameron RP, Hugelshofer DS, Shaw-Hegwer J, Thrailkill A, Gusman FD, & Sheikh JI (2003). The Primary Care PTSD Screen (PC-PTSD): Development and operating characteristics. Primary Care Psychiatry, 9(1), 9–14. 10.1185/135525703125002360 [DOI] [Google Scholar]
  44. Ruggero CJ, Kotov R, Hopwood CJ, First M, Clark LA, Skodol AE, Mullins-Sweatt SN, Patrick CJ, Bach B, Cicero DC, Docherty A, Simms LJ, Bagby RM, Krueger RF, Callahan JL, Chmielewski M, Conway CC, De Clercq B, Dornbach-Bender A, . . . Zimmermann J. (2019). Integrating the Hierarchical Taxonomy of Psychopathology (HiTOP) into clinical practice. J Consult Clin Psychol, 87(12), 1069–1084. 10.1037/ccp0000452 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Slade T, McEvoy PM, Chapman C, Grove R, & Teesson M. (2015). Onset and temporal sequencing of lifetime anxiety, mood and substance use disorders in the general population. Epidemiol Psychiatr Sci, 24(1), 45–53. 10.1017/S2045796013000577 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Smith JR, Workneh A, & Yaya S. . (2020). Barriers and facilitators to help-seeking for individuals with posttraumatic stress disorder: A systematic review. Journal of Traumatic Stress, 33(2), 137–150. [DOI] [PubMed] [Google Scholar]
  47. Starr LR, Conway CC, Hammen CL, & Brennan PA (2014). Transdiagnostic and disorder-specific models of intergenerational transmission of internalizing pathology. Psychol Med, 44(1), 161–172. 10.1017/S003329171300055X [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Sunderland M, Forbes MK, Mewton L, Baillie A, Carragher N, Lynch SJ, Batterham PJ, Calear AL, Chapman C, Newton NC, Teesson M, & Slade T. (2021). The structure of psychopathology and association with poor sleep, self-harm, suicidality, risky sexual behavior, and low self-esteem in a population sample of adolescents. Dev Psychopathol, 33(4), 1208–1219. 10.1017/S0954579420000437 [DOI] [PubMed] [Google Scholar]
  49. Sunderland M, & Slade T. (2015). The relationship between internalizing psychopathology and suicidality, treatment seeking, and disability in the Australian population. J Affect Disord, 171, 6–12. 10.1016/j.jad.2014.09.012 [DOI] [PubMed] [Google Scholar]
  50. Teesson M, Slade T, & Mills K. . (2009). Comorbidity in Australia: Findings of the 2007 National Survey of Mental Health and Wellbeing. Australian and New Zealand Journal of Psychiatry, 43(7), 606–614. 10.1080/00048670902970908 [DOI] [PubMed] [Google Scholar]
  51. van Buuren S, & Groothuis-Oudshoorn K. (2011). mice: Multivariate imputation by chained equations in R. Journal of statistical software, 45(3), 1548–7660. 10.18637/jss.v045.i03 [DOI] [Google Scholar]
  52. Watson D. (2005). Rethinking the mood and anxiety disorders: A quantitative hierarchical model for DSM-V. Journal of Abnormal Psychology, 114(4), 522–536. 10.1037/0021-843X.114.4.522 [DOI] [PubMed] [Google Scholar]
  53. Wells KB, Sturm R, & Burnam MA (2012). National Survey of Alcohol, Drug, and Mental Health Problems [Healthcare for Communities], 2000–2001. Inter-university Consortium for Political and Social Research. [Google Scholar]
  54. Wiebenga JX, Eikelenboom M, Heering HD, van Oppen P, & Penninx BW (2021). Suicide ideation versus suicide attempt: Examining overlapping and differential determinants in a large cohort of patients with depression and/or anxiety. Aust N Z J Psychiatry, 55(2), 167–179. 10.1177/0004867420951256 [DOI] [PubMed] [Google Scholar]
  55. Wright AG, Krueger RF, Hobbs MJ, Markon KE, Eaton NR, & Slade T. (2013). The structure of psychopathology: toward an expanded quantitative empirical model. J Abnorm Psychol, 122(1), 281–294. 10.1037/a0030133 [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Yapan S, Türkçapar MH, & Boysan M. (2020). Rumination, automatic thoughts, dysfunctional attitudes, and thought suppression as transdiagnostic factors in depression and anxiety. Current Psychology, 41(9), 5896–5912. 10.1007/s12144-020-01086-4 [DOI] [Google Scholar]
  57. Zalta AK, & Shankman SA (2016). Conducting psychopathology prevention research in the RDoC era. Clinical Psychology: Science and Practice, 23, 94–104. 10.1111/cpsp.12144 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES