Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Jun 1.
Published in final edited form as: Psychiatry Res. 2022 Mar 28;312:114534. doi: 10.1016/j.psychres.2022.114534

Predictive Utility of Symptom Measures in Classifying Anxiety and Depression: A Machine-Learning Approach

Kevin Liu 1, Brian Droncheff 1, Stacie L Warren 1,*
PMCID: PMC9117511  NIHMSID: NIHMS1796872  PMID: 35381506

Abstract

Major depressive disorder (MDD) and generalized anxiety disorder (GAD) are highly prevalent, co-occurring disorders with significant symptom overlap, posing challenges in accurately distinguishing and diagnosing these disorders. The tripartite model proposes that anxious arousal is specific to anxiety and anhedonia is specific to depression, though anxious apprehension may play a greater role in GAD than anxious arousal. The present study tested the efficacy of the Mood and Anxiety Symptom Questionnaire anhedonic depression (MASQ-AD) and anxious arousal (MASQ-AA) scales and the Penn State Worry Questionnaire (PSWQ) in identifying lifetime or current MDD, current major depressive episode (MDE), and GAD using binary support vector machine learning algorithms in an adult sample (n = 150). The PSWQ and MASQ-AD demonstrated predictive utility in screening for and identification of GAD and current MDE respectively, with the MASQ-AD eight-item subscale outperforming the MASQ-AD 14-item subscale. The MASQ-AA did not predict MDD, current MDE, or GAD, and the MASQ-AD did not predict current or lifetime MDD. The PSWQ and MASQ-AD are efficient and accurate screening tools for GAD and current MDE. Results support the tripartite model in that anhedonia is unique to depression, but inclusion of anxious apprehension as a separate dimension of anxiety is warranted.

Keywords: Major Depression, Generalized Anxiety Disorder, Anxious Arousal, Anxious Apprehension, Predictive Validity

1. Introduction

Major depressive disorder (MDD) and generalized anxiety disorder (GAD) are highly prevalent mental disorders in the United States, affecting 20.6% and 4.3% of individuals, respectively (Hasin et al., 2018; Kessler et al., 2012). Yet, sensitivity in detecting GAD and specificity in separating GAD from MDD are fairly low in clinical settings, which is especially concerning given that the co-occurrence of GAD and MDD is estimated to be 40–60% (Carter et al., 2001, Kessler et al., 2005). For example, only 1.5% of elderly GAD patients were correctly diagnosed with GAD, and elderly GAD patients were twice as likely to receive a diagnosis of a depressive disorder than an anxiety disorder (Calleo et al., 2009). Suboptimal diagnostic accuracy is compounded by low interrater reliability for GAD and MDD categories based on DSM-5 criteria (Regier et al., 2013). Several options for increasing diagnostic accuracy of GAD have been suggested, such as the use of biomarkers (although see Miller & Rockstroh, 2013 for a thoughtful critique), including neuroanatomical correlates of and cortisol levels associated with GAD and MDD (Beesdo et al., 2009; Canu et al., 2015; Hek et al., 2013; Hinkelmann et al., 2012; Oathes et al., 2015). However, in primary care settings, where neuroimaging or measurement of hormone levels may not be viable methods of screening, self-report screening measures are a useful tool for diagnosis and differentiation of GAD and MDD (O’Connor et al., 2009; Plummer et al., 2016).

Though multiple screening measures for GAD and MDD exist, such as the Beck Depression Inventory-II (BDI-II), the Beck Anxiety Inventory (BAI), and the General Anxiety Disorder-7 (GAD-7; Beck et al., 1988; Beck et al., 1996; Spitzer et al., 2006), research suggests that these measures may not demonstrate discriminant validity, particularly in populations with high levels of general distress (Creamer et al., 1995; Feldman, 1993; Fischer & Goethe, 1997; Rutter & Brown, 2017). For example, the BDI-II has demonstrated inadequate discriminant validity due to its high correlation with the BAI (r = 0.53–0.6; Segal et al., 2008), and the GAD-7 was found to have strong correlations with measures of general distress and depression (Kertz et al., 2013). The inclusion of several symptoms that are common to both anxiety and depression within these questionnaires suggests that these measures are more reflective of general distress than specific to anxiety or depression disorders (Feldman, 1993; Gotlib & Cane, 1989; Keogh & Reidy, 2000; Watson & Kendall, 1989). However, purification of the BDI-II and BAI by removing items that were nonspecific to depression and anxiety yielded only marginal increases in the measures’ ability to discriminate between depressive and anxiety disorders (Stulz & Crits-Christoph, 2010). To account for the observed symptom overlap and high disorder co-occurrence, the tripartite model of anxiety and depression (Clark & Watson, 1991) posits that a general psychological distress component marked by high negative affect is shared by anxiety and depression. Additionally, the model posits that physiological hyperarousal is specific to anxiety and that anhedonia is specific to depression. The Mood and Anxiety Symptom Questionnaire (MASQ) is based on the tripartite model and consists of three subscales measuring general distress, anxious arousal, and anhedonic depression (Watson et al., 1995a; Watson et al., 1995b).

The MASQ anhedonic depression scale (MASQ-AD) has shown good convergent and discriminant validity and predictive utility for both current and lifetime MDD, with greater accuracy for current rather than lifetime depressive disorders (Bredemeier et al., 2010; Buckby et al., 2007). Additionally, the MASQ-AD scale has been shown to be a better screening measure for current rather than lifetime depressive disorders (Bredemeier et al., 2010). Past literature has also demonstrated that the 22 MASQ-AD items load onto two separate factors, with eight positively-worded items loading onto a factor representing depressive symptoms and 14 reverse-scored items loading onto a positive emotionality factor (Nitschke et al., 2001, Watson et al., 1995b). The eight-item MASQ-AD subscale (MASQ-AD8) has demonstrated better accuracy in screening for depression than the 14-item MASQ-AD subscale (MASQ-AD14) in nonpatient samples (Bredemeier et al., 2010).

The MASQ anxious arousal scale (MASQ-AA) has demonstrated limited utility in the prediction of GAD (Curtiss & Klemanski, 2015). This may be due to differing contributions of anxiety dimensions, anxious arousal, or the physiological and somatic symptoms of anxiety measured by the MASQ-AA, and anxious apprehension, or the tendency to engage in worry, as primary characteristics of GAD (Heller et al., 1997; Nitschke et al., 1999). The constructs of anxious arousal and anxious apprehension have demonstrated discriminant validity, have distinct neuroanatomical correlates, and even distinct patterns of relationships with executive function (Castagna et al., 2018; Engels et al., 2007; Engels et al., 2010; Sharp et al., 2015, Warren et al., 2013; 2021). The Penn State Worry Questionnaire (PSWQ), which measures excessive worry or anxious apprehension, has shown good convergent validity with other measures of anxious apprehension and discriminant validity with measures of depression and other anxiety disorders (Brown et al., 1992; Liu et al., 2021; Meyer et al., 1990; Stöber, 1998). Importantly, the PSWQ has demonstrated low convergence with the MASQ-AA, indicating measurement of distinct constructs, anxious apprehension and anxious arousal. The PSWQ has also demonstrated predictive utility in screening for and identifying generalized anxiety, both from normal controls and from other anxiety disorders (Behar et al., 2003; Brown et al., 1992; Fresco et al., 2003).

Recent studies have leveraged machine learning algorithms to predict diagnostic categories, including GAD and MDD. For example, Hilbert et al. (2017) used binary support vector machines and found that questionnaire data, specifically the PSWQ, BDI-II (Beck et al., 1996), Intolerance of Uncertainty Scale-12 (Carleton et al., 2007), and the trait version of the State-Trait-Anxiety Index (Spielberger et al., 1983), were well-suited for case-classification of MDD and GAD against healthy controls, but were not capable of accurately discriminating between MDD and GAD. Additionally, Tennenhouse et al. (2020) used logistic regression, neural network, and random forest models and found that questionnaire data could accurately predict clinical diagnoses of MDD and anxiety disorders. Though the utility of questionnaires in predicting diagnostic categories has traditionally been evaluated through receiver operating characteristic (ROC) curves determining area under the curve and optimal cut-off scores, machine learning methods can also be used to assess classification accuracy of questionnaires in screening for or identifying diagnostic categories using item-level responses without needing to aggregate item responses into a sum score (Gonzalez, 2020).

The present study employed binary support vector machine algorithms using item-level responses to the MASQ-AD, MASQ-AA, and PSWQ to predict clinical diagnoses of MDD, current major depressive episode (MDE), and GAD to test the predictive utility and discriminant validity of these measures. By comparing model predictions with clinical diagnoses, we will also be able to assess the clinical utility of these measures in case classification, or detection of psychopathology among a sample of predominantly psychologically healthy controls, and disorder classification, or distinguishing between disorders, using machine learning models. Furthermore, we will be able to identify which constructs, anhedonic depression, anxious arousal, and anxious apprehension, are important in the classification of clinical diagnoses. Finally, by applying similar models to predict the same clinical diagnoses, we will be able to make direct comparisons of the predictive utility of each measure for each clinical diagnosis. Based on the tripartite model discussed above, we hypothesized that the MASQ-AD, measuring anhedonic depression, would demonstrate high predictive value for MDD and current MDE diagnoses with the MASQ-AD8, measuring depressive symptoms, outperforming the MASQ-D14, measuring positive emotionality. We also hypothesized that the PSWQ, measuring anxious apprehension, would demonstrate high predictive value for GAD diagnoses, while the MASQ-AA, measuring anxious arousal, would not demonstrate significant predictive value for MDD, current MDE, or GAD diagnoses.

2. Methods

2.1. Participants

The present study is a secondary data analysis on data collected as part of two separate studies examining affective, cognitive, and neural risk factors for psychopathology (e.g., Bredemeier et al., 2016; Geller et al., 2021; Madian et al., 2019; Warren et al., 2013; 2021). Previously published articles overlapping with the present samples examined different research questions and utilized different approaches and measures (e.g., fMRI, neuropsychological testing, factor analysis, structural equation modeling). Adult participants (n = 285) aged 18 to 55, were recruited from a university, the community, and two community mental health clinics if they were experiencing anxiety and/or mood symptoms, or no symptoms at all. All participants completed a structured clinical interview and a series of questionnaires, including the PSWQ, MASQ-AA, MASQ-AD, and were paid for their time. Based on diagnostic categories derived from the structured clinical interview, the participants were considered to be in one of four groups. The control group (n = 98) consisted of participants with no current or lifetime diagnoses of psychopathology. The MDD group (n = 26) consisted of participants with a diagnosis of lifetime (n = 21) or current MDD (n = 5) without any other co-morbid diagnoses outside of other depressive disorders. The current MDE group (n = 24) consisted of participants who were experiencing a current major depressive episode at the time of the clinical interview, who may have had other co-morbid diagnoses outside of depressive disorders unlike the MDD group. The GAD group (n = 26) consisted of participants with a diagnosis of current GAD, without any other co-morbid diagnoses outside of depressive disorders. 135 participants in other diagnostic categories (e.g., eating disorders, substance use disorders, etc.) were not included in the analysis, leaving a total of 150 participants, consisting of 69 undergraduates and 81 participants from the community. It should be noted that while the control, MDD, and GAD groups were non-overlapping, the participants considered to be in the current MDE group overlapped with the MDD and GAD groups. Demographic information for each group is reported in the Supplementary Materials. The present study was approved by the Palo Alto University Institutional Review Board.

2.2. Measures

The Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Non-patient Edition (SCID-I/NP; First et al., 2002) administered and/or evaluated by author S. L. Warren, was used to assess current and lifetime psychopathology. Resulting diagnostic categories were used to group participants as described above and were used as the “correct” outcome against which predictions from our machine learning models were compared. The SCID was conducted on the same day as the participants completed the questionnaires.

The PSWQ is a 16-item self-report measure of excessive worry or anxious apprehension. As noted above, it has demonstrated convergent validity, divergent validity, and predictive utility (Behar et al., 2003; Brown et al., 1992; Fresco et al., 2003; Liu et al., 2021; Meyer et al., 1990; Stöber, 1998).

The MASQ-AA is a 17-item scale of the MASQ measuring anxious arousal or somatic anxiety. The MASQ-AA scale has demonstrated acceptable reliability, convergent validity, and divergent validity (Reidy & Keogh, 2000; Watson, et al., 1995a; Watson et al., 1995b).

The MASQ-AD is a 22-item scale of the MASQ measuring anhedonic depression. The MASQ-AD has demonstrated convergent validity, divergent validity, and predictive utility (Buckby et al., 2007; Reidy & Keogh, 2000; Watson, et al., 1995a; Watson et al., 1995b). As noted above, the MASQ-AD has been found to load onto two separate factors, with the MASQ-AD8 eight-item subscale measuring depressive symptoms and the MASQ-AD14 14-item subscale measuring positive emotionality (Nitschke et al., 2001). The MASQ-AD8 has shown greater predictive utility for depression than the MASQ-AD14 (Bredemeier et al., 2010).

2.3. Data Analysis

Descriptive statistics for each measure in each group were calculated using R (R Core Team, 2020). Missing data was imputed via multiple imputation with predictive mean matching using the “mice” package (van Buuren & Groothuis-Oudshoom, 2011). Binary support vector machines, implemented using the “caret” package (Kuhn, 2020), were used to assess the classification accuracy in five use-cases:

  1. Case classification: Separating participants in the MDD or GAD groups from participants in the control group.

  2. MDD detection: Separating participants in the MDD group from participants in the control group.

  3. Current MDE detection: Separating participants in the current MDE group from participants in the control group.

  4. GAD detection: Separating participants in the GAD group from participants in the control group

  5. Disorder classification: Separating participants in the MDD group from participants in the GAD group

For each use-case, each questionnaire was used in separate binary support vector machine models to predict outcomes, namely diagnostic group, and model predictions were compared to SCID diagnoses to evaluate classification accuracy. For each measure, a model was trained and tested using 10-fold cross-validation, and the models were evaluated across 100 repeated trials. Classification metrics calculated for each measure in each use-case included accuracy, balanced accuracy, sensitivity, specificity, positive predictive validity (PPV), negative predictive validity (NPV), and Cohen’s Kappa. It should be noted that while other classification metrics are sensitive to unequal group sizes, balanced accuracy and Cohen’s Kappa adjust for unbalanced group sizes. 95% confidence intervals of Kappa were also analyzed to determine stability of classification accuracy differences between the measures after resampling.

3. Results

3.1. Descriptive statistics

Descriptive statistics for each measure in each group are reported in the Supplementary Materials. Chi-square tests found that there was not a significant difference in race/ethnicity across diagnostic groups. However, diagnostic groups differed significantly in proportion of gender (χ2(3) = 10.4, p = .015), and post-hoc testing revealed that the GAD group had a significantly higher proportion of females compared to the control group (false discovery rate corrected p = .033). Missing data rate across all groups was 0.05%, and none of the demographic variables collected were found to be related to missingness.

3.2. Case Classification

The PSWQ was best able to detect cases of GAD and/or MDD from healthy controls in our sample, demonstrating 71.9% balanced accuracy and a Kappa of .437 (see Table 1 and Figure 1). The MASQ-AA, MASQ-AD, and two MASQ-AD subscales demonstrated poor accuracy (56.8% - 66.3% balanced accuracy; 61.3% - 69.9% PPV; 52.5% - 62.5% NPV) for case classification.

Table 1.

Case Classification Accuracy Metrics for Each Measure

Measure Accuracy Balanced Accuracy Sensitivity Specificity PPV NPV Kappa
PSWQ 71.9% 71.9% 71.7% 72.1% 75.0% 68.7% 0.437
MASQ-AA 56.5% 56.8% 51.5% 62.2% 61.3% 52.5% 0.136
MASQ-AD 66.2% 66.3% 65.0% 67.6% 69.9% 62.5% 0.324
MASQ-AD14 63.0% 62.7% 67.1% 58.2% 65.2% 60.3% 0.254
MASQ-AD8 63.4% 63.7% 59.2% 68.3% 68.4% 59.1% 0.272
Open in a new tab

Note. Accuracy, balanced accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and Cohen’s Kappa for the Penn State Worry Questionnaire (PSWQ), Mood and Anxiety Symptom Questionnaire (MASQ) Anxious Arousal Scale (MASQ-AA), MASQ Anhedonic Depression Scale (MASQ-AD), MASQ-AD 14-item subscale (MASQ-AD14), and MASQ-AD 8-item subscale (MASQ-AD8) for case classification.

Figure 1.

Figure 1

Open in a new tab

Confidence Intervals of Kappa for Each Measure

Note. Vertical tick marks indicate Kappa estimates, and horizontal bars indicate 95% confidence intervals of Kappa estimates.

3.3. MDD Detection

In our sample, none of the questionnaires demonstrated accurate detection of lifetime and current diagnoses of MDD from healthy controls with balanced accuracies that were not substantially different from chance (i.e., 50%; see Table 2 and Figure 1). Sensitivity was low (0.0% - 19.0%), indicating a high false negative rate, while specificity was high (90.6% - 100%), indicating correct classification of no MDD in model predictions.

Table 2.

Major Depressive Disorder (MDD) Detection Classification Accuracy Metrics for Each Measure

Measure Accuracy Balanced Accuracy Sensitivity Specificity PPV NPV Kappa
PSWQ 76.9% 48.8% 0.4% 97.2% 4.3% 78.6% −0.027
MASQ-AA 77.3% 50.6% 4.8% 96.5% 26.3% 79.3% 0.019
MASQ-AD 75.6% 54.8% 19.0% 90.6% 35.1% 80.8% 0.105
MASQ-AD14 79.0% 50.0% 0.0% 100.0% 0.0% 79.0% 0.000
MASQ-AD8 79.0% 50.0% 0.0% 100.0% 0.0% 79.0% 0.000
Open in a new tab

Note. Accuracy, balanced accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and Cohen’s Kappa for the Penn State Worry Questionnaire (PSWQ), Mood and Anxiety Symptom Questionnaire (MASQ) Anxious Arousal Scale (MASQ-AA), MASQ Anhedonic Depression Scale (MASQ-AD), MASQ-AD 14-item subscale (MASQ-AD14), and MASQ-AD 8-item subscale (MASQ-AD8) for major depressive disorder (MDD) detection.

3.4. Current MDE Detection

The MASQ-AD scale, particularly the MASQ-AD8 subscale, demonstrated the highest ability to detect current MDE cases from healthy controls (see Table 3 and Figure 1). Interestingly, the MASQ-AD8 subscale demonstrated greater classification accuracy (balanced accuracy = 78.4%; Kappa = .580) compared to the MASQ-AD14 subscale (balanced accuracy = 65.6%; Kappa = .355). The PSWQ and MASQ-AA demonstrated lower balanced accuracies (68.5% and 56.1% respectively). For all measures, sensitivity (22.8% −62.4%) and PPV (34.6% −72.8%) were lower than specificity (89.1% - 96.3%) and NPV (82.5% - 91.1%), which may reflect the smaller proportion of current MDE in our sample compared to controls (19.7%)

Table 3.

Current Major Depressive Episode (ctMDE) Detection Classification Accuracy Metrics for Each Measure

Accuracy Balanced Accuracy Sensitivity Specificity PPV NPV Kappa
PSWQ 81.7% 68.5% 46.7% 90.3% 54.1% 87.3% 0.373
MASQ-AA 76.3% 56.1% 22.8% 89.4% 34.6% 82.5% 0.127
MASQ-AD 85.3% 74.6% 56.9% 92.3% 64.4% 89.7% 0.496
MASQ-AD14 84.2% 65.6% 35.0% 96.3% 69.7% 85.8% 0.355
MASQ-AD8 88.0% 78.4% 62.4% 94.3% 72.8% 91.1% 0.580
Open in a new tab

Note. Accuracy, balanced accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and Cohen’s Kappa for the Penn State Worry Questionnaire (PSWQ), Mood and Anxiety Symptom Questionnaire (MASQ) Anxious Arousal Scale (MASQ-AA), MASQ Anhedonic Depression Scale (MASQ-AD), MASQ-AD 14-item subscale (MASQ-AD14), and MASQ-AD 8-item subscale (MASQ-AD8) for current major depressive episode (ctMDE) detection.

3.5. GAD Detection

The PSWQ was best able to detect cases of GAD from healthy controls in our sample, demonstrating 81.2% balanced accuracy (Kappa = .625; see Table 4 and Figure 1), and the MASQ-AD scale showed some ability in GAD detection with 74.2% balanced accuracy (Kappa = .474). The MASQ-AA scale, MASQ-AD14 subscale, and MASQ-AD8 subscale demonstrated lower balanced accuracies (56.4%, 64.3%, and 65.5%, respectively). For all measures, sensitivity (20.1% - 69.0%) and PPV (42.0% - 73.6%) were lower than specificity (87.7% - 93.8%) and NPV (81.4% - 91.9%), which may reflect the smaller proportion of GAD in our sample compared to controls (21.0%)

Table 4.

Generalized Anxiety Disorder (GAD) Detection Classification Accuracy Metrics for Each Measure

Measure Accuracy Balanced Accuracy Sensitivity Specificity PPV NPV Kappa
PSWQ 88.3% 81.2% 69.0% 93.4% 73.6% 91.9% 0.625
MASQ-AA 77.5% 56.4% 20.1% 92.7% 42.0% 81.4% 0.150
MASQ-AD 83.6% 74.2% 58.1% 90.4% 61.6% 89.0% 0.474
MASQ-AD14 77.9% 64.3% 41.0% 87.7% 47.0% 84.8% 0.283
MASQ-AD8 81.9% 65.5% 37.1% 93.8% 61.4% 84.9% 0.340
Open in a new tab

Note. Accuracy, balanced accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and Cohen’s Kappa for the Penn State Worry Questionnaire (PSWQ), Mood and Anxiety Symptom Questionnaire (MASQ) Anxious Arousal Scale (MASQ-AA), MASQ Anhedonic Depression Scale (MASQ-AD), MASQ-AD 14-item subscale (MASQ-AD14), and MASQ-AD 8-item subscale (MASQ-AD8) for generalized anxiety disorder (GAD) detection.

3.6. Disorder Classification

The PSWQ was best able to distinguish cases of GAD from cases of lifetime or current MDD with 81.2% balanced accuracy (Kappa = 0.622; see Table 5 and Figure 1). The MASQ-AA scale, MASQ-AD scale, MASQ-AD14 subscale, and MASQ-AD8 subscales demonstrated lower balanced accuracies (67.2%, 66%, 53%, and 55.9% respectively). GAD cases were coded as “0” and MDD cases were coded as “1” for this model, so greater PPV for the PSWQ (85.8%) than NPV (77.6%) indicates better classification accuracy for MDD than GAD cases.

Table 5.

Disorder Classification Accuracy Metrics for Each Measure

Measure Accuracy Balanced Accuracy Sensitivity Specificity PPV NPV Kappa
PSWQ 81.1% 81.2% 74.8% 87.6% 85.8% 77.6% 0.622
MASQ-AA 67.3% 67.2% 79.2% 55.5% 63.9% 72.6% 0.341
MASQ-AD 66.0% 66.0% 68.8% 63.2% 65.2% 66.9% 0.319
MASQ-AD14 52.9% 53.0% 56.0% 50.0% 52.8% 53.2% 0.068
MASQ-AD8 56.0% 55.9% 62.4% 49.5% 55.3% 56.8% 0.121
Open in a new tab

Note. Accuracy, balanced accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and Cohen’s Kappa for the Penn State Worry Questionnaire (PSWQ), Mood and Anxiety Symptom Questionnaire (MASQ) Anxious Arousal Scale (MASQ-AA), MASQ Anhedonic Depression Scale (MASQ-AD), MASQ-AD 14-item subscale (MASQ-AD14), and MASQ-AD 8-item subscale (MASQ-AD8) for disorder classification. MDD was coded as “1” and GAD was coded as “0” for this model.

4. Discussion

This study tested the utility of three symptom questionnaires, the PSWQ, MASQ-AA, and MASQ-AD for screening for and distinguishing between clinical diagnoses of MDD and GAD. According to the tripartite model, we expected MASQ-AD to be useful in screening for MDD, and we expected the PSWQ to be useful in screening for GAD and separating GAD cases from MDD cases (Meyer et al., 1990; Watson & Clark, 1991). We did not expect the MASQ-AA to meaningfully predict MDD or GAD diagnoses. In the present study, the PSWQ demonstrated effectiveness for detection of GAD and distinguishing GAD from MDD, however the MASQ-AD was only effective in detecting current major depressive episodes rather than lifetime major depressive disorder diagnoses. In addition, the MASQ-AD8 subscale, measuring depressive symptoms, outperformed the MASQ-AD14 subscale, measuring positive emotionality, in detecting current major depressive episodes. As expected, the MASQ-AA did not demonstrate accuracy in predicting MDD or GAD diagnoses. Results indicate that the PSWQ has predictive utility for GAD diagnoses, the MASQ-AD has predictive utility for current MDE but not lifetime MDD, and the MASQ-AA does not have predictive utility for MDD nor GAD.

Overall, the results of this study are consistent with past literature. First, the PSWQ demonstrated predictive utility for both GAD detection and distinguishing GAD from MDD, which is consistent with past literature showing the PSWQ to be an effective screening measure for GAD (Behar et al., 2003; Brown et al., 1992; Fresco et al., 2003). The MASQ-AD demonstrated predictive utility for detection of current major depressive episodes, which is consistent with past literature showing the MASQ-AD scale to be sensitive to depressive disorders, particularly current depressive disorders (Bredemeier et al, 2010; Buckby et al., 2007). In addition, the MASQ-AD8 outperforming the MASQ-AD14 in current MDE detection is consistent with past studies on these two subscales (Bredemeier et al., 2010). The MASQ-AA did not demonstrate predictive utility for MDD, current MDE, or GAD. Somewhat inconsistent with past literature, neither the MASQ-AD nor its two subscales demonstrated predictive utility in the MDD detection use-case which included both current and lifetime MDD diagnoses, whereas Bredemeier et al. (2010) found that the MASQ-AD and MASQ-AD14 predicted lifetime MDD at a level above chance though not strongly.

Though the sensitivity of the MASQ-AD scale to current MDE detection is consistent with the tripartite model’s prediction that anhedonic depression is a unique feature of depressive disorders (Watson, 2005), the finding that the MASQ-AA scale did not demonstrate predictive utility for GAD is worth further consideration. The lack of association between anxious arousal and GAD may be explained by the distinction between two separate dimensions of anxiety, anxious arousal and anxious apprehension (Nitschke et al., 2001, Sharp et al., 2015). Anxious apprehension, represented by worry and measured with the PSWQ, was found to be highly relevant to GAD. In addition, anxious arousal and anxious apprehension are associated with distinct neuroanatomical correlates (Castagna et al., 2018), differences in task-related and resting-state brain activity (e.g., Burdwood et al., 2016; Engels et al., 2007; Engels et al., 2010; Warren et al., 2013), and differences in executive function (Warren et al., 2021). Therefore, the tripartite model may benefit from expanding to include anxious apprehension in order to capture this distinct dimension of anxiety and better account for GAD and other affective disorders.

4.1. Clinical Implications

Based on the results of the present study, the PSWQ and MASQ-AD represent efficient and useful measures for screening for and identifying patients with current MDE and GAD. Full clinical interviews may be an inefficient strategy in settings where the identification of patients who are at risk for or may have psychopathology within a large patient pool is the primary goal, such as primary care settings (O’Connor et al., 2009). In these cases, the use of symptom questionnaires like the PSWQ and MASQ-AD are useful for screening for patients who would likely benefit from more comprehensive evaluation, thus increasing the efficiency by which patients who might benefit from psychotherapeutic or psychopharmacological interventions can be identified and treated. Research studies focusing on specific psychiatric disorders can benefit in a similar way; questionnaires can be used to screen for potential participants who might meet criteria for the disorder of interest, allowing for increased efficiency of participant recruitment (Bredemeier et al., 2010). In both cases, machine learning models, such as the models used in this study, can use data-driven algorithms to maximize accuracy of predictions made from item-level data, representing an alternative to traditional ROC curve approaches utilizing aggregate scores (Gonzalez, 2020). However, it remains to be seen how the traditional approach using ROC curve-derived optimal cut-off scores or machine learning approaches compare in performance for screening for psychiatric disorders as well as whether one approach might be preferable in certain situations.

4.2. Limitations and Future Directions

Though the present study was able to test the efficacy of the PSWQ, MASQ-AA, MASQ-AD in screening for and identifying MDD and GAD, a larger sample size, particularly in our psychopathology groups, would maximize robustness of findings. It is unclear whether the present study’s findings are specific to the questionnaires tested or whether they apply at a broader construct level, meaning measures of the same constructs would demonstrate similar screening or classification accuracy. As a related concern, while the present study used internal cross-validation to test questionnaires’ performance, external validation, in the form of training machine learning algorithms on one dataset and testing them on a separate unrelated dataset, is needed to ensure generalizability of the present findings (Jiang et al., 2020). Finally, the present study’s diagnostic groups differed demographically, specifically in proportion of gender, creating a potential confounding factor. However, studies have supported measurement invariance across gender for the PSWQ (Brown. 2003; Carter & Bates, 2013), while other studies have demonstrated that scores on MASQ subscales do not differ by gender (Boschen & Oei, 2007), mitigating this concern.

Future work might examine the utility of these questionnaires in predicting other diagnoses. For example, the MASQ-AA has demonstrated predicative utility with social phobia, agoraphobia, and panic disorder (Williamson et al., 2005). Additionally, while binary support vector machines are a supervised machine learning method that can be utilized when outcomes are known, clinical diagnoses in this case, unsupervised machine learning methods, which seek to uncover underlying dimensions, components, or clusters that are unknown, might be used to identify specific sub-types or dimensions of psychopathology based on questionnaire data (Jiang et al., 2020). Future work might also examine the utility of questionnaires to predict future psychiatric diagnoses or clinically relevant outcomes as has been done with predicting suicide risk (Walsh et al., 2018). Finally, different sources of data, including performance testing, genetic data, neuroimaging, and hormone levels, can be explored and combined with questionnaire data in using machine learning models to predict psychiatric diagnoses or other outcomes as doing so may lead to greater predictive accuracy (Hilbert et al., 2017).

4.3. Conclusion

In summary, the findings from the present study support the use of the PSWQ and MASQ-AD scales for screening for and identifying patients with current MDE or GAD, though not lifetime MDD. These results are also generally supportive of the tripartite model of depression and anxiety disorders as well as the distinction between anxious arousal and anxious apprehension. Both clinicians and researchers might use the PSWQ and MASQ-AD to increase efficiency of screening procedures to identify patients or participants who might meet criteria for a psychiatric disorder.

Supplementary Material

1

Highlights.

  • Machine learning predicted psychopathology diagnoses using symptom questionnaires

  • A measure of anxious apprehension was effective in detection of generalized anxiety

  • Anxious apprehension distinguished generalized anxiety from major depression

  • A measure of anhedonia was effective in detecting current major depressive episodes

  • A measure of anxious arousal did not meaningfully predict generalized anxiety

ACKNOWLEDGEMENTS

This research was supported by the National Institutes of Health NIDA R21 DA14111 and NIMH R01 MH61358, and the Palo Alto University Department of Psychology. The authors wish to thank Drs. Gregory Miller and Wendy Heller for their data contributions to the work reported here.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

CRediT Author Statement

Liu: Conceptualization, Methodology, Software, Formal Analysis, Writing – Original Draft and subsequent Revision. Droncheff: Conceptualization, Writing – Review and Editing of original and revised drafts. Warren: Conceptualization, Investigation, Resources, Data Curation, Writing – Original Draft and Revision, Supervision, Project Administration, Funding Acquisition.

References

  1. Avery DH, Khan A, Dager SR, Cox GB, & Dunner DL (1990). Bright light treatment of winter depression: Morning versus evening light. Acta Psychiatrica Scandinavica, 82(5), 335–338. 10.1111/j.1600-0447.1990.tb01397.x [DOI] [PubMed] [Google Scholar]
  2. Beck AT, Epstein N, Brown G, & Steer RA (1988). An inventory for measuring clinical anxiety: Psychometric properties. Journal of Consulting and Clinical Psychology, 56(6), 893–897. 10.1037/0022-006X.56.6.893 [DOI] [PubMed] [Google Scholar]
  3. Beck AT, Steer RA, & Brown G (1996). Beck depression inventory–II. Psychological Assessment
  4. Beesdo K, Knappe S, & Pine DS (2009). Anxiety and anxiety disorders in children and adolescents: Developmental issues and implications for DSM-V. The Psychiatric Clinics of North America, 32(3), 483–524. PubMed. 10.1016/j.psc.2009.06.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Boschen MJ, & Oei TPS (2007). Discriminant validity of the MASQ in a clinical sample. Psychiatry Research, 150(2), 163–171. 10.1016/j.psychres.2006.03.008 [DOI] [PubMed] [Google Scholar]
  6. Bredemeier K, Spielberg JM, Silton RL, Berenbaum H, Heller W, & Miller GA (2010). Screening for depressive disorders using the Mood and Anxiety Symptoms Questionnaire Anhedonic Depression Scale: A receiver-operating characteristic analysis. Psychological Assessment, 22(3), 702–710. 10.1037/a0019915 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bredemeier K, Warren SL, Berenbaum H, Miller GA, & Heller W (2016). Executive function deficits associated with current and past major depressive symptoms. Journal of Affective Disorders, 204, 226–233. 10.1016/j.jad.2016.03.070 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Brown TA, Antony MM, & Barlow DH (1992). Psychometric properties of the Penn state worry questionnaire in a clinical anxiety disorders sample. Behaviour Research and Therapy, 30(1), 33–37. 10.1016/0005-7967(92)90093-V [DOI] [PubMed] [Google Scholar]
  9. Brown TA (2003). Confirmatory factor analysis of the Penn State Worry Questionnaire: Multiple factors or method effects? Behaviour Research and Therapy, 41(12), 1411–1426. 10.1016/S0005-7967(03)00059-7 [DOI] [PubMed] [Google Scholar]
  10. Burdwood EN, Infantolino ZP, Crocker LD, Spielberg JM, Banich MT, Miller GA, & Heller W (2016). Resting-state functional connectivity differentiates anxious apprehension and anxious arousal. Psychophysiology, 53(10), 1451–1459. PubMed. 10.1111/psyp.12696 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Calleo J, Stanley MA, Greisinger A, Wehmanen O, Johnson M, Novy D, Wilson N, & Kunik M (2009). Generalized anxiety disorder in older medical patients: Diagnostic recognition, mental health management and service utilization. Journal of Clinical Psychology in Medical Settings, 16(2), 178–185. 10.1007/s10880-008-9144-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Canu E, Kostić M, Agosta F, Munjiza A, Ferraro PM, Pesic D, Copetti M, Peljto A, Lecic Tosevski D, & Filippi M (2015). Brain structural abnormalities in patients with major depression with or without generalized anxiety disorder comorbidity. Journal of Neurology, 262(5), 1255–1265. 10.1007/s00415-015-7701-z [DOI] [PubMed] [Google Scholar]
  13. Carleton RN, Norton MAPJ, & Asmundson GJG (2007). Fearing the unknown: A short version of the Intolerance of Uncertainty Scale. Journal of Anxiety Disorders, 21(1), 105–117. 10.1016/j.janxdis.2006.03.014 [DOI] [PubMed] [Google Scholar]
  14. Carter JA, & Bates SC (2013). The Penn State Worry Questionnaire: Comparative model fit and invariance across gender. In National Forum Journal Of Counseling And Addiction (Vol. 2, No. 1, pp. 1–25). [Google Scholar]
  15. Carter RM, Wittchen H-U, Pfister H, & Kessler RC (2001). One-year prevalence of subthreshold and threshold DSM-IV generalized anxiety disorder in a nationally representative sample. Depression and Anxiety, 13(2), 78–88. 10.1002/da.1020 [DOI] [PubMed] [Google Scholar]
  16. Castagna PJ, Roye S, Calamia M, Owens-French J, Davis TE, & Greening SG (2018). Parsing the neural correlates of anxious apprehension and anxious arousal in the grey-matter of healthy youth. Brain Imaging and Behavior, 12(4), 1084–1098. 10.1007/s11682-017-9772-1 [DOI] [PubMed] [Google Scholar]
  17. Clark LA, & Watson D (1991). Tripartite model of anxiety and depression: Psychometric evidence and taxonomic implications. Journal of Abnormal Psychology, 100(3), 316–336. 10.1037/0021-843X.100.3.316 [DOI] [PubMed] [Google Scholar]
  18. Creamer M, Foran J, & Bell R (1995). The Beck Anxiety Inventory in a non-clinical sample. Behaviour Research and Therapy, 33(4), 477–485. 10.1016/0005-7967(94)00082-U [DOI] [PubMed] [Google Scholar]
  19. Curtiss J, & Klemanski DH (2015). Identifying individuals with 29eneralized anxiety disorder: A receiver operator characteristic analysis of theoretically relevant measures. Behaviour Change, 32(4), 255–272. Psychology Database. [Google Scholar]
  20. Engels AS, Heller W, Mohanty A, Herrington JD, Banich MT, Webb AG, & Miller GA (2007). Specificity of regional brain activity in anxiety types during emotion processing. Psychophysiology, 44(3), 352–363. 10.1111/j.1469-8986.2007.00518.x [DOI] [PubMed] [Google Scholar]
  21. Engels AS, Heller W, Spielberg JM, Warren SL, Sutton BP, Banich MT, & Miller GA (2010). Co-occurring anxiety influences patterns of brain activity in depression. Cognitive, Affective & Behavioral Neuroscience, 10(1), 141–156. 10.3758/CABN.10.1.141 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Feldman LA (1993). Distinguishing depression and anxiety in self-report: Evidence from confirmatory factor analysis on nonclinical and clinical samples. Journal of Consulting and Clinical Psychology, 61(4), 631–638. 10.1037/0022-006X.61.4.631 [DOI] [PubMed] [Google Scholar]
  23. First M, Spitzer R, Gibbon M, & Williams J (2002). Structured clinical interview for DSM-IV-TR Axis I Disorders, Research Version, Non-patient Edition. In (SCID-I/P)
  24. Fischer EH, & Goethe JW (1997). Measurement of depression and anxiety for hospitalized depressed patients. Psychiatric Services (Washington, D.C.), 48(5), 705–707. 10.1176/ps.48.5.705 [DOI] [PubMed] [Google Scholar]
  25. Fresco DM, Mennin DS, Heimberg RG, & Turk CL (2003). Using the Penn State Worry Questionnaire to identify individuals with generalized anxiety disorder: A receiver operating characteristic analysis. Journal of Behavior Therapy and Experimental Psychiatry, 34(3), 283–291. 10.1016/j.jbtep.2003.09.001 [DOI] [PubMed] [Google Scholar]
  26. Geller WN, Liu K & Warren SL (2021). Specificity of anhedonic alterations in resting-state network connectivity and structure: A transdiagnostic approach. Psychiatry Research: Neuroimaging, 317, 111349. 10.1016/j.pscychresns.2021.111349 [DOI] [PubMed] [Google Scholar]
  27. Gonzalez O (2020). Psychometric and machine learning approaches for diagnostic assessment and tests of individual classification. Psychological Methods. Advance online publication 10.1037/met0000317 [DOI] [PubMed]
  28. Gotlib IH, & Cane DB (1989). Self-report assessment of depression and anxiety. Anxiety and Depression: Distinctive and Overlapping Features, 131–169.
  29. Hasin DS, Sarvet AL, Meyers JL, Saha TD, Ruan WJ, Stohl M, & Grant BF (2018). Epidemiology of adult DSM-5 major depressive disorder and its specifiers in the United States. JAMA Psychiatry, 75(4), 336–346. 10.1001/jamapsychiatry.2017.4602 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Hek K, Demirkan A, Lahti J, Terracciano A, Teumer A, Cornelis MC, Amin N, Bakshis E, Baumert J, Ding J, Liu Y, Marciante K, Meirelles O, Nalls MA, Sun YV, Vogelzangs N, Yu L, Bandinelli S, Benjamin EJ, … Murabito J (2013). A genome-wide association study of depressive symptoms. Biological Psychiatry, 73(7), 667–678. 10.1016/j.biopsych.2012.09.033 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Heller W, Nitschke JB, Etienne MA, & Miller GA (1997). Patterns of regional brain activity differentiate types of anxiety. Journal of Abnormal Psychology, 106(3), 376–385. 10.1037//0021-843x.106.3.376 [DOI] [PubMed] [Google Scholar]
  32. Hilbert K, Lueken U, Muehlhan M, & Beesdo-Baum K (2017). Separating generalized anxiety disorder from major depression using clinical, hormonal, and structural MRI data: A multimodal machine learning study. Brain and Behavior, 7(3), e00633. 10.1002/brb3.633 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Hinkelmann K, Moritz S, Botzenhardt J, Muhtz C, Wiedemann K, Kellner M, & Otte C (2012). Changes in cortisol secretion during antidepressive treatment and cognitive improvement in patients with major depression: A longitudinal study. Psychoneuroendocrinology, 37(5), 685–692. 10.1016/j.psyneuen.2011.08.012 [DOI] [PubMed] [Google Scholar]
  34. Jiang T, Gradus JL, & Rosellini AJ (2020). Supervised machine learning: A brief primer. Behavior Therapy, 51(5), 675–687. 10.1016/j.beth.2020.05.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Kendall PC, & Watson D (Eds.). (1989). Anxiety and depression: Distinctive and overlapping features. Anxiety and Depression: Distinctive and Overlapping Features, xviii, 519–xviii, 519. [Google Scholar]
  36. Keogh E, & Reidy J (2000). Exploring the factor structure of the Mood and Anxiety Symptom Questionnaire (MASQ). Journal of Personality Assessment, 74(1), 106–125. 10.1207/S15327752JPA740108 [DOI] [PubMed] [Google Scholar]
  37. Kertz S, Bigda‐Peyton J, & Bjorgvinsson T (2013). Validity of the Generalized Anxiety Disorder-7 scale in an acute psychiatric sample. Clinical Psychology & Psychotherapy, 20(5), 456–464. Psychology and Behavioral Sciences Collection. [DOI] [PubMed] [Google Scholar]
  38. Kessler RC, Chiu WT, Demler O, Merikangas KR, & Walters EE (2005). Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62(6), 617–627. 10.1001/archpsyc.62.6.617 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Kessler RC, Petukhova M, Sampson NA, Zaslavsky AM, & Wittchen H-U (2012). Twelve-month and lifetime prevalence and lifetime morbid risk of anxiety and mood disorders in the United States. International Journal of Methods in Psychiatric Research, 21(3), 169–184. 10.1002/mpr.1359 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Liu K, Nijmeh JS, & Warren SL (2021). Factor structure, measurement invariance, and concurrent validity of the Penn State Worry Questionnaire across development, psychopathology, and culture. Assessment, 1073191121993223. 10.1177/1073191121993223 [DOI] [PubMed]
  41. Madian N, Bredemeier K, Heller W, Miller GA, & Warren SL (2019). Repetitive negative thought and executive dysfunction: An interactive pathway to emotional distress. Cognitive Therapy and Research, 43(2), 464–480. doi: 10.1007/s10608-018-9966-x [DOI] [Google Scholar]
  42. Meyer TJ, Miller ML, Metzger RL, & Borkovec TD (1990). Development and validation of the Penn State Worry Questionnaire. Behaviour Research and Therapy, 28(6), 487–495. 10.1016/0005-7967(90)90135-6 [DOI] [PubMed] [Google Scholar]
  43. Miller GA, & Rockstroh B (2013). Endophenotypes in psychopathology research: Where do we stand? Annual Review of Clinical Psychology, 9(1), 177–213. 10.1146/annurev-clinpsy-050212-185540 [DOI] [PubMed] [Google Scholar]
  44. Nitschke JB, Heller W, Palmieri PA, & Miller GA (1999). Contrasting patterns of brain activity in anxious apprehension and anxious arousal. Psychophysiology, 36(5), 628–637. [PubMed] [Google Scholar]
  45. Nitschke Jack B., Heller W, Imig JC, McDonald RP, & Miller GA (2001). Distinguishing dimensions of anxiety and depression. Cognitive Therapy and Research, 25(1), 1–22. 10.1023/A:1026485530405 [DOI] [Google Scholar]
  46. Oathes DJ, Hilt LM, & Nitschke JB (2015). Affective neural responses modulated by serotonin transporter genotype in clinical anxiety and depression. PLOS ONE, 10(2), e0115820. 10.1371/journal.pone.0115820 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. O’Connor EA, Whitlock EP, Beil TL, & Gaynes BN (2009). Screening for depression in adult patients in primary care settings: A systematic evidence review. Annals of Internal Medicine, 151(11), 793–803. 10.7326/0003-4819-151-11-200912010-00007 [DOI] [PubMed] [Google Scholar]
  48. Plummer F, Manea L, Trepel D, & McMillan D (2016). Screening for anxiety disorders with the GAD-7 and GAD-2: A systematic review and diagnostic metaanalysis. General Hospital Psychiatry, 39, 24–31. 10.1016/j.genhosppsych.2015.11.005 [DOI] [PubMed] [Google Scholar]
  49. Regier DA, Narrow WE, Clarke DE, Kraemer HC, Kuramoto SJ, Kuhl EA, & Kupfer DJ (2013). DSM-5 field trials in the United States and Canada, part II: Test-retest reliability of selected categorical diagnoses. American Journal of Psychiatry, 170(1), 59–70. 10.1176/appi.ajp.2012.12070999 [DOI] [PubMed] [Google Scholar]
  50. Reidy J, & Keogh E (1997). Testing the discriminant and convergent validity of the mood and anxiety symptoms questionnaire using a British sample. Personality and Individual Differences, 23(2), 337–344. 10.1016/S0191-8869(97)00048-2 [DOI] [Google Scholar]
  51. Rutter LA, & Brown TA (2017). Psychometric properties of the Generalized Anxiety Disorder Scale-7 (GAD-7) in outpatients with anxiety and mood disorders. Journal of Psychopathology and Behavioral Assessment, 39(1), 140–146. PubMed. 10.1007/s10862-016-9571-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Segal DL, Coolidge FL, Cahil BS, & O’Riley AA (2008). Psychometric properties of the Beck Depression Inventory II (BDI-II) among community-dwelling older adults. Behavior Modification, 32(1), 3–20. 10.1177/0145445507303833 [DOI] [PubMed] [Google Scholar]
  53. Sharp PB, Miller GA, & Heller W (2015). Transdiagnostic dimensions of anxiety: Neural mechanisms, executive functions, and new directions. Psychophysiological Science and the Research Domain Criteria, 98(2, Part 2), 365–377. 10.1016/j.ijpsycho.2015.07.001 [DOI] [PubMed] [Google Scholar]
  54. Spitzer RL, Kroenke K, Williams JBW, & Löwe B (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092–1097. 10.1001/archinte.166.10.1092 [DOI] [PubMed] [Google Scholar]
  55. Stöber J (1998). Reliability and validity of two widely-used worry questionnaires: Self-report and self-peer convergence. Personality and Individual Differences, 24(6), 887–890. 10.1016/S0191-8869(97)00232-8 [DOI] [Google Scholar]
  56. Stulz N, & Crits‐Christoph P (2010). Distinguishing anxiety and depression in self-report: Purification of the beck anxiety inventory and beck depression inventory-II. Journal of Clinical Psychology, 66(9), 927–940. Psychology and Behavioral Sciences Collection. [DOI] [PubMed] [Google Scholar]
  57. Tennenhouse LG, Marrie RA, Bernstein CN, & Lix LM (2020). Machine-learning models for depression and anxiety in individuals with immune-mediated inflammatory disease. Journal of Psychosomatic Research, 134, 110126. 10.1016/j.jpsychores.2020.110126 [DOI] [PubMed] [Google Scholar]
  58. van Buuren S, & Groothuis-Oudshoorn K (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software; Vol 1, Issue 3 (2011). 10.18637/jss.v045.i03 [DOI] [Google Scholar]
  59. Warren SL, Crocker LD, Spielberg JM, Engels AS, Banich MT, Sutton BP, Miller GA, & Heller W (2013). Cortical organization of inhibition-related functions and modulation by psychopathology. Frontiers in Human Neuroscience, 7, 271. 10.3389/fnhum.2013.00271 [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Warren SL, Heller W, & Miller GA (2021). The structure of executive dysfunction in depression and anxiety. Journal of Affective Disorders, 279, 208–216. 10.1016/j.jad.2020.09.132 [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Watson D, Clark LA, Weber K, Assenheimer JS, Strauss ME, & McCormick RA (1995). Testing a tripartite model: II. Exploring the symptom structure of anxiety and depression in student, adult, and patient samples. Journal of Abnormal Psychology, 104(1), 15–25. 10.1037//0021-843x.104.1.15 [DOI] [PubMed] [Google Scholar]
  62. Watson D, Weber K, Assenheimer JS, Clark LA, Strauss ME, & McCormick RA (1995). Testing a tripartite model: I. Evaluating the convergent and discriminant validity of anxiety and depression symptom scales. Journal of Abnormal Psychology, 104(1), 3–14. 10.1037//0021-843x.104.1.3 [DOI] [PubMed] [Google Scholar]
  63. Williamson RJ, Neale BM, Sterne A, Prince M, & Sham P (2005). The value of four mental health self-report scales in predicting interview-based mood and anxiety disorder diagnoses in sibling pairs. Twin Research and Human Genetics : The Official Journal of the International Society for Twin Studies, 8(2), 101–107. 10.1375/1832427053738773 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1796872-supplement-1.docx (33.5KB, docx)

RESOURCES