TABLE 4.
Classifier performance for identifying individual disorders. The number of features used for classification was a parameter tuned separately from the other hyperparameters. The AROC is the Area under the Receiver Operating Characteristic curve. The P-values represent the significance of the AROCs compared to a completely random classifier. (a) Leave-one-out (LOO) cross-validation performance using the first interviews with all participants. The feature selection for the cross-validation was performed within each fold; however, the “top features” presented here use the same method applied to the entire dataset. (b) Training sets constructed from stratified random samples of 50% of first interviews and 50% of second interviews with performance tested against all remaining interviews
| (a) | ||||
|---|---|---|---|---|
| Suicidality (n = 30/122) | Despressive Disorders s (n = 29/122) | Anxiety Disorders (n = 15/122) | Bipolar Disorders (n = 6/122) | |
| AROC (95% CI) | 57 (44-71)% | 65 (54-76)% | 66 (52-80)% | 78 (64-92)% |
| P-value | .22 | .011 | .04 | .003 |
| Total No. Features | 16 | 512 | 32 | 64 |
| Top 5 features for presence of comorbidity (SVM weights), | try to, sometimes, things, I do not really, my | its, is, yes, like i, hope | I feel, yes, myself, my, try to | it is, be, weird, now, things |
| Top 5 features for absence of comorbidity (SVM weights) | I have, in, try, assent (LIWC), inhibit (LIWC) | fears, if i, on, get, know | no, and I, to, that I, feel | get, angry, for, think, so |
| (b) | ||||
| Suicidality (n = 26/114) | Depressive Disorders (n = 25/113) | Anxiety Disorders (n = 12/112) | Bipolar Disorders (n = 4/114) | |
| AROC (95% CI) | 71 (59-83)% | 69 (58-81)% | 64 (46-81)% | 66 (34-99)% |
| P-value | .0011 | .001 | .12 | .3 |
| Total No. Features | 64 | 128 | 256 | 32 |
| Top 5 features for presence of comorbidity (SVM weights) | like a, fear, job, things, always | uh, like that, in, that i, was | kinda, going, no, more, you |
time, i just, or, pretty, thats |
| Top 5 features for absence of comorbidity (SVM weights) | wrong, a lot, makes me, feel like, really | i am, she, at, talk to, come | myself, most, lot of, a lot of, of | secrets, give, anything to, to explain, thats really |