TABLE 3.

Classifier performance for identifying three distinct groups: suicidality, non-suicidal psychiatric disorders, and no psychiatric disorders. The number of features used for classification was a parameter tuned separately from the C and γ SVM kernel hyperparameters. The AROC is the Area under the Receiver Operating Characteristic curve. The P-values represent the significance of the AROCs compared to a completely random classifier. “First group vs second group” describes, for example, “Suicidality vs No disorder.” “LIWC” indicates features that represent LIWC word categories. (a) Leave-one-out (LOO) cross-validation performance using the first interviews with all participants. The feature selection for the cross-validation was performed within each fold; however, the “top features” presented here use the same method applied to the entire dataset. (b) Training sets constructed from stratified random samples of 50% of first interviews and 50% of second interviews with performance tested against all remaining interviews

(a)
	Suicidality (n = 30) vs No disorder (n = 75)	Suicidality (n = 27) vs Non-suicidal disorder (n = 16)	Suicidality (n = 30) vs No disorder + Non-suicidal disorder (n = 92)	Non-suicidal disorder (n = 17) vs No disorder (n = 75)
AROC (95% CI)	72 (61-83)%	71 (55-89)%	57 (44-71)%	73 (62-85)%
P-value	.00018	.015	.22	<.0001
Total No. Features	16	256	16	64
Top 5 features for first group (SVM weights)	out, other, its, how, in a	some, them, from, in a, things	try to, sometimes, things, I do not really, my	or, can, you know, do not i, if
Top 5 features for second group (SVM weights)	no, things, people, my, it	okay, is, had, i guess, guess	I have, in, try, assent (LIWC), inhibit (LIWC)	i guess, guess, better, i do, though
(b)
	Suicidality (n = 25) vs No disorder (n = 73)	Suicidality (n = 26) vs Non-suicidal disorder (n = 14)	Suicidality (n = 26) vs No disorder + Non-suicidal disorder (n = 88)	Non-suicidal disorder (n = 15) vs No disorder (n = 73)
AROC (95% CI)	71 (59-83)%	67 (48-87)%	71 (59-83)%	64 (48-79)%
P-value	.003	.09	.0011	.09
Total No. Features	512	128	64	512
Top 5 features for first group (SVM weights)	little, for, have a, how, now	been, its, think of, fear, i guess	like a, fear, job, things, always	you know, you, thats, probably, them
Top 5 features for second group (SVM weights)	little, for, have a, how, now	sometimes, things, you know, you, they are	wrong, a lot, makes me, feel like, really	better, it like, being angry, right, i would