Table 1. Accuracy (ACC), AUC, false positive rate (FPR), and false negative rate (FNR) are obtained using supervised (logistic regression) and unsupervised (Gaussian mixture modeling) classification.
All metrics are based on a probability of 0.5 classification threshold. Columns contain tables for clustering using the first canonical correlation variable and first n = 1, 2 principle components from the 9 questionnaires. Raw ordinal scores are used for comparison. Results are grouped into two column groups: the left group contains clustering comparing C vs. CS, the left group shows CC vs. S. Results for logistic regression are the mean of a 10-fold across-validation analysis.
Supervised (logistic regression) | ||||||||
C vs. CS | ||||||||
CCV | 1 PC | 2 PCs | ||||||
Embeddings | Raw Scores | Embeddings | Raw Scores | Embeddings | Raw Scores | |||
ACC | 0.85 (0.8,0.89) | 0.73 (0.66,0.78) | 0.83 (0.78,0.87) | 0.83 (0.78,0.87) | 0.86 (0.81,0.91) | 0.85 (0.81,0.9) | ||
AUC | 0.91 (0.88,0.94) | 0.75 (0.69,0.81) | 0.89 (0.85,0.93) | 0.89 (0.85,0.93) | 0.9 (0.86,0.94) | 0.9 (0.86,0.94) | ||
FPR | 0.13 (0.08,0.19) | 0.16 (0.09,0.25) | 0.16 (0.1,0.22) | 0.15 (0.1,0.21) | 0.12 (0.07,0.18) | 0.13 (0.08,0.18) | ||
FNR | 0.18 (0.11,0.26) | 0.48 (0.34,0.65) | 0.19 (0.12,0.27) | 0.2 (0.12,0.29) | 0.17 (0.09,0.26) | 0.18 (0.11,0.27) | ||
Supervised (logistic regression) | ||||||||
CC vs. S | ||||||||
CCV | 1 PC | 2 PCs | ||||||
Embeddings | Raw Scores | Embeddings | Raw Scores | Embeddings | Raw Scores | |||
ACC | 0.85 (0.8,0.89) | 0.77 (0.72,0.82) | 0.85 (0.8,0.89) | 0.8 (0.7,0.87) | 0.86 (0.81,0.91) | 0.8 (0.7,0.88) | ||
AUC | 0.89 (0.85,0.93) | 0.76 (0.7,0.82) | 0.89 (0.84,0.93) | 0.83 (0.78,0.88) | 0.89 (0.85,0.93) | 0.83 (0.78,0.88) | ||
FPR | 0.11 (0.07,0.15) | 0.08 (0.03,0.13) | 0.11 (0.07,0.15) | 0.04 (0,0.1) | 0.09 (0.05,0.13) | 0.05 (0,0.12) | ||
FNR | 0.29 (0.16,0.46) | 0.65 (0.46,0.85) | 0.28 (0.17,0.45) | 0.62 (0.27,1) | 0.26 (0.15,0.39) | 0.59 (0.22,1) | ||
Unsupervised (Gaussian mixture model) | ||||||||
C vs. CS | ||||||||
CCV | 1 PC | 2 PCs | ||||||
Embeddings | Raw Scores | Embeddings | Raw Scores | Embeddings | Raw Scores | |||
ACC | 0.81 (0.75,0.86) | 0.7 (0.53,0.79) | 0.81 (0.74,0.86) | 0.79 (0.71,0.84) | 0.81 (0.75,0.86) | 0.77 (0.69,0.83) | ||
AUC | 0.84 (0.71,0.92) | 0.75 (0.58,0.93) | 0.84 (0.72,0.92) | 0.83 (0.69,0.9) | 0.83 (0.63,0.92) | 0.83 (0.74,0.89) | ||
FPR | 0.24 (0.11,0.36) | 0.33 (0.1,0.98) | 0.25 (0.14,0.4) | 0.27 (0.1,0.46) | 0.24 (0.1,0.47) | 0.2 (0.09,0.38) | ||
FNR | 0.16 (0.08,0.28) | 0.29 (0.1,0.63) | 0.16 (0.06,0.29) | 0.18 (0.06,0.360 | 0.15 (0.04,0.3) | 0.26 (0.09,0.39) | ||
Unsupervised (Gaussian mixture model) | ||||||||
C vs. CS | ||||||||
CCV | 1 PC | 2 PCs | ||||||
Embeddings | Raw Scores | Embeddings | Raw Scores | Embeddings | Raw Scores | |||
ACC | 0.79 (0.72,0.85) | 0.7 (0.53,0.8) | 0.79 (0.71,0.86) | 0.79 (0.69,0.86) | 0.8 (0.71,0.86) | 0.76 (0.67,0.84) | ||
AUC | 0.84 (0.73,0.91) | 0.76 (0.59,0.94) | 0.85 (0.74,0.91) | 0.85 (0.73,0.91) | 0.83 (0.65,0.91) | 0.85 (0.78,0.9) | ||
FPR | 0.19 (0.06,0.32) | 0.3 (0.06,0.99) | 0.19 (0.07,0.34) | 0.19 (0.05,0.37) | 0.18 (0.04,0.42) | 0.11 (0.03,0.33) | ||
FNR | 0.22 (0.13,0.34) | 0.3 (0.12,0.55) | 0.21 (0.11,0.34) | 0.22 (0.1,0.4) | 0.21 (0.08,0.36) | 0.29 (0.12,0.42) |