Table 2.
Performance metrics for nonlinear simulations across four scenarios.
|
Setting1 (n1 = 200, n2 = 150, p1 = 500, p2 = 500) |
Setting2 (n1 = 3000, n2 = 2250, p1 = 500, p2 = 500) |
|||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Method | Accuracy | AUROC | F1 | Precision | Recall | Accuracy | AUROC | F1 | Precision | Recall |
| Deep IDA on top 50 selected features view 1 | 0.82(0.19) | 0.81(0.21) | 0.86(0.15) | 0.85(0.18) | 0.90(0.15) | 0.93(0.07) | 0.93(0.07) | 0.94(0.06) | 0.93(0.07) | 0.96(0.05) |
| Deep IDA on top 50 selected features | 0.63(0.10) | 0.61(0.11) | 0.70(0.10) | 0.66(0.08) | 0.75(0.16) | 0.91(0.07) | 0.90(0.08) | 0.92(0.06) | 0.91(0.08) | 0.93(0.06) |
| Deep IDA view 1 | 0.86(0.09) | 0.85(0.09) | 0.88(0.07) | 0.86(0.08) | 0.90(0.07) | 0.94(0.06) | 0.94(0.06) | 0.94(0.05) | 0.96(0.05) | 0.93(0.08) |
| Deep IDA | 0.60(0.07) | 0.59(0.07) | 0.65(0.10) | 0.66(0.08) | 0.67(0.17) | 0.85(0.07) | 0.85(0.07) | 0.87(0.06) | 0.88(0.07) | 0.86(0.07) |
| Deep CCA on top 50 selected features view 1 | 0.61(0.05) | 0.61(0.05) | 0.62(0.05) | 0.69(0.05) | 0.57(0.05) | 0.57(0.03) | 0.58(0.03) | 0.59(0.03) | 0.65(0.03) | 0.55(0.03) |
| Deep CCA on top 50 selected features | 0.58(0.04) | 0.59(0.04) | 0.60(0.05) | 0.66(0.04) | 0.56(0.06) | 0.57(0.03) | 0.58(0.03) | 0.60(0.03) | 0.65(0.03) | 0.55(0.02) |
| Deep CCA view 1 | 0.59(0.04) | 0.60(0.04) | 0.61(0.04) | 0.67(0.04) | 0.56(0.05) | 0.61(0.02) | 0.61(0.02) | 0.63(0.02) | 0.68(0.02) | 0.59(0.02) |
| Deep CCA | 0.57(0.03) | 0.58(0.03) | 0.60(0.03) | 0.65(0.04) | 0.57(0.04) | 0.60(0.02) | 0.60(0.02) | 0.62(0.02) | 0.67(0.02) | 0.58(0.02) |
| Sparse CCA with logistic regression view 1 | 0.58(0.01) | 0.60(0.01) | 0.43(0.01) | 0.51(0.01) | 0.37(0.01) | 0.58(0.02) | 0.60(0.01) | 0.43(0.01) | 0.51(0.01) | 0.37(0.01) |
| Sparse CCA with logistic regression | 0.58(0.01) | 0.60(0.01) | 0.43(0.01) | 0.52(0.02) | 0.37(0.01) | 0.58(0.01) | 0.60(0.01) | 0.43(0.01) | 0.51(0.01) | 0.37(0.01) |
| PMA with logistic regression view 1 | 0.58(0.01) | 0.60(0.01) | 0.43(0.01) | 0.51(0.01) | 0.37(0.01) | 0.58(0.01) | 0.60(0.01) | 0.43(0.01) | 0.51(0.01) | 0.37(0.01) |
| PMA with logistic regression | 0.58(0.02) | 0.60(0.01) | 0.42(0.02) | 0.52(0.04) | 0.36(0.03) | 0.58(0.01) | 0.60(0.01) | 0.43(0.01) | 0.51(0.01) | 0.37(0.01) |
| SVM on top 50 selected features view 1 | 0.50(0.01) | 0.51(0.01) | 0.52(0.01) | 0.58(0.01) | 0.47(0.01) | 0.53(0.01) | 0.53(0.01) | 0.56(0.01) | 0.60(0.01) | 0.52(0.02) |
| SVM on top 50 selected features | 0.50(0.01) | 0.51(0.01) | 0.52(0.01) | 0.58(0.01) | 0.47(0.02) | 0.54(0.01) | 0.54(0.01) | 0.58(0.01) | 0.61(0.01) | 0.55(0.02) |
| SVM view 1 | 0.50(0.01) | 0.51(0.01) | 0.52(0.01) | 0.58(0.01) | 0.48(0.02) | 0.53(0.01) | 0.53(0.01) | 0.56(0.01) | 0.60(0.01) | 0.52(0.02) |
| SVM | 0.54(0.01) | 0.54(0.01) | 0.58(0.02) | 0.61(0.02) | 0.55(0.03) | 0.54(0.01) | 0.54(0.01) | 0.58(0.01) | 0.61(0.01) | 0.55(0.02) |
| Setting3 | (n1 = 200, | n2 = 150, | p1 = 2000, | p2 = 2000) | Setting4 | (n1 = 3000, | n2 = 2250, | p1 = 2000, | p2 = 2000) | |
|---|---|---|---|---|---|---|---|---|---|---|
| Method | Accuracy | AUROC | F1 | Precision | Recall | Accuracy | AUROC | F1 | Precision | Recall |
| Deep IDA on top 200 selected features view 1 | 0.71(0.07) | 0.70(0.07) | 0.74(0.07) | 0.74(0.06) | 0.75(0.11) | 0.70(0.06) | 0.69(0.06) | 0.74(0.06) | 0.73(0.07) | 0.77(0.12) |
| Deep IDA on top 200 selected features | 0.58(0.04) | 0.56(0.04) | 0.67(0.04) | 0.62(0.04) | 0.74(0.11) | 0.69(0.05) | 0.67(0.05) | 0.73(0.04) | 0.72(0.05) | 0.76(0.08) |
| Deep IDA view 1 | 0.53(0.07) | 0.53(0.06) | 0.56(0.15) | 0.59(0.05) | 0.58(0.22) | 0.73(0.10) | 0.72(0.11) | 0.78(0.09) | 0.76(0.11) | 0.81(0.12) |
| Deep IDA | 0.55(0.05) | 0.51(0.02) | 0.61(0.24) | 0.53(0.18) | 0.80(0.34) | 0.67(0.07) | 0.65(0.07) | 0.72(0.06) | 0.69(0.06) | 0.75(0.07) |
| Deep CCA on top 200 selected features view 1 | 0.68(0.05) | 0.69(0.05) | 0.69(0.05) | 0.77(0.05) | 0.62(0.05) | 0.62(0.08) | 0.62(0.07) | 0.64(0.08) | 0.69(0.07) | 0.60(0.10) |
| Deep CCA on top 200 selected features | 0.61(0.03) | 0.61(0.04) | 0.63(0.03) | 0.69(0.04) | 0.58(0.03) | 0.62(0.08) | 0.62(0.07) | 0.64(0.08) | 0.69(0.07) | 0.60(0.09) |
| Deep CCA view 1 | 0.70(0.02) | 0.70(0.02) | 0.71(0.02) | 0.78(0.03) | 0.66(0.03) | 0.59(0.05) | 0.60(0.05) | 0.62(0.04) | 0.67(0.05) | 0.57(0.04) |
| Deep CCA | 0.62(0.02) | 0.63(0.02) | 0.64(0.02) | 0.70(0.03) | 0.60(0.03) | 0.61(0.03) | 0.61(0.03) | 0.64(0.02) | 0.68(0.03) | 0.60(0.03) |
| Sparse CCA with logistic regression view 1 | 0.58(0.01) | 0.60(0.01) | 0.43(0.01) | 0.51(0.01) | 0.37(0.01) | 0.58(0.01) | 0.60(0.01) | 0.43(0.01) | 0.51(0.01) | 0.37(0.01) |
| Sparse CCA with logistic regression | 0.57(0.01) | 0.60(0.01) | 0.43(0.01) | 0.50(0.02) | 0.37(0.01) | 0.58(0.01) | 0.60(0.01) | 0.43(0.01) | 0.51(0.01) | 0.37(0.01) |
| PMA with logistic regression view 1 | 0.57(0.02) | 0.60(0.01) | 0.43(0.01) | 0.50(0.03) | 0.38(0.02) | 0.58(0.01) | 0.60(0.01) | 0.43(0.01) | 0.51(0.01) | 0.37(0.01) |
| PMA with logistic regression | 0.58(0.01) | 0.60(0.01) | 0.43(0.01) | 0.51(0.01) | 0.37(0.01) | 0.58(0.01) | 0.60(0.01) | 0.43(0.01) | 0.51(0.01) | 0.37(0.01) |
| SVM on top 200 selected features view 1 | 0.50(0.02) | 0.51(0.02) | 0.52(0.02) | 0.58(0.02) | 0.46(0.03) | 0.54(0.01) | 0.54(0.01) | 0.57(0.01) | 0.61(0.01) | 0.54(0.01) |
| SVM on top 200 selected features | 0.51(0.02) | 0.52(0.01) | 0.53(0.02) | 0.59(0.02) | 0.48(0.03) | 0.53(0.01) | 0.53(0.01) | 0.58(0.01) | 0.60(0.01) | 0.56(0.01) |
| SVM view 1 | 0.54(0.02) | 0.54(0.02) | 0.58(0.02) | 0.61(0.01) | 0.55(0.04) | 0.54(0.01) | 0.54(0.01) | 0.57(0.01) | 0.61(0.01) | 0.54(0.01) |
| SVM | 0.55(0.02) | 0.53(0.02) | 0.62(0.02) | 0.59(0.02) | 0.66(0.03) | 0.53(0.01) | 0.53(0.01) | 0.58(0.01) | 0.60(0.01) | 0.56(0.01) |
This table summarizes the evaluation of nonlinear simulations on the testing set across four distinct scenarios, showcasing metrics such as Accuracy, F1 Score, Precision, Recall, and AUROC curve. The top selected features are obtained by our proposed Deep IDA + Bi-Bootstrap. Each entry displays the mean values derived from 20 simulations, with the standard deviation provided in parentheses. All values rounded to 2 digits.
We bold highest values for each metric.