Table 6.
Classification performance on predicting FDV “ADVO breach” offense types (best results are highlighted in green).
| Models | Resampling | Epochs | Accuracy | ROC | F1 | Precision | Setup Description |
|---|---|---|---|---|---|---|---|
| Naïve Bayes | - | - | 0.4504 | 0.5643 | 0.3775 | 0.8159 | - |
| MLP | - | 10 | 0.6826 | 0.6220 | 0.8028 | 0.6900 | 3 Dense; Dropout; L1 Reg. |
| 10% | 10 | 0.6588 | 0.6067 | 0.7726 | 0.7054 | 3 Dense; L1 Reg. | |
| 10% | 10 | 0.6706 | 0.6195 | 0.7935 | 0.6905 | 3 Dense; Dropout; L1 Reg. | |
| 50% | 10 | 0.6776 | 0.6121 | 0.7994 | 0.6919 | 3 Dense; Dropout | |
| 50% | 10 | 0.6693 | 0.6196 | 0.7827 | 0.7063 | 3 Dense; L1 Reg. | |
| LSTM | - | 10 | 0.6824 | 0.6158 | 0.8024 | 0.6900 | 3 LSTM; Dropouts |
| - | 10 | 0.6824 | 0.6237 | 0.8039 | 0.6900 | 3 LSTM; Dropouts; L1 Reg. | |
| 10% | 10 | 0.6629 | 0.6277 | 0.7705 | 0.7162 | 3 LSTM; L1 Reg. | |
| 50% | 10 | 0.6742 | 0.6210 | 0.7884 | 0.7048 | 3 LSTM; Dropouts | |
| 50% | 10 | 0.6768 | 0.6280 | 0.7930 | 0.7014 | 3 LSTM; Dropouts; L1 Reg. | |
| Bi-LSTM | - | 10 | 0.6836 | 0.6180 | 0.8024 | 0.7000 | 3 Bi-LSTM; Dropouts; L1 Reg. |
| Bi-GRU | - | 10 | 0.6774 | 0.6279 | 0.7927 | 0.7000 | 3 Bi-GRU; Dropouts |
| - | 10 | 0.6822 | 0.6082 | 0.8047 | 0.6900 | 3 Bi-GRU; Dropouts; L1 Reg. | |
| BERT | - | 3 | 0.6882 | 0.6576 | 0.8012 | 0.7100 | MaxLen = 400; Batch size = 12 |