Table 1. Performance comparison of supervised multi-class classifiers on internal and external testing sets.
Classifier | Internal Testing Set (n = 978) | External Testing Set (n = 465) | ||||
---|---|---|---|---|---|---|
Precision | Recall | F1-score | Precision | Recall | f1-score | |
KNN | 0.82 | 0.86 | 0.84 | 0.81 | 0.82 | 0.81 |
Logistic Regression | 0.85 | 0.90 | 0.87 | 0.91 | 0.93 | 0.92 |
Naïve Bayes | 0.78 | 0.82 | 0.80 | 0.70 | 0.47 | 0.38 |
Random Forest | 0.83 | 0.88 | 0.86 | 0.84 | 0.80 | 0.80 |
SVM | 0.85 | 0.90 | 0.88 | 0.91 | 0.93 | 0.92 |
BiLSTM-fastText | 0.93 | 0.93 | 0.93 | 0.91 | 0.91 | 0.91 |
BiLSTM-random | 0.91 | 0.91 | 0.91 | 0.72 | 0.58 | 0.57 |
BiLSTM-Att-fastText | 0.94 | 0.94 | 0.94 | 0.90 | 0.91 | 0.90 |
BiLSTM-Att-random | 0.90 | 0.91 | 0.90 | 0.73 | 0.55 | 0.53 |
BERT | 0.92 | 0.93 | 0.92 | 0.94 | 0.93 | 0.93 |
DistilBERT | 0.91 | 0.91 | 0.91 | 0.93 | 0.93 | 0.93 |
XLNet | 0.93 | 0.93 | 0.93 | 0.95 | 0.95 | 0.95 |
RoBERTa | 0.91 | 0.91 | 0.91 | 0.93 | 0.93 | 0.93 |