Table 2.
Model selection metrics (testing folds) using different machine learning methods.
| Method | Accuracy | Slice | Precision | Recall | score |
|---|---|---|---|---|---|
| Linear SVC | 0.875 ± 0.009 | 0 | 0.86 ± 0.006 | 0.92 ± 0.003 | 0.89 ± 0.005 |
| 1 | 0.89 ± 0.011 | 0.82 ± 0.008 | 0.86 ± 0.010 | ||
| Macro avg | 0.88 ± 0.005 | 0.87 ± 0.003 | 0.87 ± 0.009 | ||
| Micro avg | 0.87 ± 0.008 | 0.88 ± 0.008 | 0.88 ± 0.008 | ||
| Logistic regression | 0.879 ± 0.006 | 0 | 0.86 ± 0.004 | 0.93 ± 0.004 | 0.90 ± 0.004 |
| 1 | 0.91 ± 0.007 | 0.81 ± 0.005 | 0.86 ± 0.006 | ||
| Macro avg | 0.88 ± 0.005 | 0.87 ± 0.004 | 0.88 ± 0.005 | ||
| Micro avg | 0.88 ± 0.004 | 0.88 ± 0.004 | 0.88 ± 0.004 | ||
| Random Forest (n=50) | 0.914 ± 0.003 | 0 | 0.89 ± 0.005 | 0.96 ± 0.004 | 0.93 ± 0.005 |
| 1 | 0.95 ± 0.007 | 0.86 ± 0.005 | 0.90 ± 0.006 | ||
| Macro avg | 0.92 ± 0.004 | 0.91 ± 0.003 | 0.91 ± 0.004 | ||
| Micro avg | 0.92 ± 0.003 | 0.92 ± 0.003 | 0.92 ± 0.003 | ||
| MLP ([100, 50], ReLU | 0.902 ± 0.007 | 0 | 0.90 ± 0.006 | 0.93 ± 0.003 | 0.91 ± 0.005 |
| 1 | 0.91 ± 0.006 | 0.87 ± 0.004 | 0.89 ± 0.005 | ||
| Macro avg | 0.90 ± 0.004 | 0.90 ± 0.003 | 0.90 ± 0.003 | ||
| Micro avg | 0.90 ± 0.004 | 0.90 ± 0.004 | 0.90 ± 0.004 | ||
| AdaBoost (n=50) | 0.886 ± 0.012 | 0 | 0.87 ± 0.011 | 0.93 ± 0.012 | 0.90 ± 0.011 |
| 1 | 0.91 ± 0.013 | 0.84 ± 0.011 | 0.87 ± 0.012 | ||
| Macro avg | 0.89 ± 0.011 | 0.88 ± 0.011 | 0.88 ± 0.011 | ||
| Micro avg | 0.88 ± 0.010 | 0.89 ± 0.009 | 0.89 ± 0.009 | ||
| FastText + FFNN | 0.897 ± 0.006 | 0 | 0.87 ± 0.005 | 0.96 ± 0.006 | 0.91 ± 0.005 |
| 1 | 0.94 ± 0.004 | 0.83 ± 0.004 | 0.89 ± 0.004 | ||
| Macro avg | 0.91 ± 0.005 | 0.89 ± 0.004 | 0.89 ± 0.005 | ||
| Micro avg | 0.90 ± 0.003 | 0.90 ± 0.004 | 0.90 ± 0.003 | ||
| BETO + FFNN | 0.852 ± 0.021 | 0 | 0.85 ± 0.018 | 0.89 ± 0.019 | 0.87 ± 0.018 |
| 1 | 0.85 ± 0.023 | 0.81 ± 0.021 | 0.83 ± 0.022 | ||
| Macro avg | 0.85 ± 0.020 | 0.85 ± 0.021 | 0.85 ± 0.020 | ||
| Micro avg | 0.85 ± 0.019 | 0.85 ± 0.020 | 0.85 ± 0.020 | ||
| RoBERTa-tw + FFNN | 0.867 ± 0.008 | 0 | 0.82 ± 0.007 | 0.95 ± 0.008 | 0.88 ± 0.008 |
| 1 | 0.93 ± 0.006 | 0.78 ± 0.007 | 0.85 ± 0.007 | ||
| Macro avg | 0.88 ± 0.007 | 0.86 ± 0.006 | 0.87 ± 0.007 | ||
| Micro avg | 0.88 ± 0.006 | 0.87 ± 0.006 | 0.87 ± 0.006 | ||
| XLM-T + FFNN | 0.874 ± 0.007 | 0 | 0.84 ± 0.006 | 0.95 ± 0.007 | 0.89 ± 0.006 |
| 1 | 0.93 ± 0.003 | 0.79 ± 0.004 | 0.85 ± 0.004 | ||
| Macro avg | 0.89 ± 0.004 | 0.87 ± 0.004 | 0.87 ± 0.004 | ||
| Micro avg | 0.88 ± 0.004 | 0.87 ± 0.003 | 0.87 ± 0.004 |
Bold fonts indicate the best results and italicized fonts the seconds.