Table 4. Comparison of classification accuracy for BERT (which we use in this study), TextBlob library, and two scikit-learn algorithms.
| Model name | Class | Metric | Value |
|---|---|---|---|
| Decision Tree | Overall | Accuracy | 0.62 |
| Precision | 0.64 | ||
| Recall | 0.8816 | ||
| Negative | F1 | 0.744 | |
| Precision | 0.537 | ||
| Recall | 0.2562 | ||
| Neutral | F1 | 0.3465 | |
| Precision | 0.478 | ||
| Recall | 0.2104 | ||
| Positive | F1 | 0.2922 | |
| Naive Bayes | Overall | Accuracy | 0.658 |
| Precision | 0.664 | ||
| Recall | 0.918 | ||
| Negative | F1 | 0.7707 | |
| Precision | 0.638 | ||
| Recall | 0.3607 | ||
| Neutral | F1 | 0.4608 | |
| Precision | 0.617 | ||
| Recall | 0.0979 | ||
| Positive | F1 | 0.1686 | |
| BERT | Overall | Accuracy | 0.7398 |
| Precision | 0.821 | ||
| Recall | 0.806 | ||
| Negative | F1 | 0.813 | |
| Precision | 0.6246 | ||
| Recall | 0.666 | ||
| Neutral | F1 | 0.6348 | |
| Precision | 0.612 | ||
| Recall | 0.584 | ||
| Positive | F1 | 0.598 | |
| Logistic Regressor | Overall | Accuracy | 0.635 |
| Precision | 0.652 | ||
| Recall | 0.9 | ||
| Negative | F1 | 0.7535 | |
| Precision | 0.543 | ||
| Recall | 0.303 | ||
| Neutral | F1 | 0.389 | |
| Precision | 0.72 | ||
| Recall | 0.13125 | ||
| Positive | F1 | 0.217 | |
| Support Vector Machine | Overall | Accuracy | 0.64 |
| Precision | 0.639 | ||
| Recall | 0.94 | ||
| Negative | F1 | 0.762 | |
| Precision | 0.637 | ||
| Recall | 0.238 | ||
| Neutral | F1 | 0.3465 | |
| Precision | 0.665 | ||
| Recall | 0.128 | ||
| Positive | F1 | 0.199 |