Table 3.
The performance evaluation of different models on test set.
| Method | Suicide notes |
Last statements |
Neutral posts |
Avg. |
||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| P (%) | R (%) | F1 (%) | P (%) | R (%) | F1 (%) | P (%) | R (%) | F1 (%) | P (%) | R (%) | F1 (%) | |
| J48 | 67.5 | 64.4 | 65.9 | 64.5 | 73.1 | 68.5 | 86.1 | 84.7 | 85.4 | 79.3 | 79.1 | 79.2 |
| Naive Bayes | 66.7 | 69.0 | 67.8 | 53.3 | 83.6 | 65.1 | 95.4 | 82.3 | 88.4 | 83.7 | 80.0 | 81.8 |
| Bayes Net | 88.1 | 67.8 | 76.6 | 66.3 | 94.0 | 77.8 | 93.5 | 91.0 | 92.2 | 88.4 | 87.0 | 87.7 |
| LMT | 82.6 | 65.5 | 73.1 | 100 | 65.7 | 79.3 | 87.7 | 99.7 | 93.3 | 88.5 | 88.1 | 88.3 |
| CNN | 90.0 | 72.4 | 80.2 | 93.9 | 91.0 | 92.4 | 91.9 | 97.7 | 94.7 | 91.8 | 91.9 | 91.7 |
| BiLSTM | 42.9 | 83.9 | 56.8 | 40.0 | 3.0 | 5.6 | 93.6 | 87.0 | 90.2 | 75.9 | 74.0 | 74.9 |
| BiLSTMAttention | 87.2 | 78.2 | 82.4 | 96.9 | 92.5 | 94.7 | 94.2 | 98.0 | 96.1 | 93.3 | 93.4 | 93.3 |
| DLSTMAttention | 85.5 | 81.6 | 83.5 | 96.9 | 92.5 | 94.7 | 94.8 | 97.0 | 95.9 | 93.3 | 93.4 | 93.3 |
| TransformerRNN | 87.5 | 88.5 | 88.0 | 94.0 | 94.0 | 94.0 | 97.4 | 97.0 | 97.2 | 95.0 | 94.9 | 94.9 |
Values in bold are the maximum scores attained.