Table 6. Comparison of fine-tuned BERT classifier with benchmarks (damage vs non-damage).
| Disasters | Model | Accuracy | Macro-precision | Macro-recall | Macro-f1-score |
|---|---|---|---|---|---|
| Iraq–Iran Earthquake | Rudra et al. (2018) | 56.67 | 28.33 | 50.00 | 36.00 |
| Alam, Ofli & Imran (2020) | – | 70.04 | 63.40 | 64.8 | |
| Kumar, Singh & Saumya (2019) | – | 55.60 | 63.40 | 57.80 | |
| Alam et al. (2021) | – | 67.10 | 68.29 | 67.60 | |
| Madichetty & Sridevi (2021) | 79.58 | 78.73 | 78.86 | 78.48 | |
| Proposed | 95.96 | 96.10 | 94.28 | 95.12 | |
| Sri Lanka Floods | Rudra et al. (2018) | 50 | 35 | 50 | 40 |
| Alam, Ofli & Imran (2020) | – | 70.04 | 63.4 | 64.8 | |
| Kumar, Singh & Saumya (2019) | – | 76.5 | 68 | 67 | |
| Alam et al. (2021) | – | 67.1 | 68.29 | 67.6 | |
| Madichetty & Sridevi (2021) | 86.92 | 87.26 | 86.92 | 86.89 | |
| Proposed | 93.94 | 79.17 | 96.69 | 85.13 | |
| Mexico Earthquake | Rudra et al. (2018) | 55.83 | 41.67 | 52.5 | 43.33 |
| Alam, Ofli & Imran (2020) | – | 70.04 | 63.4 | 64.8 | |
| Kumar, Singh & Saumya (2019) | – | 55.6 | 63.4 | 57.8 | |
| Alam et al. (2021) | – | 67.1 | 68.29 | 67.6 | |
| Madichetty & Sridevi (2021) | 79.37 | 79.9 | 79.37 | 79.26 | |
| Proposed | 92.80 | 88.50 | 89.11 | 88.80 | |
| California Wildfires | Rudra et al. (2018) | 55.17 | 52.5 | 55.83 | 50.02 |
| Kumar, Singh & Saumya (2019) | – | 44.4 | 46.2 | 44.8 | |
| Alam et al. (2021) | – | 67.1 | 68.29 | 67.6 | |
| Madichetty & Sridevi (2021) | 73.31 | 73.78 | 73.45 | 73.24 | |
| Proposed | 86.26 | 78.24 | 82.58 | 80.03 | |
| Hurricane Harvey | Rudra et al. (2018) | 45.18 | 29.03 | 43.75 | 33.69 |
| Alam, Ofli & Imran (2020) | – | 70.04 | 63.4 | 64.8 | |
| Kumar, Singh & Saumya (2019) | – | 66 | 63.4 | 57.8 | |
| Alam et al. (2021) | – | 67.1 | 68.29 | 67.6 | |
| Madichetty & Sridevi (2021) | 77.32 | 76.14 | 76.72 | 76.28 | |
| Proposed | 89.17 | 77.29 | 80.88 | 78.89 | |
| Hurricane Maria | Rudra et al. (2018) | 65 | 62.17 | 63.33 | 61.13 |
| Alam, Ofli & Imran (2020) | – | 70.04 | 63.4 | 64.8 | |
| Kumar, Singh & Saumya (2019) | – | 66 | 63.4 | 57.8 | |
| Alam et al. (2021) | – | 67.1 | 68.29 | 67.6 | |
| Madichetty & Sridevi (2021) | 79.14 | 79.41 | 79.14 | 79.1 | |
| Proposed | 92.21 | 76.70 | 84.18 | 79.79 | |
| Hurricane Irma | Rudra et al. (2018) | 48.51 | 36.73 | 48.33 | 40.07 |
| Alam, Ofli & Imran (2020) | – | 70.04 | 63.4 | 64.8 | |
| Kumar, Singh & Saumya (2019) | – | 66 | 63.4 | 57.8 | |
| Alam et al. (2021) | – | 67.1 | 68.29 | 67.6 | |
| Madichetty & Sridevi (2021) | 77.13 | 77.27 | 77.13 | 77.1 | |
| Proposed | 91.75 | 82.99 | 81.98 | 82.47 |
Notes.
The bold values are the highest performances achieved by the proposed model for each disaster.