Table 9.
Ablation study of the proposed components in three pre-training settings: image-only, text-only, and image-text.
| Settings | Methods | Reporting: Spanish | Reporting: English | Diagnosis | Prognosis | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BLEU-2 | BLEU-3 | BLEU-4 | ROUGE-L | CIDEr | BLEU-2 | BLEU-3 | BLEU-4 | ROUGE-L | CIDEr | Image-Text | Image-Text | ||
| Full | Med-MLLM | 47.6 | 42.0 | 38.1 | 55.4 | 73.4 | 32.4 | 28.7 | 24.9 | 46.7 | 53.0 | 78.2 | 92.0 |
| (a) | w/o image-only (PCL) | 45.8 | 39.9 | 34.7 | 54.3 | 70.9 | 31.2 | 27.5 | 23.0 | 45.3 | 51.8 | 75.7 | 90.2 |
| (b) | w/o image-only | 39.5 | 35.4 | 32.2 | 48.7 | 57.8 | 28.1 | 24.0 | 19.3 | 42.1 | 45.9 | 69.4 | 82.3 |
| (c) | w/o text-only (MLM) | 45.7 | 39.6 | 34.2 | 54.7 | 71.1 | 31.3 | 27.2 | 21.8 | 44.9 | 50.7 | 73.3 | 86.7 |
| (d) | w/o text-only (SR) | 44.2 | 38.5 | 33.6 | 54.1 | 70.3 | 30.2 | 26.7 | 21.5 | 44.0 | 49.4 | 76.3 | 89.1 |
| (e) | w/o text-only (FIA) | 46.0 | 40.1 | 36.5 | 54.2 | 70.6 | 31.8 | 27.5 | 23.2 | 45.5 | 51.6 | 75.4 | 88.3 |
| (f) | w/o image-text | 42.4 | 38.3 | 34.2 | 53.7 | 66.5 | 29.8 | 25.6 | 20.7 | 44.5 | 48.9 | 74.1 | 85.4 |
We perform the analysis on COVID-19 reporting, diagnosis, and prognosis. All values are reported in percentage (%).