Table 3. The accuracy of ChatGPT on GAT Arabic questions with English and Arabic instructions and on GAT English questions with English and Arabic instructions.
ArabicAR refers to using Arabic instructions with Arabic questions (Arabic default). ArabicEN refers to using English instructions with Arabic questions. EnglishEN refers to using English instructions with English questions (English default). EnglishAR refers to using Arabic instructions with English questions. The best performance, for both Arabic and English, is indicated in bold.
| Question type | ArabicEN | ArabicAR | EnglishEN | EnglishAR |
|---|---|---|---|---|
| Reading comprehension | 52.86% | 55.71% | 80.22% | 75.82% |
| Analogy | 39.67% | 37.19% | 54.03% | 51.61% |
| Contextual error | 34.65% | 38.61% | 68.13% | 62.64% |
| Sentence completion | 45.28% | 35.85% | 82.67% | 77.33% |
| Micro-average | 43.80% | 42.74% | 71.49% | 67.10% |
| Macro-average | 43.12% | 41.84% | 71.26% | 66.85% |