Table 2. The matches between correct and incorrect GPT-3.5 and GPT-4 answers for questions in each category.
GPT: Generative Pre-trained Transformer
GPT-3.5 Correct (%) | GPT-3.5 Incorrect (%) | |
Compulsory Questions | ||
GPT-4 Correct | 28 (56.0%) | 17 (34.0%) |
GPT-4 Incorrect | 1 (2.0%) | 4 (8.0%) |
General Questions | ||
GPT-4 Correct | 75 (59.1%) | 21 (16.5%) |
GPT-4 Incorrect | 7 (5.5%) | 24 (18.9%) |
Scenario-Based Questions | ||
GPT-4 Correct | 28 (46.7%) | 20 (33.3%) |
GPT-4 Incorrect | 3 (5.0%) | 9 (15.0%) |
Total | ||
GPT-4 Correct | 131 (55.3%) | 58 (24.5%) |
GPT-4 Incorrect | 11 (4.6%) | 37 (15.6%) |