Table 2.
Scores and mean values of accuracy assigned by Raters to the answers provided by GPT-3.5 and GPT-4.0.
Q | Items | Groups | Item Mean | Mean 3.5 | Mean 4.0 | Total Mean | |||
---|---|---|---|---|---|---|---|---|---|
Rater 1 | Rater 2 | ||||||||
GPT-3.5 | GPT-4.0 | GPT-3.5 | GPT-4.0 | ||||||
1 | Correctness | 3 | 3 | 3 | 4 | 3.25 | 2.67 | 3.50 | 3.08 |
Clarity | 2 | 3 | 2 | 3 | 2.50 | ||||
Exhaustiveness | 3 | 4 | 3 | 4 | 3.50 | ||||
2 | Correctness | 3 | 4 | 3 | 4 | 3.50 | 3.17 | 4.00 | 3.58 |
Clarity | 3 | 4 | 2 | 4 | 3.25 | ||||
Exhaustiveness | 4 | 4 | 4 | 4 | 4.00 | ||||
3 | Correctness | 2 | 3 | 1 | 2 | 2.00 | 1.17 | 2.17 | 1.67 |
Clarity | 1 | 2 | 1 | 2 | 1.50 | ||||
Exhaustiveness | 1 | 2 | 1 | 2 | 1.50 | ||||
4 | Correctness | 3 | 4 | 3 | 3 | 3.25 | 3.17 | 3.33 | 3.25 |
Clarity | 3 | 3 | 3 | 3 | 3.00 | ||||
Exhaustiveness | 3 | 4 | 4 | 3 | 3.50 | ||||
5 | Correctness | 4 | 3 | 3 | 4 | 3.50 | 3.33 | 3.83 | 3.58 |
Clarity | 3 | 4 | 3 | 4 | 3.50 | ||||
Exhaustiveness | 3 | 4 | 4 | 4 | 3.75 | ||||
6 | Correctness | 3 | 4 | 3 | 3 | 3.25 | 3.33 | 3.83 | 3.58 |
Clarity | 3 | 4 | 4 | 4 | 3.75 | ||||
Exhaustiveness | 3 | 4 | 4 | 4 | 3.75 | ||||
7 | Correctness | 4 | 4 | 4 | 4 | 4.00 | 3.50 | 4.00 | 3.75 |
Clarity | 3 | 4 | 3 | 4 | 3.50 | ||||
Exhaustiveness | 3 | 4 | 4 | 4 | 3.75 | ||||
8 | Correctness | 4 | 4 | 4 | 4 | 4.00 | 4.00 | 4.00 | 4.00 |
Clarity | 4 | 4 | 4 | 4 | 4.00 | ||||
Exhaustiveness | 4 | 4 | 4 | 4 | 4.00 | ||||
9 | Correctness | 4 | 4 | 4 | 3 | 3.75 | 4.00 | 3.83 | 3.92 |
Clarity | 4 | 4 | 4 | 4 | 4.00 | ||||
Exhaustiveness | 4 | 4 | 4 | 4 | 4.00 | ||||
10 | Correctness | 3 | 4 | 3 | 3 | 3.25 | 3.33 | 3.83 | 3.58 |
Clarity | 3 | 4 | 4 | 4 | 3.75 | ||||
Exhaustiveness | 3 | 4 | 4 | 4 | 3.75 | ||||
11 | Correctness | 4 | 2 | 4 | 3 | 3.25 | 4.00 | 3.17 | 3.58 |
Clarity | 4 | 3 | 4 | 4 | 3.75 | ||||
Exhaustiveness | 4 | 3 | 4 | 4 | 3.75 | ||||
Total items’ mean | Correctness | 3.36 | 3.55 | 3.18 | 3.36 | 3.36 | |||
Clarity | 3.00 | 3.55 | 3.09 | 3.64 | 3.32 | ||||
Exhaustiveness | 3.18 | 3.73 | 3.64 | 3.73 | 3.57 | ||||
Total | mean | 3.18 | 3.61 | 3.30 | 3.58 | 3.42 | 3.24 | 3.59 | 3.42 |
DS | 0.80 | 0.65 | 0.94 | 0.65 | 0.65 | 0.87 | 0.65 | 0.79 | |
% | 79.5 | 90.2 | 82.6 | 89.4 | 85.4 | 81.1 | 89.8 | 85.4 |