Table 3.

Results of the correlation analysis with Pearson correlation coefficient and obtained p-value given in the brackets along with p-value obtained from the Mann–Whitney U test comparing the values of the index of difficulty for correct and incorrect answers for temperature parameter equal to 0.

	S22	A22	S23
Polish
GPT-3.5
Pearson correlation coefficient (p-value)	0.333 (< 0.001***)	0.329 (< 0.001***)	0.111 (0.122 ns)
p-value from Mann–Whitney U test	< 0.001***	< 0.001***	< 0.001***
Cohen’s d	0.706	0.694	0.224
GPT-4
Pearson correlation coefficient (p-value)	0.373 (< 0.001***)	0.325 (< 0.001***)	0.311 (< 0.001***)
p-value from Mann–Whitney U test	< 0.001***	< 0.001***	< 0.001***
Cohen’s d	0.935	0.886	0.837
English
GPT-3.5
Pearson correlation coefficient (p-value)	0.237 (< 0.001***)	0.245 (< 0.001***)	0.198 (0.006**)
p-value from Mann–Whitney U test	< 0.001***	< 0.001***	0.001**
Cohen’s d	0.494	0.514	0.430
GPT-4
Pearson correlation coefficient (p-value)	0.405 (< 0.001***)	0.286 (< 0.001***)	0.224 (0.002**)
p-value from Mann–Whitney U test	< 0.001***	0.002**	0.002**
Cohen’s d	1.022	0.745	0.615