Table 2.
Zero-shot accuracy results.
| Model | Rationale | Rationale RAG | Chain-of-Thought | Chain-of-Thought RAG | P-value |
|---|---|---|---|---|---|
| Gemma-2–2B | 43.48 | 41.30 | 43.48 | 40.22 | 0.908 |
| Llama-3.2–3B | 60.87 | 63.04 | 52.17 | 52.17 | 0.091 |
| Phi-3.5-mini | 67.39 | 66.30 | 56.52 | 57.61 | 0.037 |
| Llama-3.1–8B | 69.57 | 65.22 | 66.30 | 66.30 | 0.717 |
| Gemma-2–9B | 67.39 | 64.13 | 61.96 | 65.22 | 0.630 |
| Phi-4 | 72.83 | 73.91 | 75.00 | 77.17 | 0.677 |
| Gemma-3–27B | 68.48 | 69.57 | 69.57 | 71.74 | 0.843 |
| Llama-3.3-70B | 73.91 | 75.00 | 78.26 | 79.35 | 0.303 |
Zero-shot prompting overall accuracy for rationale and chain-of-thought prompting, with and without RAG, for the eight studied LLMs. The best performance for each setting is bolded, and the second-best is underlined.