Table 2. Comparing the fact-checking performance of multiple open-source large language models (LLMs) and the proprietary GPT-4o (the baseline used in previous experiments)^a.

Evaluation metricb	GPT-4o (baseline)	Gemma 3 27B	MedGemma 27B	Llama 3 70B	OpenBioLLM 70B	Qwen 3 32B	Qwen 3 Medical 32B
TPc	6	6	11	12	7	8	8
FPd	3	3	3	2	1	2	3
TNe	89	84	84	78	79	83	82
FNf	2	7	2	8	14	6	7
Sensitivity (recall)	78	46	83	61	33	56	55
Specificity	96	97	97	97	99	98	96
Precision (PPVg)	66	68	79	83	88	80	72
F1-score	71	55	81	70	48	66	63
Accuracy	95	90	95	90	86	92	90
Balanced accuracy	87.3	71	90	79	66	77	76
TP improved atoms	100	100	91	98	100	82	89
FP falsified atoms	0	30	44	50	0	0	14
Overall answer quality improved	20	20	30	8	4	18	24
Overall answer quality equal	14	8	6	4	4	8	22
Overall answer quality worse	8	4	6	6	10	16	10
Hallucination rate	1	2	3	6	11	4	2
Hallucination detection	50	71	80	52	50	69	60
Inaccuracy rate	3	2	3	5	3	3	4
Inaccuracy detection	58	38	90	68	22	21	44

^aThe focus was on general-purpose vs medical fine-tuned models of different sizes: Gemma 3 27B vs MedGemma 27B; Llama 3 70B vs OpenBioLLM 70B; and Qwen 3 32B vs Qwen 3 Medical 32B.

^bNumbers represent the percentage values.

^cTP: true positive.

^dFP: false positive.

^eTN: true negative.

^fFN: false negative.

^gPPV: positive predictive value.