Table 4:
Multi-model safety evaluation on OCA-ClinHarm MIMICEXT200 (MIMIC-IV-Ext derived clinical scenarios) with escalation-handoff enabled. Deploy VRa with Wilson 95% CI and verdict distribution for N=200 scenarios per model per configuration.
| Model | Config | Deploy VR (%) [95% CI] | Allow (%) | Block (%) | Escalate (%) | Strict VR (%) [95% CI] |
|---|---|---|---|---|---|---|
| Qwen 7B | C0 | 69.5 [62.8, 75.5] | 100.0 | 0.0 | 0.0 | 69.5 [62.8, 75.5] |
| Qwen 7B | C1 | 2.5 [1.1, 5.7] | 36.0 | 64.0 | 0.0 | 64.0 [57.1, 70.3] |
| Qwen 7B | C2 | 3.5 [1.7, 7.0] | 39.0 | 32.0 | 29.0 | 61.0 [54.1, 67.5] |
| Qwen 7B | C2+SRP | 2.0 [0.8, 5.0] | 56.0 | 32.0 | 12.0 | 44.0 [37.3, 50.9] |
| Llama 8B | C0 | 72.5 [65.9, 78.2] | 100.0 | 0.0 | 0.0 | 72.5 [65.9, 78.2] |
| Llama 8B | C1 | 1.5 [0.5, 4.3] | 58.5 | 41.5 | 0.0 | 42.5 [35.9, 49.4] |
| Llama 8B | C2 | 1.0 [0.3, 3.6] | 65.0 | 35.0 | 0.0 | 36.0 [29.7, 42.9] |
| Llama 8B | C2+SRP | 1.0 [0.3, 3.6] | 65.0 | 34.5 | 0.5 | 36.0 [29.7, 42.9] |
| Meditron 7B | C0 | 19.5 [14.6, 25.5] | 100.0 | 0.0 | 0.0 | 19.5 [14.6, 25.5] |
| Meditron 7B | C1 | 0.0 [0.0, 1.9] | 80.5 | 19.5 | 0.0 | 19.5 [14.6, 25.5] |
| Meditron 7B | C2 | 0.0 [0.0, 1.9] | 84.0 | 0.0 | 16.0 | 16.0 [11.6, 21.7] |
| Meditron 7B | C2+SRP | 0.0 [0.0, 1.9] | 84.5 | 0.0 | 15.5 | 15.5 [11.1, 21.2] |
| Sonnet 4.5 | C0 | 16.0 [11.6, 21.7] | 100.0 | 0.0 | 0.0 | 16.0 [11.6, 21.7] |
| Sonnet 4.5 | C1 | 1.0 [0.3, 3.6] | 92.0 | 8.0 | 0.0 | 9.0 [5.8, 13.8] |
| Sonnet 4.5 | C2 | 0.0 [0.0, 1.9] | 98.5 | 0.0 | 1.5 | 1.5 [0.5, 4.3] |
| Sonnet 4.5 | C2+SRP | 0.0 [0.0, 1.9] | 95.0 | 0.0 | 5.0 | 5.0 [2.7, 9.0] |
| Gemma 4 E4B | C0 | 14.0 [9.9, 19.5] | 100.0 | 0.0 | 0.0 | 14.0 [9.9, 19.5] |
| Gemma 4 E4B | C1 | 0.0 [0.0, 1.9] | 96.0 | 4.0 | 0.0 | 4.0 [2.0, 7.7] |
| Gemma 4 E4B | C2 | 0.0 [0.0, 1.9] | 99.5 | 0.0 | 0.5 | 0.5 [0.1, 2.8] |
| Gemma 4 E4B | C2+SRP | 0.0 [0.0, 1.9] | 99.5 | 0.0 | 0.5 | 0.5 [0.1, 2.8] |
VR = deploy violation rate (violations reaching the user); Strict VR also counts blocked/escalated cases. Wilson 95% CIs for N=200.