Algorithm 3 RAG Retrieval Evaluation (Hit@K, MRR@K)

Require: FAISS index I, chunk ids ids, embedding meta meta, evaluation set Q = ${\left\{\right(q_i,E_i\left)\right\}}_{i=1}^n$ where Ei is expected_ids, top-K Ensure: Hit@K, MRR@K 1:  HITS ← 0, RR ← [ ] 2:  for i = 1 to n do 3:      zi ← EncodeQuery(qi, meta)    ▷ Alg. 2 4:      (_, Ii) ← I.search(zi, K) 5:      Ri ← [ids[j] : j ∈ Ii]    ▷ retrieved ids 6:      if Ri ∩ Ei ≠ ∅ then 7:          HITS ← HITS + 1 8:      end if 9:      r ← 0 10:    for t = 1 to K do 11:          if Ri[t] ∈ Ei then 12:              r ← t; break 13:          end if 14:      end for 15:      if r > 0 then 16:          RR.append(1/r) 17:      else 18:          RR.append(0) 19:      end if 20: end for 21: Hit@K ← HITS/n 22: MRR@K ← mean(RR) 23: return Hit@K, MRR@K