Figure 3: RAG-HPO has superior recall and precision compared to established HPO analysis tools.

A) We compared the output of Llama-3 70B alone to RAG-HPO paired with Llama-3 70B in assigning HPO terms to 20 previously published case reports. Retrieval Augmented Generation greatly improved the ability of large language models to accurately assign HPO terms (F1 of .12 vs 78 respectively, p< .0001). B) We then compared the capabilities of RAG-HPO paired with Llama3–70B to Doc2HPO, ClinPhen, and FastHPOCR on a cohort of 112 previously published cases. As illustrated by these scores, RAG-HPO demonstrates significantly better performance in precision, recall, and F1 score compared to other analysis tools.