Table 5.

Performance of NCATS XGBoost-HLM and GCNN-HLM models with and without using RLM predictions as a descriptor on the three external test sets. N/A: not applicable.

Test Set	Model (Descriptors)	Accuracy	AUC	Sensitivity	Specificity
E1	NCATS XGBoost (RDKit)	0.64	0.69	0.58	0.72
E1	NCATS XGBoost (RDKit + RLM)	0.66	0.73	0.66	0.66
E1	NCATS GCNN (RDKit)	00.67	0.70	0.62	0.73
E1	NCATS GCNN (RDKit + RLM)	0.67	0.77	0.65	0.70
E1	Genentech Model	0.67	N/A	0.67	0.68
E2	NCATS XGBoost (RDKit)	0.67	0.72	0.66	0.68
E2	NCATS XGBoost (RDKit + RLM)	0.73	0.80	0.75	0.73
E2	NCATS GCNN (RDKit)	0.74	0.77	0.62	0.78
E2	NCATS GCNN (RDKit + RLM)	0.76	0.68	0.62	0.70
E3	NCATS XGBoost (RDKit)	0.74	0.84	0.50	0.80
E3	NCATS XGBoost (RDKit + RLM)	0.84	0.87	0.75	0.86
E3	NCATS GCNN (RDKit)	0.82	0.87	0.50	0.90
E3	NCATS GCNN (RDKit + RLM)	0.85	0.79	0.58	0.84
E3	PredMS Model	N/A	0.74	0.70	0.86