Table 5.

Sensitivity, PPV, F1 score, and NPV average across all trials for individual classes (containing any of the 3 concepts or none) on bootstrapped external validation for the rule-based model, Machine Learning model, Bi-LSTM_simple model, and Bi-LSTM_dropout model, where 0 = no concept and 1 = concept present

		Sensitivity	PPV	F1 score	NPV
Rule-based model	Overall	0.73	0.87	0.74	0.73
	0	0.99	0.75	0.86	0.99
	1	0.46	0.98	0.63	0.65
ML model	Overall	0.50	0.56	0.39	0.50
	0	1.0	0.62	0.77	1
	1	0.004	0.50	0.009	0.50
Bi-LSTM_simple model	Overall	0.55	0.57	0.54	0.56
	0	0.83	0.66	0.73	0.63
	1	0.28	0.49	0.34	0.54
Bi-LSTM_dropout model	Overall	0.50	0.54	0.39	0.50
	0	0.99	0.62	0.77	0.65
	1	0.006	0.47	0.01	0.50