Table 1.

Total time (in minutes) to annotate test collections

	BioScope (negation)	NCBI disease	ShARe corpus	i2b2 2010	LHC test
MetaMap	24 m 2 s	29 m 58 s	42 m 36 s	52 m 15 s	17 m 8 s
cTAKES (DL)	No evaluation	25 m 14.04 s	35 m 74 s	35 m 13 s	10 m 1 s
DNorm	No evaluation	1 m 59.800 s	Failed to run	Failed to run	2 m 54 s
MetaMap Lite	4 m 41.995 s	1 m 35.906 s	2 m 50.54 s	2 m 33.06 s	1 m 15.62 s

Note that only the cTAKES dictionary lookup (cTAKES DL) model was used in our experiments. Independent experiments indicate that this model is 30 times slower than the fast lookup model.⁵