Table 8.

Statistical significance tests for differences in performance using approximate randomization on the test set from the 2009 i2b2 challenge

	Customized MedEx	CRF	SVM	Simple Majority Voting	Local CRF-Based Voting	Local SVM-Based Voting
Sydney	all, m, mo, do, f	m	m	all, m, du, r	m,du	m
Customized MedEx		all, m, mo, do, f, du	all, m, mo, do, f, du, r	all, m, mo, do, f	all, m, mo, do, f, du	all, m, mo, do, f, du
CRF			NS	all, m, du	du	du
SVM				all, du, r	NS	NS
Simple Majority Voting					du, r	all, du, r
Local CRF-Based Voting						du

The entries in cells indicate that the two systems are significantly different in F-scores for the whole system (all), medication (m), dosage (do), mode (mo), frequency (f), duration (du), and reason (r). NS means “not significant different”. Significance is decided at p = 0.05.