Table 3.

Average precision of cosine scores

	Same-Stem Pairs, word2vec	Different-Stem Pairs, fastText
Average precision	0.9396	0.6164
Random shuffled mean average precision (95% CI, reshuffled)	0.8003 (0.7659–0.8343)	0.4248 (0.3842–0.4700)
Bootstrap resampled mean average Precision (95% CI, resampled)	0.9395 (0.9185–0.9579)	0.6178 (0.5491–0.6855)

Note: Data are ranked for same-stem and different-stem pairs, compared with the mean average precisions and CIs obtained when the cosine similarity scores are randomly shuffled between data points. Results for both sets of word pairs are far above the upper 95% confidence limits. The resampled confidence limits show where, with 95% confidence, we can expect performance of a method on the whole data space to lie. CI: confidence interval.