Table 3.
Same-Stem Pairs, word2vec | Different-Stem Pairs, fastText | |
---|---|---|
Average precision | 0.9396 | 0.6164 |
Random shuffled mean average precision (95% CI, reshuffled) | 0.8003 (0.7659–0.8343) | 0.4248 (0.3842–0.4700) |
Bootstrap resampled mean average Precision (95% CI, resampled) | 0.9395 (0.9185–0.9579) | 0.6178 (0.5491–0.6855) |
Note: Data are ranked for same-stem and different-stem pairs, compared with the mean average precisions and CIs obtained when the cosine similarity scores are randomly shuffled between data points. Results for both sets of word pairs are far above the upper 95% confidence limits. The resampled confidence limits show where, with 95% confidence, we can expect performance of a method on the whole data space to lie. CI: confidence interval.