Bilingual Embeddingswith Random Walks over Multilingual Wordnets
Table 9: Summary of results: average across all datasets and languages (all), average on relatedness datasets (rel), average on similarity datasets (sim) and number of best results across all datasets and languages (win). Underline for best in each group, and bold for best overall.
['[HTML]FFFFFF', 'all', 'rel', 'sim', 'win']
The summary shows clearly that our proposal to produce hybrid knowledge-based embeddings gets the best overall results, also in relatedness and similarity, and wins in 10 datasets out of the 12. Random walks over multilingual wordnets improve results over just using dictionaries: According to our experiments, the state of the art in bilingual embeddings using monolingual corpora and bilingual dictionaries (row maptxt) underperforms our best system (jointchyb) in all datasets except one (WS for EUIT). Overall, we are able to improve results an average of 7 points, with the biggest gain on similarity (10 points, cf. This shows that multilingual wordnets are not mere bilingual dictionaries, and that our random walk method is able to exploit the internal structure of wordnets. This fact is stressed by the fact that the joint method needs the synthetic corpora based on random walks to outperform maptxt. Regarding the state of the art in hybrid knowledge- and text-based methods, we show that our embeddings overperform NASARI in 5 out of 7 datasets, even though our embeddings rely only on wordnets, in contrast to the richer BabelNet. The other two datasets involve the Italian wordnet, where our results are weaker (see above). The two-step mapping method and our joint methods produce good results: All three methods, when applied to the hybrid corpora containing natural and synthetic text (maphyb, jointhyb and jointchyb), are able to overperform the current state of the art, with our joint method using bilingual constraints providing the best results. When using the synthetic corpora, the situation is reversed, with the joint method providing better results, specially when combined with the bilingual constraints in the loss function. This might be caused by the strong signal in the bilingual synthetic corpora used by the joint method, which the joint method does not take profit from.