Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints
Table 4: Multilingual SimLex-999. The effect of Attract-Repel (A-R) on alternative sets of starting word vectors (Random = xavier initialisation).
['[BOLD] Word Vectors', '[BOLD] EN', '[BOLD] DE', '[BOLD] IT', '[BOLD] RU']
skip-gram with negative sampling vectors trained on the latest Wikipedia dumps. The randomly initialised vectors serve to decouple the impact of injecting external knowledge from the information embedded in the distributional vectors. The random vectors benefit from both mono- and cross-lingual specialisation: the English performance is surprisingly strong, with other languages suffering more from the lack of initialisation.