A Common Semantic Space for Monolingual and Cross-Lingual Meta-Embeddings
Table 9: STS training on English and testing on Spanish.
['Embedding', 'Test EN', 'Test ES']
That is, using the training parameters learned from English and applying them to Spanish and English. The meta-embedding MVM4enes has been obtained by ensembling Jenes, Nenes, FTen, ATen, Pen and FTes. It should be noted that, contrary to previous MVM1, MVM2 and MVM3 meta-embeddings used for word similarity (where we independently generate the best possible meta-embeddings in English and the best possible meta-embeddings in Spanish, and project them to the same space) MVM4 has been generated as a single meta-embedding using English, Spanish and English-Spanish source embeddings. The results show that our meta-embeddings not only improve the STS results in both, English and Spanish. They also exhibit very robust cross-lingual transfer learning capabilities, allowing us to train a Spanish STS model with English data only. Apparently, these results suggest a very promising cross-lingual transfer learning research. Although current STS state-of-the art results are currently being obtained by contextual embeddings, these experiments confirm that our meta-embeddings outperform other existing static embeddings and meta-embeddings. Furthermore, since contextual embeddings require huge resources, our cross-lingual approach could be useful for under-resourced languages.