Multilingual Factor Analysis
Table 4: Spearman correlation for Semantic Textual Similarity (STS) tasks in English. All results use the sentence embeddings described in arora2016simple. First row represents monolingual FastText vectors JoulinGBM16fasttext in English, the rest are bilingual embeddings. *STS13 excludes the proprietary SMT dataset.
['Embeddings', 'STS12', 'STS13*', 'STS14', 'STS15', 'STS16']
Semantic Textual Similarity (STS) is a standard benchmark used to assess sentence similarity metrics (agirre2012semeval; agirre2013sem; agirre2014semeval; agirre2015semeval; agirre2016semeval). In this work, we use it to show that our alignment procedure does not degrade the quality of the embeddings at the sentence level. For both IBFA and CCA, we align English and one other language (from French, Spanish, German) using the entire dictionaries (of about 100k word pairs each) provided by conneau2017word. We then use the procedure defined in arora2016simple to create sentence embeddings and use cosine similarity to output sentence similarity using those embeddings. The method’s performance on each set of embeddings is assessed using Spearman correlation to human-produced expert similarity scores.