Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmax
Table 5: [P]recision, [R]ecall and [F]-score of BUCC training set with cosine similarity score used to optimize the threshold for best F score. Following the naming of [Zweigenbaum et al.2018], we treat en as the target language and the other language as source in forward search. Backward is vice versa. We show the cosine similarity performance of [Artetxe and Schwenk2018] as baseline.
['Models', 'Direction', 'fr-en P', 'fr-en R', 'fr-en F', 'de-en P', 'de-en R', 'de-en F', 'ru-en P', 'ru-en R', 'ru-en F', 'zh-en P', 'zh-en R', 'zh-en F']
For each language pair, we iteratively retrieve the nearest neighbors for each source sentence. We then filter the retrieved nearest-neighbor pairs using the cosine similarity scores. The BUCC forward search identifies English translations of source sentences from another language. Rows 3 and 4 show the performance of our model. Both the forward and backward search achieve very good performance on all metrics, with F-score ranges from 84.6 to 89.2. Both results perform better than the baseline models for fr-en and de-en. We obtain a large F-score performance gain of around 10 points. Once again, the search from non-English to English (the forward direction in this case) works better than starting from English to retrieve the other languages. The performance level is surprisingly stable across languages even for distinct language pairs such as en-zh.