Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion
Table 7: Comparison between different functions in CSLS on four language pairs. Linear is the standard criterion, while logSumExp is equivalent to a logistic regression with hard mining.
['[EMPTY]', 'en-es', 'es-en', 'en-ru', 'ru-en']
As discussed in the main paper, the dot product in the CSLS terms can be replaced by any convex function of W and still yield a convex objective. Using a logSumExp function, i.e., f(x)=log(∑i(exp(xi))) is equivalent to a “local” logistic regression classifier, or equivalently, to a logistic regression with hard mining. We observe that this choice does not significantly changes the performance. This suggests that the local property of the criterion is more important than the form of the loss.