1 Introduction
Table 8: Cross-lingual classification accuracy for 3 language pairs, with 1000 labeled examples.
['[EMPTY]', 'EN → DE', 'DE → EN', 'EN → FR', 'FR → EN', 'EN → ES', 'ES → EN']
They were obtained using 1000 RCV training examples. We report results in both directions, i.e. language X to Y and vice versa. The best performing method in all the pairs except one is CorrNet. In particular, CorrNet often outperforms the approach of Klementiev et al. They proposed two neural network architectures for learning word and document representations using sentence-aligned data only. Instead of an autoencoder paradigm, they propose a margin-based objective that aims to make the representation of aligned sentences closer than non-aligned sentences. While their trained embeddings are not publicly available, they report results for the EN/DE classification experiments, with representations of the same size as here (D=40) and trained on 500K EN/DE sentence pairs. Their best model in that setting reaches accuracies of 83.7% and 71.4% respectively for the EN → DE and DE → EN tasks. One clear advantage of our model is that unlike their model, it can use additional monolingual data. Indeed, when we train CorrNet with 500k EN/DE sentence pairs, plus monolingual RCV documents (which come at no additional cost), we get accuracies of 87.9% (EN → DE) and 76.7% (DE → EN), still improving on their best model. If we do not use the monolingual data, CorrNet’s performance is worse but still competitive at 86.1% for EN → DE and 68.8% for DE → EN.