Offline bilingual word vectors, orthogonal transformations and the inverted softmax
Table 5: Translation performance, using the Europarl corpus as a phrase dictionary.
['Precision', 'English to Italian:  [ITALIC] Mikolov et al.', 'English to Italian:  [ITALIC] Dinu et al.', 'English to Italian: This work', 'Italian to English:  [ITALIC] Mikolov et al.', 'Italian to English:  [ITALIC] Dinu et al.', 'Italian to English: This work']
As outlined earlier, we can form simple sentence vectors in the word vector space by summing and normalising over the words contained in a sentence. These sentence vectors can be used in two different tasks. First, we can use the Europarl corpus as a training dictionary, whereby the matrices XD and YD are formed from the sentence vectors of translation pairs. By applying the SVD to the first 500k sentences in this “phrase dictionary”, we obtain a set of bilingual word vectors from which we can retrieve translations of individual words. We achieve 42.8% precision @1 when translating from English into Italian and 37.5% precision when translating from Italian into English, comparable to the accuracy achieved using the expert word dictionary on the same test set. It is difficult to compare the two approaches, since they require different training data. However our performance appears competitive with Bilbowa, a leading method for learning bilingual vectors online from monolingual corpora and aligned text We do not include results for CCA due to the computational complexity on a dictionary of this size.