Density Matching for Bilingual Word Embedding
Table 4: Ablation study on different components of DeMa-BME.
['[EMPTY]', 'en-fr', 'fr-en', 'en-ja', 'ja-en']
Finally, we perform ablation studies in Tab. In comparison to the full model, we remove the density matching loss Lxy & Lyx, the weakly supervised loss Lsup, the back-translation loss Lbt respectively. First, we observe that without the identical strings as the supervised loss, DeMa-BWE fails to converge as the density matching is difficult given a high-dimensional embedding space to search. Second, when we remove the proposed density matching loss, the model is able to produce reasonable accuracy for fr-en and ja-en, but has undesirable results on en-fr and en-ja, which verifies the necessity of the unsupervised density matching. Third, the back-translation loss is not a crucial component in DeMa-BME; removing it only degrades the model’s performance by a small margin. This indicates that orthogonality is not must-have constraint given the model has enough capacity to learn a good transformation. As shown in the fourth row of Tab.