A Robust Self-Learning Method for Fully Unsupervised Cross-Lingual Mappings of Word Embeddings: Making the Method Robustly Reproducible as Well
Table 3: Best accuracy (best), average accuracy (avg), success rate (s) and average runtime in minutes (t) of the full system on four other languages, Estonian (ET), Persian (FA), Latvian (LV) and Vietnamese (VI). These languages have been carefully selected to illustrate the robustness of the algorithm. They have very different roots then the English language. We also conducted an ablation study to illustrate the stability of the algorithm on such languages.
['[EMPTY]', '[BOLD] EN-ET best', '[BOLD] EN-ET avg', '[BOLD] EN-ET s', '[BOLD] EN-ET t', '[BOLD] EN-FA best', '[BOLD] EN-FA avg', '[BOLD] EN-FA s', '[BOLD] EN-FA t', '[BOLD] EN-LV best', '[BOLD] EN-LV avg', '[BOLD] EN-LV s', '[BOLD] EN-LV t', '[BOLD] EN-VI best', '[BOLD] EN-VI avg', '[BOLD] EN-VI s', '[BOLD] EN-VI t']
We carefully selected four new languages that are characterized by very different roots than the one used in the original paper. We used Estonian (ET) which is a language that gets its root from Finno-Ugric, the same as Finnish. We also selected Persian (FA), Latvian (LV) and Vietnamese (VI). We observed similar behavior for the Persian language. Interestingly, even with the full system, the algorithm poorly performs on Latvian and Vietnamese languages. We conducted the same ablation study as with the original languages. We can see that the algorithm does not converge without an unsupervised initialization and without the stochastic procedure. It also struggles to converge on three languages out of four when the CSLS component is turned off. These results clearly show that the proposed method may become brittle when the target language shares fewer commonalities with the source language.