Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies
Table 5: Translation results with different sizes of the source vocabulary.
['BPE merges', 'Bleu [%] sl-en', 'Bleu [%] be-en']
We varied the number of BPE merges on the source side, fixing the target vocabulary to 50k merges. The best results are with 10k or 20k of BPE merges, which shows that the source vocabulary should be reasonably small to maximize the transfer performance. Less BPE merges lead to more language-independent tokens; it is easier for the cross-lingual embedding to find the overlaps in the shared semantic space.