Acknowledgements
Table 4.7: Comparison of various approaches to replacing tokens in parent vocabulary.
['Language pair', 'EN→Estonian', 'Estonian→EN']
All approaches reach comparable performance except the "Everything random" assignment. Thus we found no significant differences in the performance as long as subwords common to both the parent and child keep their embeddings, i.e. are mapped to the same index in the vocabulary. The subwords unique to the child vocabulary can be assigned randomly to the unused parent embeddings. A similar result was observed by Zoph et al. They show that the random assignment of words in their approach works as well as an assigning based on lexical similarity. BiRNN Bidirectional Recurrent Neural Network BP Brevity Penalty BPE Byte Pair Encoding CNN Convolutional Neural Network IWSLT International Workshop on Spoken Language Translation LanideNN Language Identification by Neural Networks LM Language Model LSTM Long Short-Term Memory cells MT Machine Translation NLP Natural Language Processing NMT Neural Machine Translation NN Neural Network PBMT Phrase-Based Machine Translation POS Part-of-Speech OOV Out-of-Vocabulary RNN Recurrent Neural Network SMT Statistical Machine Translation T2T Tensor2tensor WAT Workshop on Asian Translation WMT Workshop on Statistical Machine Translation