Acknowledgements
Table 5.8: Child BLEU scores when trained with most parameters frozen. Each row represents a parameter set that was free to train; all other parameters were fixed at their pre-trained values. Best result for non-frozen parts in each column in bold. The results marked with * diverged as the model could not train anything.
['Non-frozen part', 'EN→Estonian', 'Estonian→EN']
encoder for Estonian→EN) can learn the new language. Whenever we freeze everything except a particular layer, we get a completely new picture. Especially the parent embeddings are the least useful for the child because keeping only them leads to diverged training. BiRNN Bidirectional Recurrent Neural Network BP Brevity Penalty BPE Byte Pair Encoding CNN Convolutional Neural Network IWSLT International Workshop on Spoken Language Translation LanideNN Language Identification by Neural Networks LM Language Model LSTM Long Short-Term Memory cells MT Machine Translation NLP Natural Language Processing NMT Neural Machine Translation NN Neural Network PBMT Phrase-Based Machine Translation POS Part-of-Speech OOV Out-of-Vocabulary RNN Recurrent Neural Network SMT Statistical Machine Translation T2T Tensor2tensor WAT Workshop on Asian Translation WMT Workshop on Statistical Machine Translation