Acknowledgements
Table 5.17: Testset BLEU scores of our setup. Except for the baseline, each column shows improvements obtained after fine-tuning a single model on different datasets beginning with the score on a trained parent model. The circled names points to the systems in the right side of the table.
['Training dataset', 'EN→GU', 'GU→EN', 'EN→KK', 'KK→EN']
After transfer learning, we select one of the translation directions to translate monolingual data (model \⃝raisebox{-0.9pt}{1}). As the starting system for the backtranslation process, we have selected the English→Gujarati and Kazakh→English. This is however only a rough estimate because higher BLEU scores across various language pairs do not always need to indicate better performance; the properties of the target language such as its morphological richness affect the absolute value of the score. For the Gujarati–English, we decided to start with the model with English source side in contrast to Kazakh→English. (we obtained performance of 0.5 to 4.2 BLEU). The remaining rows show incremental improvements as we perform the various training stages. The last stage of model averaging takes the best performing model and averages it with the previous seven checkpoints that are one and a half hours of training time from each other. We can notice that after each change of parallel corpus, there is a substantial improvement in the performance. The learning curve is computed on the development data. BiRNN Bidirectional Recurrent Neural Network BP Brevity Penalty BPE Byte Pair Encoding CNN Convolutional Neural Network IWSLT International Workshop on Spoken Language Translation LanideNN Language Identification by Neural Networks LM Language Model LSTM Long Short-Term Memory cells MT Machine Translation NLP Natural Language Processing NMT Neural Machine Translation NN Neural Network PBMT Phrase-Based Machine Translation POS Part-of-Speech OOV Out-of-Vocabulary RNN Recurrent Neural Network SMT Statistical Machine Translation T2T Tensor2tensor WAT Workshop on Asian Translation WMT Workshop on Statistical Machine Translation