Neural Polysynthetic Language Modelling
Table 4.6: BLEU scores of baseline and vocabulary experiments for Yupik–English and Guaraní–Spanish machine translation experiments. All BPE vocabularies in this table are of size 5000, learned separately.
['[EMPTY]', 'RNN BPE', 'Transf. BPE', 'Transf. Word', 'Transf. FST', 'Transf. FST+BPE']
We first compare RNN and Transformer translation models using BPE vocabularies of 5000. The size of 5000 was selected for consistency with other experiments and because it was among the highest performing vocabulary size on initial RNN experiments for several language pairs (not reported here). The RNN models were trained using OpenNMT (klein-etal-2017-opennmt) with default settings, and the Transformer models were trained using Sockeye (Sockeye:17) with a 3 layer encoder, 3 layer decoder, batch size 2048, optimized toward perplexity, and the remaining parameters set to defaults. For Guaraní–Spanish, we also experiment with full-word vocabularies, FST-segmented vocabulary (Guaraní side only; Spanish side BPE 5000), and an FST-segmented vocabulary with backoff to BPE (all Guaraní words left unsegmented by the FST were segmented by a BPE model learned for a BPE 5000 vocabulary on Guaraní; Spanish side BPE 5000). Yupik Language Experiments Our Yupik language experiments begin with baseline RNN and Transformer models. Finding that the Transformer strongly outperforms the RNN