The Flores Evaluation Datasets for Low-Resource Machine Translation: Nepali–English and Sinhala–English
Table 3: BLEU scores of NMT using various learning settings on devtest (see §3). We report detokenized SacreBLEU Post (2018) for {Ne,Si}→En and tokenized BLEU for En→{Ne,Si}.
['[EMPTY]', '[BOLD] Supervised', '[BOLD] Supervised +mult.', '[BOLD] Unsupervised', '[BOLD] Unsupervised + mult.', '[BOLD] Semi-supervised it. 1', '[BOLD] Semi-supervised it. 2', '[BOLD] Semi-supervised it 1. + mult.', '[BOLD] Semi-supervised it 2. + mult.', '[BOLD] Weakly supervised']
In the supervised setting, PBSMT performed quite worse than NMT, achieving BLEU scores of 2.5, 4.4, 1.6 and 5.0 on English–Nepali, Nepali–English, English–Sinhala and Sinhala–English, respectively. There are several observations we can make.