Multilingual Extractive Reading Comprehension by Runtime Machine Translation
Table 2: MT ablation study on {Japanese, French}-to-English translation showing the development set BLEU score. The ablations are 1) removing beam search. 2) using manually translated questions without oversampling them. 3) not using manually translated questions.
['Translation method', 'Ja-En  [BOLD] Wiki', 'Ja-En  [BOLD] Question', 'Fr-En  [BOLD] Wiki', 'Fr-En  [BOLD] Question']
As a standard practice, beam search is effective in improving the BLEU scores, except for the Japanese-to-English question translations. More importantly, the use of the small amount of manually translated questions is crucial. Without using the question translations as shown in the row of “w/o questions”, the BLEU scores are almost halved. Our oversampling technique for the question translations is effective in better translating the question sentences.