Factored Neural Machine Translation
Table 2: Comparison of the performances between standard NMT system and the Factored NMT system in terms of %BLEU computed at word, lemma and factors level. The first line corresponds to 3 standard NMT systems built to generate at the output words, lemmas and factors, respectively.
['[BOLD] Model', '[BOLD] %BLEU  [BOLD] word', '[BOLD] %BLEU  [BOLD] lemma', '[BOLD] %BLEU  [BOLD] factors']
4.3.1 Evaluating each output We compare the values with the Factored NMT system results which models lemmas and factors at the same time. We observe that the difference between the results in BLEU for lemmas using the FNMT are similar to the NMT system. However, the differences evaluating factors are big between the two systems (2.44 difference of %BLEU). This experiment confirms that the task to predict factors managing very different output sizes respect to the source words is not easy. In future we will implement factors also in the input side of the neural network to verify this hypothesis. Also, we have to take into account that we are giving more priority to the length of the lemmas sequence than the factors one during beam search. This also suggests that we adapt our architecture so that factors are better predicted to obtain a final better BLEU at word evaluation.