A Multifaceted Evaluation of Neural versus Phrase-Based Machine Translation for 9 Language Directions
Table 4: Perplexity scores for the outputs of the best NMT and PBMT systems on language models built on 4 million sentences randomly selected from the News Crawl 2015 corpora.
['[BOLD] Language', '[BOLD] PBMT', '[BOLD] NMT', '[BOLD] Rel. diff.']
For all the language directions considered but one, perplexity is higher on the PBMT output compared to the NMT output. The average relative difference, i.e. considering all language directions, is notable at −10.45%. Thus, our experiment shows that the outputs produced by NMT systems are, in general, more fluent than those produced by PBMT systems.