What do Neural Machine Translation Models Learn about Morphology?
Table 7: POS accuracy and BLEU using decoder representations from different language pairs.
['[EMPTY]', 'En-De', 'En-Cz', 'De-En', 'Fr-En']
There is a modest drop in representation quality with the decoder. This drop may be correlated with lower BLEU scores when translating English to Arabic vs. Arabic to English. We report here results that were omitted from the paper due to the space limit. As noted in the paper, all the results consistently show that i) layer 1 performs better than layers 0 and 2; and ii) char-based representations are better than word-based for learning morphology.