Encoders Help You Disambiguate Word Senses in Neural Machine Translation
Table 2: BLEU scores of NMT models, and WSD accuracy on the test set using word embeddings or hidden states to represent ambiguous nouns. The hidden states are from the highest layer.555For encoders in RNNS2Ss, this is the last backward RNN. RNN. and Trans. denote RNNS2S and Transformer models, respectively.
['[EMPTY]', 'DE→EN  [ITALIC] RNN.', 'DE→EN  [ITALIC] Trans.', 'DE→FR  [ITALIC] RNN.', 'DE→FR  [ITALIC] Trans.']
ENC denotes encoder hidden states; DEC means decoder hidden states. We also concatenate the hidden states from both forward and backward RNNs and get higher accuracy, 96.8% in DE→EN and 95.7% in DE→FR. The WSD accuracy of using bidirectional hidden states are competitive to using hidden states from Transformer models. However, concatenating forward and backward hidden states doubles the dimension. Thus, the comparison is not completely fair.