Handling Homographs in Neural Machine Translation
Table 1: WMT’14, WMT’15 English-German results - We show perplexities (Ppl) on development set and tokenized BLEU on WMT’14 and WMT’15 test set of various NMT systems. We also show different settings for different systems. → represents uni-directional, and ↔ represents bi-directional. We also highlight the best baseline model and the best proposed model in bold. The best baseline model will be referred as base or baseline and the best proposed model will referred to as best for further experiments.
['[BOLD] Context', '[BOLD] Integration', '[BOLD] uni/bi', '[BOLD] #layers', '[BOLD] #params', '[BOLD] Ppl', '[BOLD] WMT14', '[BOLD] WMT15']
We share the parameters in fe with fc (i.e. Me=Mc) since the vocabulary space is the same for context network and encoder. As a result, our context network only slightly increases the number of model parameters. The first thing we observe is that the best context-aware model (results in bold in the table) achieved improvements of around 0.7 BLEU on both WMT14 and WMT15 over the respective baseline methods with 2 layers. This is in contrast to simply using a 3-layer network, which actually degrades performance, perhaps due to the vanishing gradients problem it increases the difficulty in learning.