A Stable and Effective Learning Strategy for Trainable Greedy Decoding
Table 5: Results when trained with different decoding objectives on IWSLT16 De-En translation using Transformer. MTR denotes METEOR. We report greedy decoding and beam search (k=4) results using the original model, and results with trainable greedy decoding (lower half).
['[EMPTY]', 'Obj.', 'BLEU↑', 'MTR↑', 'TER↓']
Decoding Objectives As our approach is capable of using an arbitrary decoding objective, we investigate the effect of different objectives on BLEU, METEOR (MTR) and TER scores with Transformer for IWSLT16 De-En translation. When trained with one objective, our model yields relatively good performance on that objective. For example, negative sentence-level TER (i.e., -sTER) leads to -3.0 TER improvement over greedy decoding and -0.5 TER improvement over beam search. However, since these objectives are all well correlated with each other, training with different objectives do not differ dramatically.