Neural Machine Translation with Error Correction
Table 3: Ablation study of different component in our model. The second, third and forth column are results on IWSLT14 De→En, Es→En and WMT14 En→De. The second row is our method. The third row is equal to the second row removing error correction mechanism (ECM). The forth row is equal to the third row removing scheduled sampling (SS). The last raw is equal to the forth row removing two-stream self-attention, i.e., the standard NMT. The prefix “-” means removing this part.
['Method', 'De→En', 'Es→En', 'En→De']
To demonstrate the necessity of each component in our method, we take a series of ablation study on our model on IWSLT14 De→En, Es→En and WMT14 En→De. When disabling error correction mechanism (ECM), we find the model accuracy decreases 0.30, 0.48 and 0.50 points respectively in different tasks. When further removing scheduled sampling (SS), the model accuracy drop to 34.81, 41.16, 28.48 points in three tasks. We observe that in the large-scale dataset, the improvements for scheduled sampling are limited, while our model still achieves stable improvements in the large-scale dataset, which proves the effectiveness of the error correction mechanism. In addition, we also make a comparison between the original NMT and the NMT with two-stream self-attention (TSSA) to verify whether two-stream self-attention mechanism will contribute to model accuracy improvement. This phenomenon also explains the improvement of our method is mainly brought by the error correction, rather than the two-stream self-attention. In summary, every component all plays an indispensable role in our model.