Understanding and Improving TransformerFrom a Multi-Particle Dynamic System Point of View
Table 1: Translation performance (BLEU) on IWSLT14 De-En and WMT14 En-De testsets.
['[BOLD] Method', '[BOLD] IWSLT14 De-En small', '[BOLD] WMT14 En-De base', '[BOLD] WMT14 En-De big']
For the IWSLT14 dataset, our Macaron small outperforms the Transformer small by 1.0 point in terms of BLEU. For the WMT14 dataset, our Macaron base outperforms its Transformer counterpart by 1.6 BLEU points. Our Macaron big outperforms the Transformer big by 1.8 BLEU points. Comparing with other concurrent works, the improvements in our proposed method are still significant.