Capsule-Transformer for Neural Machine Translation
Table 1: Comparing with existing NMT systems on WMT17 Chinese-to-English (Zh-En) and WMT14 English-to-German (En-De) tasks.
['[BOLD] System', '[BOLD] Architecture', '[BOLD] Zh-En', '[BOLD] En-De']
As shown in the table, our capsule-Transformer model consistently improves the performance across both language pairs and model variations, which shows the effectiveness and generalization ability of our approach. For WMT17 Zh-En task, our model outperforms all the models listed above, especially only the capsule-Transformer-Base model could achieve a score higher even than the other Big model. For WMT14 En-De task, our model outperforms the corresponding baseline while inferior to the Big model proposed by Li et al. Considering their model introduces over 33M new parameters (while for our Big model, this number is 1.6K) and uses a much larger batch size than ours (4096 vs. 1024) in the training process, it is reasonable for us to believe that our model would achieve a more promising score if the condition was the same.