Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection
Table 1: Results on the En-De, En-Vi and De-En test sets. Compared with the baseline models, “impl.” denotes our own implementation.
['[BOLD] Model', 'En-De', 'En-Vi', 'De-En']
For En-De, Transformer-based models outperform the previous methods. Compared with the result of Transformer (transformer), Explicit Sparse Transformer reaches 29.4 in BLEU score evaluation, outperforming vanilla Transformer by 0.3 BLEU score. Our model, Explicit Sparse Transformer, achieves a much better performance, 31.1, by a margin of 0.5 over vanilla Transformer. For De-En, we demonstrate that Transformer-based models outperform the other baselines. Compared with Transformer, our Explicit Sparse Transformer reaches a better performance, 35.6. Its advantage is +0.3. To the best of our knowledge, Explicit Sparse Transformer reaches a top line performance on the dataset.