Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection
Table 3: Comparison with state-of-the-art results on enwiki8. Explicit Sparse Transformer-XL refers to the Transformer with our sparsification method.
['[BOLD] Model', '[BOLD] Params', '[BOLD] BPC']
Compared with the other strong baselines, Transformer-XL can reach a better performance, and Explicit Sparse Transformer outperforms Transformer-XL with an advantage.