Sparse Sinkhorn Attention
Table 7: Experimental results on natural language inference.
['Model', 'SNLI', 'MNLI']
We find that both Sinkhorn and Sortcut Sinkhorn are able to outperform the vanilla Transformer. This task demonstrates the effectiveness of the SortCut variant despite the improvement of memory complexity over the standard Sinkhorn Transformer.