Efficient softmax approximation for GPUs
Table 3: Europarl. Perplexity after 5 epochs for different languages as a function of time t (minutes).
['Language:  [ITALIC] k=', 'bg 50k', 'bg 50k', 'cs 83k', 'cs 83k', 'da 128k', 'da 128k', 'de 143k', 'de 143k', 'el 100k', 'el 100k', 'es 87k', 'es 87k']
Our approach significantly outperforms all competitors by a large margin. For reference, we also show the performance (D-softmax [*]) obtained by improving the D-softmax, to make it more comparable to our method. Our method is 2× to 3× faster than this improved competitor, which demonstrates how critical is our optimization strategy.