Accelerated Reinforcement Learning for Sentence Generationby Vocabulary Prediction
Table 5: Training time, and maximum memory consumption on our GPU devices for the text generation models. For the full softmax baseline on the En-Ja (2M) experiments, the mini-batch splitting strategy (described in Section 4.4) is applied. CE: Cross-Entropy, Small: Small softmax (our proposed method), Full: Full softmax (the baseline).
['[EMPTY]', '[EMPTY]', '[EMPTY]', 'Training time [minutes/epoch] CE', 'Training time [minutes/epoch] CE', 'Training time [minutes/epoch] REINFORCE w/ CE', 'Training time [minutes/epoch] REINFORCE w/ CE', 'GPU memory [MiB] CE', 'GPU memory [MiB] CE', 'GPU memory [MiB] REINFORCE w/ CE', 'GPU memory [MiB] REINFORCE w/ CE']
We have examined the training-time efficiency of our method. We selected the five datasets to show results with different vocabulary sizes and different maximum sentence lengths, and we observed the same trend on the other datasets. The vocabulary size |V| and the maximum sentence length max(N) are shown for each training dataset. In the training with the standard cross entropy loss, the speedup by our method is not impressive as long as the vocabulary size |V| can be easily handled by the GPUs. Our method is also efficient in terms of GPU memory consumption at training time. These results show that our method easily fits in the memory of the single GTX 1080 GPU, whereas “Full softmax” is very sensitive to the vocabulary size |V| and the sentence lengths. In particular, we observe about 56% reduction in memory usage when using the En-Ja (2M) dataset. By saving the memory usage, one could try using larger models, larger mini-batches, larger vocabularies, and longer target sentences without relying on multiple GPUs.