Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
Table 3: Comparison of the three models on the task of language modeling, using the PTB dataset (Zaremba et al., 2014). The lower the test perplexity (PPL), the better. We also report the lines of code for implementing the respective model (including model construction, and loss/optimization specification, excluding data reading). In comparison, the official implementation of SeqGAN on TensorFlow involves 480 lines of model code (Yu et al., 2017). Texar implementations have limited the maximum length of each line to 80 chars.
['[BOLD] Models', '[BOLD] Test PPL', '[BOLD] Lines of Model Code']
is the most widely used model for language modeling, due to its simplicity and prominent performance. (The generator is pre-trained with MLE.) The test set perplexity is significantly higher than the LSTM RNNs in our experiments, which is not unexpected because LSTM RNN models are well studied for language modeling and a number of optimal modeling and optimization choices are already known. Texar makes the development very efficient.