Recurrent Neural Network-Based Semantic Variational Autoencoder for Sequence-to-Sequence Learning
Table 2: BLEU score of each model for the language modeling task
['ModelData', 'News Crawl’13', 'News Crawl’14', 'News Crawl’15', 'News Crawl’16', 'TED Talk']
Although there exists other “teacher forcing” metrics, such as negative log likelihood and perplexity, these metrics are insufficient in evaluating whether the semantic space or vector, i.e. the output of the encoder, reflects the global latent feature of the input sentence because the target token at every time step is provided for those “teacher forcing” metrics. As a result, we used BLEU score as a performance measure that target tokens are not provided in each decoding step. For all five test data sets, the proposed RNN–SVAE significantly outperformed the benchmark models. The BLEU scores of RNN–SVAE were almost as twice those of the RNN–AE. Compared to RNN–VAE, RNN–SVAE improved the BLEU score by at least 3.27 (News Crawl’14) and at most 4.17 (News Crawl’13). The relative BLEU improvements of RNN–SVAE against RNN– VAE were between 8.42% and 11.12%.