Extending Neural Generative Conversational Model using External Knowledge Sources
Table 1: Comparison of BLEU (sentence_bleu) and Perplexity scores on validation set across different models.
['[BOLD] Model', '[BOLD] PPL', '[BOLD] BLEU-4']
Without the inclusion of L3, Ext-ED did not find the external context useful and the performance was not very different from a Vanilla Seq2Seq dialogue model The encoder context had enough variance to be a viable information source and hence the external context was ignored. Though the improvements are reasonable, the metrics used are not strong indicators for evaluating the influence of external contexts in dialogue. But, they do indicate that the prediction accuracy is improved with the inclusion of external knowledge sources. The poor perplexity for Ext-ED Ablation is because the model is conditioned to predict the next utterance using eci and when not provided the context alone is not sufficiently informative. Another way to interpret this would be to see that the external context and and the dialogue context provide complementary information for better predicting the next utterance. Further, the experiments illustrated that the model, when provided with an informative source of knowledge (the one that has higher variance), will let the model converge faster. One possible hypothesis is that eci, which has high variance and is provided as input in every step of decoding, is learned before the RNN parameters converge. The information in eci is relevant to the context and subsequently helps in training the decoder faster.