Deep Learning Based Chatbot Models
Table 1: Perplexity and Bleu scores on validation data. S2S stands for the baseline seq2seq model, Cornell stands for the training setup presented in Section 4.3.2, Cornell S stands for Section 4.3.3, OpenSubtitles stands for Section 4.3.4 and OpenSubtitles F stands for Section 4.3.5.
['[BOLD] Metrics', '[BOLD] S2S', '[BOLD] Cornell', '[BOLD] Cornell S', '[BOLD] OpenSubtitles', '[BOLD] OpenSubtitles F']
This means that for the Cornell model the scores were computed after 20K training steps and for the Cornell S model, scores were computed after 16K steps. For the OpenSubtitles training the model didn’t seem to overfit, however after a quick drop the loss stagnated for the remainder of the training, which could be because the learning rate was not adequate or the model was too small to be able to learn more.