Data Augmentation for Neural Online Chat Response Selection
Table 6: Numbers on recall-at-1. Best results for each dataset and each model are highlighted.
['[EMPTY]', '[BOLD] Ubuntu 100%', '[BOLD] Ubuntu 1%', '[BOLD] Taobao 100%', '[BOLD] Taobao 1%', '[BOLD] Douban 100%', '[BOLD] Douban 1%', '[BOLD] Frames 100%']
For each full-scale dataset, nearly all models gain around 1 to 3 points with one of the proposed data augmentation methods. Permutation works best for LSTM-DE, less so for HRE-DE, and has almost no effect on SMN. This is probably because HRE-DE and SMN have an utterance-level recurrent component which makes them better at capturing long range dependencies. Permutation 1 does not improve on Frames dataset for any model. This might be that Frames has perfect turn-taking, and wizards’ responses are mostly addressing their immediately preceding messages, so moving away the last message in context does not help. In small-scale setting, LSTM-DE with data augmentation outperforms HRE-DE on some of the datasets. SMN gains even more with flipping than in full-scale setting.