Challenging Neural Dialogue Models with Natural Data: Memory Networks Fail on Incremental Phenomena
Table 2: MemN2N API call accuracy (%) with extended training data
['training bAbI+ dialogues', 'memory hops', 'embedding size', 'train accuracy', 'test accuracy']
As MemN2N ’s performance on bigger data highly depends on the model’s hyperparameters, in this experiment we perform a grid search over the number of memory hops (1, 2, 3), and the embeddings dimensionality (32, 64, 128) for each train set size independently – everything else is fixed as in the previous experiment. The table only shows the best performing hyperparameter configuration for each of the train set sizes.