Question Dependent Recurrent Entity Network for Question Answering
Table 2: Test set error rate comparison between n-gram, LSTM, QDREN, REN and End To End Memory Network (MemN2N). All the results have been taken from the original articles. In bold we highlight the task in which we greatly outperform the other models. On the right the hyper-parameters used in QDREN.
['[BOLD] Task', '[ITALIC] n [BOLD] -gram', '[BOLD] LSTM', '[BOLD] MemN2N', '[BOLD] REN', '[BOLD] QDREN', '[BOLD] Blk', '[ITALIC] λ', '[ITALIC] α', '[BOLD] Dr']
We compared our results with four models: n-gram model, LSTM, original REN (with no question in the gating function) and End To End Memory Network (MemN2N) [sukhbaatar2015end], which is currently the State-Of-The-Art in this setting. To the best of our knowledge we achieved the lowest number of failed tasks, failing just 8 tasks compared with the previous State-Of-The-Art which was 11. Comparing our QDREN with the original Recurrent Entity Network (REN) we achieved, on average, an improvement of 11% in the average error rate and we passed 7 tasks more. We improve the mean error compared to the original REN, however we still do know reach the error rate achieved by the End To End Memory Network (even if we passed more tasks). It is worth to notice the following two facts: first, in task 14 and 18 the error is very close to the threshold for passing the task (5%); second, in task 2, we achieved a slightly worse result (10% error more) with respect to the original REN.