Question Dependent Recurrent Entity Network for Question Answering
Table 4: Test set accuracy comparison between REN+SENT, QDREN+SENT, REN+WIND and QDREN+WIND. We show the best hyper-parameters picked by the model selection, and the accuracy values.
['[EMPTY]', '[BOLD] REN+SENT', '[BOLD] QDREN+SENT', '[BOLD] REN+WIND', '[BOLD] QDREN+WIND']
As before, we selected the models that achieved the highest accuracy in the validation set, and then we estimate its generalization error using the provided test set. The window-based version without question supervision could not achieve an accuracy higher than 0.401. Indeed, saving only facts relative to the question seems to be the key to achieving a good score in this task. We also noticed that using plain sentences, even with QDREN, we cannot achieve a higher accuracy. This might be due to the sentence encoder, since just using the multiplicative masks does not provide enough expressive power for getting key features of the sentence. Moreover, we notice that the accuracy achieved in the training set is always lower than that in the validation and test set. The same phenomenon is present also in other models, in our particular case this might be due to the strong regularization term used in our models.