Linguistic Knowledge as Memory for Recurrent Neural Networks
Table 5: Validation and Test set accuracies on CNN dataset. “MAGE” refers to our proposed model, where the number within parentheses denotes the size of coreference states. “one-hot” refers to a model where coreference ids are appended to the input word embeddings. “BiDAF” refers to the Bidirectional Attention Flow model. Results marked with † are cf (Seo et al., 2017a) and with ‡ from (Dhingra et al., 2016).
['[BOLD] CNN', '[BOLD] val', '[BOLD] test']
Lastly, we test our models on the CNN dataset from Hermann et al. Since its release, the dataset has recieved much attention and several deep learning architectures have been introduced with impressive results. An interesting property of the dataset is that in each article named entities and expressions referring to them have been anonymized by replacing with placeholders (such as @entity24) to make the task purely a comprehension one. For our purposes, without the use of any external tools, we augment the article with extra edges connecting mentions of the same entity, and compare the performance with and without these links. Augmenting the bi-GRU model with MAGE leads to an improvement of 2.5% on the test set. The previous best results for this dataset were achieved by the GA Reader, and we see that adding MAGE to it leads to a further improvement of 0.7%, setting a new state of the art. This is an impressive improvement, given that previous works have reported that we are already close to the upper bound possible on this dataset Note that we are not adding any information beyond what is already available in the dataset, since entity mentions are anonymized.