Neural Models for Reasoning over Multiple Mentions using Coreference
Table 3: Accuracy on LAMBADA test set, averaged across two runs with random initializations. context: passages for which the answer is in context. overall: full test set for comparison to prior work. †p<0.0001 using Mcnemar’s test compared to GA w/ GRU.
['[BOLD] Method', '[BOLD] overall', '[BOLD] context']
We use the same setup as Chu et al. In this manner only 80% of the questions are answerable, but the performance increases substantially compared to pure language modeling based approaches. We see a significant gain in performance when using the layer with coreference bias. Furthermore, the 1-hot baseline which uses the same coreference information, but with sequential recency bias fails to improve over the regular GRU layer. While the improvement for C-GRU is small, it is significant, and we note that questions in this dataset involve several different types of reasoning out of which we only tackle one specific kind. The proposed GA w/ C-GRU layer sets a new state-of-the-art on this dataset.