Bridging Knowledge Gaps in Neural Entailment via Symbolic Models
Table 1: Entailment accuracies on the SciTail dataset. NSnet substantially improves upon its base model and marginally outperforms DGEM.
['[BOLD] Entailment Model', '[BOLD] Valid.', '[BOLD] Test']
The DecompAttn model achieves 74.3% on the test set but drops by 1.6% when the hypotheses are decomposed. The Ensemble approach uses the same hypothesis decomposition and is able to recover 2.1% points by using the KB. The end-to-end NSnet network is able to further improve the score by 3.1% and is statistically significantly (at p-value 0.05) better than the baseline neural entailment model. The model is marginally better than DGEM, a graph-based entailment model proposed by the authors of the SciTail dataset We show significant gains over our base entailment model by using an external knowledge base, which are comparable to the gains achieved by DGEM through the use of hypothesis structure. These are orthogonal approaches and one could replace the base DecompAttn model with DGEM or more recent models Tay et al.