Representing Meaning with a Combination of Logical and Distributional Models
Table 7: Ablation experiment for the system components without eclassif
['[BOLD] Components Enabled', 'Train Acc.', 'Test Acc.']
Because eclassif has the most impact on the system’s accuracy, and when enabled suppresses the contribution of the other components, we evaluate the other components first without eclassif. In the following section, we add the eclassif rules. The results show that each component plays a role in improving the system accuracy. Our best accuracy without eclassif is 80.4%. Without handling the problem of negated hypotheses (logic alone), P(¬H|T) is almost always 1 and this additional inference becomes useless, resulting in an inability to distinguish between Neutral and Contradiction. Adding cwa significantly improves accuracy because the resulting system has P(¬H|T) equal to 1 only for Contradictions.