Representing Meaning with a Combination of Logical and Distributional Models
Table 6: Cross-validation accuracy on Entailment on lexical rules for Asym evaluation
['[BOLD] Feature set', '[BOLD] Intrinsic', '[BOLD] RTE Train', '[BOLD] RTE Test']
In a qualitative analysis comparing the predictions of a classifier using only Asymmetric features with a classifier using only cosine similarity, we found that the Asymmetric classifier does substantially better at distinguishing hypernymy from co-hyponymy. This is what we had hoped to find, as we had previously found an Asymmetric classifier to perform well at identifying hypernymy in other data roller:coling14, and cosine is known to heavily favor co-hyponymy baronilenci:gems2011. However, we also find that cosine features are better at discovering synonymy, and that Asymmetric frequently mistakes antonymy as an entailing. We did a quantitative analysis comparing the predictions of a classifier using only Asymmetric features to a classifier that tries to learn typical hyponyms or hypernyms by using only the LHS vectors, or the RHS vectors, or both.