Integrating Multiplicative Featuresinto Supervised Distributional Methods for Lexical Entailment
Table 3: Best test performance (F1) across different datasets and evaluation setups, using Glove. The number in brackets indicates the performance gap between the best performing combination and base representation setups.
['[BOLD] Setup', '[BOLD] Dataset', '[BOLD] Linear classifiers (LR, LIN) → [ITALIC] vy', '[BOLD] Linear classifiers (LR, LIN)  [BOLD] Base', '[BOLD] Linear classifiers (LR, LIN)  [BOLD] Base', '[BOLD] Linear classifiers (LR, LIN)  [BOLD] Combination', '[BOLD] Linear classifiers (LR, LIN)  [BOLD] Combination', '[BOLD] Non-linear classifiers (RBF, MLP) → [ITALIC] vy', '[BOLD] Non-linear classifiers (RBF, MLP)  [BOLD] Base', '[BOLD] Non-linear classifiers (RBF, MLP)  [BOLD] Base', '[BOLD] Non-linear classifiers (RBF, MLP)  [BOLD] Combination', '[BOLD] Non-linear classifiers (RBF, MLP)  [BOLD] Combination', '[BOLD] KSIM']
The results across the datasets vary substantially in some cases due to the differences between the datasets’ relations, class balance, and the source from which they were created. For instance, K&H+N is imbalanced between the number of instances across relations and domains. ROOT09 was designed to mitigate the lexical memorization issue by adding negative switched hyponym-hypernym pairs to the dataset, making it an inherently more difficult dataset. EVALution contains a richer set of semantic relations. Overall, the addition of multiplicative features improves upon the performance of the base representations.