Can neural networks understand monotonicity reasoning?
Table 7: Evaluation results on types of monotonicity reasoning. –Hyp: Hypothesis-only model.
['Training set', 'Upward', 'Downward', 'Non', 'All']
This indicates that the MED test set does not allow models to predict from hypotheses alone. Data augmentation by HELP improved the overall accuracy to 71.6%, but there is still room for improvement. In addition, while adding HELP increased the accuracy on downward inferences, it slightly decreased accuracy on upward inferences. The size of downward examples in HELP is much larger than that of upward examples. This might improve accuracy on downward inferences, but might decrease accuracy on upward inferences.