Can neural networks understand monotonicity reasoning?
Table 6: Accuracies (%) for different models and training datasets.
['Model', 'Train', 'Upward', 'Downward', 'Non', 'All']
The overall accuracy of each model was low. In particular, all models underperformed the majority baseline on downward inferences, despite some models having rich lexical knowledge from a knowledge base (KIM) or pretraining (BERT). This indicates that downward inferences are difficult to perform even with the expansion of lexical knowledge. In addition, it is interesting to see that if a model performed better on upward inferences, it performed worse on downward inferences. We will investigate these results in detail below.