Do Neural Models Learn Systematicity of Monotonicity Inference in Natural Language?
Table 4: Results for localism.
['[BOLD] Train', '[BOLD] Dev/Test', '[BOLD] BERT', '[BOLD] LSTM', '[BOLD] TreeLSTM']
When the models were trained with 0pt3, 0pt4 or 0pt5, all performed at around chance on the test set of non-embedding inferences 0pt1 and the test set of inferences involving one embedded clause 0pt2. These results indicate that even if models are trained with a set of inferences containing complex syntactic structures, the models fail to locally interpret their constituents.