Multi-Perspective Inferrer:Reasoning Sentences Relationship from Holistic Perspective
Table 5: Linguistic Error Analysis on MultiNLI development mismatched sets. Baseline is the cross alignment baseline
['[BOLD] Category', '[BOLD] Baseline', '[BOLD] SMPI', '[BOLD] IMPI']
We perform the linguistic error analysis using the supplementary annotations provided by the MultiNLI dataset. We compare the MPIs against the outputs of the our base model across 13 categories of linguistic phenomena. Both two MPIs outperform our base model on overall accuracy on the matched and mismatched sets. The word overlap category in the supplementary annotations is whether both sentences share more than 70% of their tokens, containing 37 instances. Their statistical method of word overlap is much different from ours: the word overlap rate Rwo is calculated by the amount of shared tokens and the amount of hypothesis, which is related to the intrinsically asymmetric in NLI MacCartney et al. and the fact that the perspectives mainly is expressed by the hypothesis.