Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering
Table 6: Categorical performance (accuracy) on Multi-NLI dataset. Overlap is the percentage of shared tokens between two sentences. Length is calculated based on the number of tokens of the longer sentence.
['[EMPTY]', '[BOLD] Category', '[BOLD] #Examples', '[BOLD] InferSent', '[BOLD] SSE', '[BOLD] DecAtt', '[BOLD] ESIM [ITALIC] seq', '[BOLD] PWIM']
4.3.5 Categorical Performance Comparison We conducted an in-depth analysis of model performance on the Multi-domain NLI dataset based on different categories: text genre, sentence pair overlap, and sentence length. Sentence length and overlap turn out to be two important factors – the longer the sentences and the fewer tokens in common, the more challenging it is to determine their semantic relationship. How to incorporate asymmetry into model design will be worth more exploration in future research.