Efficient Sentence Embedding via Semantic Subspace Analysis
TABLE II: Experimental results on supervised tasks, where sentence embeddings are fixed during the training process and the best results for parameterized and non-parameterized models are marked in bold respectively.
['Model', 'Dim', 'MR', 'CR', 'SUBJ', 'MPQA', 'SST', 'TREC', 'MRPC', 'SICK-E', 'Avg.']
The S3E method adopts a word embedding dimension smaller than p-means and VLAWE and also flexible in choosing embedding dimensions. As implemented in other weight-based methods, the S3E method does not consider the order of words in a sentence but splits a sentence into different semantic groups. The S3E method performs the best on the paraphrase identification (MRPC) dataset among all non-parameterized and parameterized methods excluding SBERT-WK. This is attributed to that, when paraphrasing, the order is not important since words are usually swapped. In this context, the correlation between semantic components play an important role in determining the similarity between a pair of sentences and paraphrases.