P-SIF: Document Embeddings Using Partition Averaging
Table 2: P-SIF comparison with the recent embedding techniques on various STS tasks. Baselines taken from [10], [47], [7], [12], [55] and [15]
['[BOLD] Task', '[BOLD] ELMo  [BOLD] orig+all', '[BOLD] ELMo  [BOLD] orig+top', '[BOLD] Bert(pr)  [BOLD] Avg.', '[BOLD] USE', '[BOLD] p-mean', '[BOLD] Fast  [BOLD] Text', '[BOLD] Skip  [BOLD] Thoughts', '[BOLD] Infer  [BOLD] Sent', '[BOLD] Char  [BOLD] pharse', '[BOLD] WME  [BOLD] +PSL', '[BOLD] PSIF  [BOLD] +PSL', '[BOLD] u-SIF  [BOLD] +PSL']
Results and Analysis We denote our embeddings by P-SIF + PSL (+ PSL denotes using the PSL word vectors). We report the average results for the STS tasks. The detailed results on each sub-dataset are in the Supplementary material 5. We observe that P-SIF + PSL outperforms PSL + WR on all datasets, thus supporting the usefulness of our partitioned averaging. We observe that partitioning through overlapping clustering algorithms such as GMM generates a better performance compared to partitioning through sparse dictionary algorithms such as k-svd for some Semantic Textual Similarity (STS) task datasets. The main reason for this peculiar observation is related to the fact that some STS datasets contain documents which are single sentences of a maximum length of 40 words. As discussed earlier (sparse dictionary learning vs. overlapping clustering), for single sentence documents with a small number of topics, overlapping clustering optimizes better than sparse dictionary learning. Therefore, we use GMM for partitioning words into suitable clusters for some STS tasks. But both k-svd and GMM outperform simple averaging (SIF) by significant margins on most STS tasks. We also report qualitative results with real examples in the Supplementary 5.