Synonyms and Antonyms: Embedded Conflict
Table 3: Performance on the downstream tasks. Concatenation of the original FastText embeddings with transformed embeddings improves the accuracy of logistic regression based classifiers trained on various datasets, such as IMDB reviews, Cornell movie reviews, Toxic Comments and Multi-Domain Sentiment Dataset on various topics.
['Dataset', 'FastText only', 'FastText + New']
For every dataset we trained a logistic regression using pre-trained FastText embedddings and measured its accuracy on test. Then we concatenated the FastText embeddings with the new embeddings produced by the contrasting map and retrained the same logistic regression with these concatenated embeddings. We did not train a contrasting map for every task. The same empeddings that distinguish synonyms and antonyms were concatenated with raw fast text embeddings in every task. Indeed, concatenation with transformed embeddings produced by the contrasting map improves the classification accuracy on various datasets.