Synonyms and Antonyms: Embedded Conflict
Table 1: Comparison of four different embeddings. For every type of embedding, XGBoost classifier is trained to distinguish two input vectors as synonyms of antonyms. The training was performed on raw embeddings, on the new obtained low dimensional embeddings and the concatenation of the original raw vector with the new transformed one.
['Embeddings type', 'Raw', 'New', 'Concatenated']
We train an XGBoost classifier on four different raw embeddings and check the resulting accuracy of the classifiers on the test subset of synonym and antonym pairs. One can also see that a classifier trained on the concatenation of the raw embeddings with the new ones also outperforms the classifier trained solely on the original embeddings. it is not clear what is the nature of this information. In this subsection we demonstrate that the distinguishing features are non-lineary encoded in the geometry of the original embedding space. It also has no additional trainable parameters but for the parameters of EmbeddingNet. In fact, it entirely relies on the EmbeddingNet performance. Let us regard it as a baseline and ask ourselves if the proposed contrasting map, indeed, manages to capture information that is nonlinearly encoded in the original embeddings.