Probabilistic Relation Induction in Vector Space Embeddings
Table 1: Results of the relation induction experiments (macro-averages).
['[EMPTY]', '[EMPTY]', 'SG-Wiki Google', 'SG-Wiki BATS', 'SG-Wiki DV', 'GloVe-Wiki Google', 'GloVe-Wiki BATS', 'GloVe-Wiki DV', 'SG-GN Google', 'SG-GN BATS', 'SG-GN DV', 'GloVe-CC Google', 'GloVe-CC BATS', 'GloVe-CC DV']
As can be observed, our translation model consistently outperforms all other methods in both MAP and F1 score. Moreover, the regression model consistently outperforms the baselines in terms of MAP score, and outperforms the baselines in for the Google and DV test sets in terms of F1 score (but not for the BATS test set). Among the baselines, 3CosAvg is clearly outperformed by LRCos and SVM. On average, LRCos is the strongest baseline (except for the GloVe-Wiki embedding, where it is outperformed by SVM). This highlights the importance of explicitly modeling the fact that source and target words are expected to belong to a given type. Models which are only trained on translation vectors, such as in the SVM approach, cannot capture this. On the other hand, LRCos relies on cosine similarity to connect source and target words, which is far from optimal, as is evidenced by the large difference in performance between LRCos and our translation model.