Network Features Based Co-hyponymy Detection
Table 4: Percentage F1 scores of our model along with the models described in [Santus et al.2016] on a 10-fold cross validation for binary classification.
['[BOLD] Method', '[BOLD] Co-Hyp vs Random', '[BOLD] Co-Hyp vs Hyper']
Experiment 2: In the second experiment, we use ROOT9 dataset, prepared by \newciteSANTUS16.455. It contains 9600 The dataset is evenly distributed among the three classes (hypernyms, co-hyponyms and random) and involves three types of parts of speech (noun, verb, adjective). The full dataset contains a total of 4,263 distinct terms consisting of 2,380 nouns, 958 verbs and 972 adjectives. Here also, we use the same experimental setup of using Random Forest classifier with ten-fold cross validation as done by \citelanguageresourcesantus2015evalution. We have put all the five network measures as features to the classifier. We try with all the combinations of the five features and get the best performance when all of those features are used together. We see that in the binary classification task of Co-hyponym vs Random, we outperform all the state-of-the-art models in terms of F1 score whereas for Co-hyponym vs Hypernym classification task, our model beats the performance of most of the baseline models and produces comparable performance to the best models. Note that, using only five simple network measures as features we are able to get good performance, which leads to the fact that coming up with some useful features intelligently can help in improving the performance of the otherwise difficult task of co-hyponymy detection. Investigating the DT network more deeply and coming up with some more sophisticated measures for co-hyponymy discrimination specially from hypernymy would definitely be the immediate future work.