Network Features Based Co-hyponymy Detection
Table 5: Accuracy scores of on a 10-fold cross validation for binary classification using SVM and Random forest classifier.
['[BOLD] Classification', '[BOLD] svmSS', '[BOLD] random forestALL']
Experiment 3: The two experiments discussed so far show that using the proposed five network measures in classifiers gives better performance than the state-of-the-art models in the baseline datasets. Further, in order investigate the robustness of our approach, we create our own dataset extracted from BLESS Co-Hypo vs Hyper, Co-Hypo vs Mero, Co-Hypo Vs Random. For each of these tasks we have taken 1,000 randomly extracted pairs for positive instance (co-hyponymy pair) and 1,000 randomly extracted pairs for negative instance (hypernymy, meronymy and random pair, respectively). We have tried with both SVM and Random Forest classifiers with different combination of the proposed five features. We see that the performance of SVM classifier with only one feature structural similarity (SS) and Random Forest classifier with all the five features together provide good performance for all three binary classification tasks, consistent with the first two experiments. Note that, even though we get accuracy in the range of 0.86-0.97 while discriminating co-hyponym pairs from meronym or random pairs, we do not achieve highly accurate results when it comes to classification against hypernym pairs, indicating the fact that words having hypernymy relation and words having co-hyponymy relation may be having similar kind of neighborhood in the DT network, and further research is needed to discriminate between these using network measures only.