Modelling Semantic Categories using Conceptual Neighborhood
Table 1: Cross-validation results on the training split of the text classifier (accuracy and macro-average F1, precision and recall).
['[EMPTY]', '[BOLD] Acc', '[BOLD] F1', '[BOLD] Pr', '[BOLD] Rec']
To inspect how well conceptual neighborhood can be predicted from text, we performed a 10-fold cross validation over the training data, removing for this experiment the unclear cases (i.e., those category pairs with sAB scores between 0.4 and 0.6). We also considered a simple baselineWE based on the number of co-occurring sentences for each pairs, which we might expect to be a reasonably strong indicator of conceptual neighborhood, i.e. the more often two categories are mentiond in the same sentence, the more likely that they are conceptual neighbors. Surprisingly, perhaps, the word vector averaging method seems more robust overall, while being considerably faster than the method using BERT. The results also confirm the intuition that the number of co-occurring sentences is positively correlated with conceptual neighborhood, although the results for this baseline are clearly weaker than those for the proposed classifiers.