Differentiable Reasoning over a Virtual Knowledge Base
Figure 4: Left: F1 scores on Shared Entity and Shared Relation negatives. The negative examples are for the Query : (Neil Herron, occupation, ?). Right: Macro-avg accuracy on the levy2017zero relation extraction dataset. We split the results based on frequency of the relations in our WikiData training data. DrKIT-all spans refers to a variant of our model which selects from all spans in the corpus, instead of only entity-linked mentions.
['[BOLD] Probing  [BOLD] Task', '[BOLD] Negative Example', '[BOLD] BERT', '[BOLD] DrKIT']
We report results on 2 subsets of relations in addition to all relations. The Rare subset comprises of relations with frequencies < 5 in the training data while the ’Frequent’ subset contains the rest. DrKIT on entity-mentions consistently outperforms the other phrase-based models showing the benefit of indexing only entity-mentions in single-hop questions over all spans. Note that DrKit-entities has a high Hits@1 performance on the Rare relations subset, showing that there is generalization to less frequent data due to the natural language representations of entities and relations. In the first experiment, we train and test on shared-entity negatives. Good performance here means the BERT model being probed