Inducing Relational Knowledge from BERT
Table 4: Breakdown of results for the DiffVec dataset (F1).
['[EMPTY]', 'DiffVec', 'Transsg', 'SVMsg', 'BERTmax100']
For the morphological relations, in this case, our model performs particularly poorly. For instance, for the Verb 3rd relation (e.g. accept-accepts), the Trans baseline achieves an F1 score of 97.0, compared to only 20.3 for our model. As already mentioned, however, it is not unexpected that our model is unsuitable for such relations. More surprising, perhaps, is the fact that our model also performs poorly on lexical relations such as hypernymy, where the Trans baseline achieves an F1 score of 73.6 compared to only 54.3 for our model. For the other types of relations however, our model consistently outperforms the baseline (with the Expression relation as the only exception). These other relations are about attributive knowledge, causality, and other forms of commonsense knowledge.