Inducing Relational Knowledge from BERT
Table 2: Overview of the experimental results.
['[EMPTY]', 'Google pr', 'Google rec', 'Google f1', 'DiffVec pr', 'DiffVec rec', 'DiffVec f1', 'BATS pr', 'BATS rec', 'BATS f1']
In this table, for our model, we show results for three different values of K (i.e. the number of selected templates after filtering), which are indicated in subscript. We can see that there are no consistent differences between the BERTmax and BERT∗ variants, and that the choices K=100 outperforms K=50 and K=1000. The weaker performance for that setting thus clearly shows the usefulness of our proposed scoring function. When comparing the results to the baselines, we can see that our model does not consistently outperform the Trans baseline. This is most notable in the case of BATS, where Trans performs overall much better. However, this is not unexpected given that these datasets contain a large number of morphological relationships (often also referred to as syntactic relations in this context), and there is not reason to expect why our proposed method should be able to perform well on such relations. For instance, we are unlikely to find many meaningful templates which express that t is the plural of s.