KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation
Table 5: Precision, recall and F-1 results on TACRED (%). Except BERTLARGE and MTB, all other models use the BASE size. BERT and BERTLARGE results are reported in zhang-etal-2019-ernie; baldini-soares-etal-2019-matching. Other baseline results are from their corresponding papers.
['Model', 'P', 'R', 'F-1']
KEPLER has a great performance promotion over RoBERTa∗ (our baseline model) and also shows improvements compared to other methods, even if they use a LARGE architecture (BERTLARGE and MTB). Besides, KEPLER-WordNet also shows a slight improvement over RoBERTa∗, while KEPLER-W+W achieves comparable results with KEPLER-Wiki. It suggests that pre-training with WordNet has limited benefits and combining the two KGs cannot bring better performance in our framework.