ERNIE: Enhanced Language Representation with Informative Entities
Table 7: Ablation study on FewRel (%).
['Model', 'P', 'R', 'F1']
In this subsection, we explore the effects of the informative entities and the knowledgeable pre-training task (dEA) for ERNIE using FewRel dataset. w/o entities and w/o dEA refer to fine-tuning ERNIE without entity sequence input and the pre-training task dEA respectively. still injects knowledge information into language representation during pre-training, which increases the F1 score of BERT by 0.9%. (2) Although the informative entities bring much knowledge information which intuitively benefits relation classification, ERNIE without dEA takes little advantage of this, leading to the F1 increase of 0.7%.