ERNIE: Enhanced Language Representation with Informative Entities
Table 5: Results of various models on FewRel and TACRED (%).
['Model', 'FewRel P', 'FewRel R', 'FewRel F1', 'TACRED P', 'TACRED R', 'TACRED F1']
As FewRel does not have any null instance where there is not any relation between entities, we adopt macro averaged metrics to present the model performances. Since FewRel is built by checking whether the sentences contain facts in Wikidata, we drop the related facts in KGs before pre-training for fair comparison. As the training data does not have enough instances to train the CNN encoder from scratch, CNN just achieves an F1 score of 69.35%. However, the pre-training models including BERT and ERNIE increase the F1 score by at least 15%. (2) ERNIE achieves an absolute F1 increase of 3.4% over BERT, which means fusing external knowledge is very effective. In TACRED, there are nearly 80% null instances so that we follow the previous work Zhang et al. The results of CNN, PA-LSTM, and C-GCN come from the paper by \newcitezhang2018graph, which are the best results of CNN, RNN, and GCN respectively. The entity mask strategy refers to replacing each subject (and object similarly) entity with a special NER token, which is similar to our proposed pre-training task dEA. (2) ERNIE achieves the best recall and F1 scores, and increases the F1 of BERT by nearly 2.0%, which proves the effectiveness of the knowledgeable module for relation classification.