E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT
Table 4: RC macro precision, recall and F1 (%).
['[EMPTY]', 'dev set P', 'dev set R', 'dev set F1', 'test set P', 'test set R', 'test set F1']
S5SS1SSS0Px4 Results and discussion. Recall that ERNIE required additional encoder pretraining to achieve this result. Interestingly, E-BERT-replace (which is entity-only) beats original BERT (which is surface-form-only) , i.e., aligned entity vectors seem to be more useful than entity names for this task. The drop in F1 from E-BERT to Wikipedia2Vec-BERT shows the importance of vector space alignment.