IDEL: In-Database Entity Linking with Neural Embeddings
Table 2. Accuracy in Precision@k of the trained model for each entity type. In test we observe for all entity types, except City and ComicsCharacter, a very high Precision@1>0.80. Even for a cold start scenario (columns unseen) we can still observe decent Precision@1>0.60 for all entity types, except Airports and ComicsCharacter. In addition, we included columns Prec@5 and Prec@10 to illustrate that our system de facto can retrieve the correct matching entities at lower ranks. Recall is not shown in this diagram, since we measure precision numbers for all entities in our test data set.
['[EMPTY]', 'Prec@1 test', 'Prec@1 train', 'Prec@1 unseen', 'Prec@5 test', 'Prec@5 train', 'Prec@5 unseen', 'Prec@10 test', 'Prec@10 train', 'Prec@10 unseen']
We observe high values (≥0.80) during testing Precsion@1 for all entity types, except City (0.73) and ComicsCharacter (0.76). This indicates that our system can return the correct entity linking with very high precision. If we measure the effectiveness of our system at Precsion@5, we observe that our system returns the correct result for each entity, independent of the type, and with a high accuracy of ≥0.93. We report an accuracy of ≥0.95 at Precision@10 for all entity types with a perfect result for entity type university. Note that the columns train denote “ideal systems” for the given training data. We observe that even an ideal system fails in rare cases for Astronaut, WrittenWork, City, Food and Airport.