EARL: Joint Entity and Relation Linking for Question Answering over Knowledge Graphs
Table 5: Evaluating EARL’s Entity Linking performance
['[BOLD] System', '[BOLD] Accuracy LC-QuAD', '[BOLD] Accuracy - QALD']
EARL uses a series of sequential modules with little to no feedback across them. Hence, the errors in one module propagate down the line. To trammel this, we implement an adaptive approach especially for curbing the errors made in the pre-processing modules. While conducting experiments, it was observed that most of the errors are in the shallow parsing phase, mainly because of grammatical errors in LC-QuAD which directly affects the consecutive E/R prediction and candidate selection steps. If the E/R prediction is erroneous, it will search in a wrong Elasticsearch index for probable candidate list generation. In such a case none of the candidates ∈ ci for a keyword would contain ¯ci as is reflected by the probabilities assigned to ci by the re-ranker module. If the maximum probability assigned to ci is less than a very small threshold value, empirically chosen as 0.01, we re-do the steps from ER prediction after altering the original prediction. Aim: To evaluate the performance of EARL with other state-of-the-art systems on the entity linking task. This also evaluates our hypothesis H3. Metrics: We are reporting the performance on accuracy. Accuracy is defined by the ratio of the correctly identified entities over the total number of entities present. Result: The value of k is set to 30 while re-ranking and fetching the most probable entity.