EARL: Joint Entity and Relation Linking for Question Answering over Knowledge Graphs
Table 4: Evaluation of joint linking performance
['[BOLD] Value of k', 'R [ITALIC] f [BOLD]  based on R [ITALIC] i', 'R [ITALIC] f [BOLD]  based on C,H', 'R [ITALIC] f [BOLD]  based on R [ITALIC] i,C,H']
Metrics: We use the mean reciprocal rank of the correct candidate ¯ci for each entity/relation in the query. From the probable candidate list generation step, we fetch a list of top candidates for each identified phrase in a query with a k value of 10, 30, 50 and 100, where k is the number of results from text search for each keyword spotted. To evaluate the robustness of our classifier and features we perform two tests. i) ii) On the bottom half of Table 4 we artificially insert the correct candidate into each list to purely test re-ranking abilities of our system (this portion of the table contains k∗ as the number of items in each candidate list). We inject the correct uris at the lowest rank (see k∗), if it was not retrieved in the top k results from previous step. Results: In case correct URIs were missing in the candidate list, we inserted URIs artificially as the last candidate . The MRR then increased from 0.568 to 0.905.