Question Answering from Unstructured Text by Retrieval and Comprehension
Table 3: Performance of retrieval methods. (WikiMovies-WE)
['[BOLD] Model Type', '[BOLD] R@1', '[BOLD] R@10', '[BOLD] R@30', '[BOLD] R@100', '[BOLD] P@1', '[BOLD] P@10', '[BOLD] P@30', '[BOLD] P@100']
We evaluate the retrieval models based on precision and recall of the oracle articles. The evaluation is done on the test set. R@k is the ratio of cases where the highest ranked oracle article is in the top k retrieved articles. P@k is the ratio of oracle articles which are in the top k retrieved results. We can see that both (r1) and (R2) significantly outperform (r0), with (R2) doing slightly better. We emphasize that (R2) uses no domain specific knowledge, and can be readily applied to other datasets where articles may not be about specific types of entities.