RankQA: Neural Question Answering with Answer Re-Ranking
Table 2: Exact matches of RankQA compared to DrQA as natural baseline without re-ranking and state-of-the-art systems for neural QA. We use a general model that is trained on all datasets, and a task-specific model that is trained individually for every dataset. The two best results for every dataset are marked in bold.
['[EMPTY]', '[BOLD] SQuADOPEN', '[BOLD] CuratedTREC', '[BOLD] WebQuestions', '[BOLD] WikiMovies']
Tbl. The DrQA system Chen et al. Furthermore, we compare ourselves against other extensions of the DrQA pipeline such as the Paragraph Ranker Lee et al. Finally, we compare against other state-of-the-art QA pipelines, namely, R3 Wang et al. Context system from Min et al. For RankQA, we use, on the one hand, a general model that is trained on all four datasets simultaneously. On the other hand, we account for the different characteristics of the datasets and thus employ task-specific models that are trained separately on every dataset. Finally, in the last row of Tbl. This system represents an upper bound of the degree to which re-ranking could improve results without changing the information retrieval or machine comprehension models.