Large Scale Question Answering using Tourism Data
Table 5: Performance of different systems including the CsrQA model on our task. Accuracy reported in %.
['[BOLD] Method', '[BOLD] Acc@3', '[BOLD] Acc@5', '[BOLD] Acc@30', '[BOLD] MRR']
We find that all non-neural baselines perform poorly on the task. Even the strong baseline of BM25 retrieval, which is commonly used in retrieval tasks, is not as effective for this dataset. Methods such as BM25 are primarily aimed at addressing challenges of semantic gap while in our task, answers require reasoning over subjective opinions in entity documents. We also observe that the performance of the neural model, Review-AVG, is comparable to that of BM25.