DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension
Table 1: Comparison between various RC datasets
['[BOLD] Metrics for Comparative Analysis', '[BOLD] Movie QA', '[BOLD] NarrativeQA over plot-summaries', '[BOLD] Self-RC', '[BOLD] Paraph-raseRC']
Fig. We use NER and noun phrase/verb phrase extraction over the entire dataset to identify key entities in the question, plot and answer which are in turn used to compute the metrics mentioned in the table. The metrics “Avg word distance” and “Avg sentence distance” indicate the average distance (in terms of words/sentences) between the occurrence of the question entities and closest occurrence of the answer entities in the passage. “Number of sentences for inferencing” is indicative of the minimum number of sentences required to cover all the question and answer entities. It is evident that tackling ParaphraseRC is much harder than the others on account of (i) larger distance between the query and answer, (ii) low word-overlap between query & passage, and (iii) higher number of sentences required to infer an answer.