MatInf: A Jointly Labeled Large-Scale Dataset for Classification, Question Answering and Summarization
Table 7: Experimental results of baseline methods on MatInf-QA.
['Method', 'MatInf-QA R-1', 'MatInf-QA R-2', 'MatInf-QA R-L']
Question Answering. The high scores of Best Passage (maximum possible performance) indicate that using training data as a document set is completely feasible. Seq2Seq with Attention outperforms the retrieval-based baseline by a margin of 2.56 in terms of ROUGE-L. It suggests that a generation-based neural network can effectively learn from multiple relevant samples and generalize. Besides, since we do the matching between each question and every entry within the same class in the training set, the inference of BERT Matching takes quite a long time. Similar to MS-MARCO msmarco, it is possible to use a search engine (e.g., Elastic Search) to pre-filter the documents and reduce the computational cost. Meanwhile, MTF-S2S is effective on QA task and outperforms its single-task version by 0.74 on ROUGE-L.