Contrastive Multi-document Question Generation
Table 1: Retrieval performance. “Out-Sample IR” refers to the evaluation data sample that consists of 10+10 documents D+ and D−. “Search-Engine Augmented IR” refers to augmenting the out-sample into 100 documents in total through Lucene.
['[BOLD] Model', '[BOLD] Out-Sample IR mAP', '[BOLD] Out-Sample IR RPrec', '[BOLD] Out-Sample IR MRR (=MRR@10)', '[BOLD] Out-Sample IR nDCG', '[BOLD] Search-Engine Augmented IR mAP', '[BOLD] Search-Engine Augmented IR RPrec', '[BOLD] Search-Engine Augmented IR MRR', '[BOLD] Search-Engine Augmented IR MRR@10', '[BOLD] Search-Engine Augmented IR nDCG']
In particular, it shows that policy gradient or set-induced contrastive regularization alone is effective in improving performance. The coordinator performs better when optimized for both policy gradient and regularization objectives.