Content based Weighted Consensus Summarization
Table 1: System performance comparison
['System', 'DUC 2003 R-1', 'DUC 2003 R-2', 'DUC 2003 R-4', 'DUC 2004 R-1', 'DUC 2004 R-2', 'DUC 2004 R-4']
The DUC 2003 and DUC 2004 datasets were used for evaluating the experiments. We report ROUGE-1, ROUGE-2 and ROUGE-4 recall. We use three baseline aggregation techniques against which the proposed method is compared. Besides Borda Count and WCS, we also compare the results with the choose-best Oracle technique. In case of the Oracle method, we assume that the performance of each candidate system, in terms of ROUGE score, is known to us. For each document, we directly select summary generated by the system that scored highest for that particular document and call it the meta-summary. This is a very strong baseline, and average ROUGE-1 score for this meta-system, on the DUC 2003 dataset, was 0.394 compared to a maximum ROUGE-1 of 0.357 for the best performing LexRank system.