Massive Multi-Document Summarization of Product Reviews with Weak Supervision
Table 5: ROUGE F1 scores on variants of our model and the baselines on two of the categories. The model variant name indicates the hierarchical level from which the output summary is taken (level-1 or top) and the the metric used for weak reference extraction. ROUGE score intervals express ≥95% confidence.
['[EMPTY]', '[BOLD] Model', '[BOLD] Electronics  [BOLD] ROUGE-1', '[BOLD] Electronics  [BOLD] ROUGE-1', '[BOLD] Electronics  [BOLD] ROUGE-2', '[BOLD] Electronics  [BOLD] ROUGE-2', '[BOLD] Electronics  [BOLD] ROUGE-L', '[BOLD] Electronics  [BOLD] ROUGE-L', '[BOLD] Books  [BOLD] ROUGE-1', '[BOLD] Books  [BOLD] ROUGE-1', '[BOLD] Books  [BOLD] ROUGE-2', '[BOLD] Books  [BOLD] ROUGE-2', '[BOLD] Books  [BOLD] ROUGE-L', '[BOLD] Books  [BOLD] ROUGE-L']
We first observe that applying the full summarization hierarchy (top) is almost consistently worse than choosing a medoid summary from the first level (level1). This could be explained by the fact that details are lost on the way up the hierarchy levels, causing the final summary to capture more generic common information. Additionally, clusters of summaries at higher levels in the summary hierarchy may contain elements with low pairwise similarity, quite different from the clusters that were used for training the CSS.