Generating Wikipedia by Summarizing Long Sequences
Table 2: Percentiles for different aspects of WikiSum dataset. Size is in number of words.
['[BOLD] Percentile', '[BOLD] 20', '[BOLD] 40', '[BOLD] 50', '[BOLD] 60', '[BOLD] 80', '[BOLD] 100']
Many articles have few citations, motivating our supplementation of the source documents with web search results. On the other hand, citations when available, tend to be of higher-quality. When counting the total words in the entire dataset, it is orders-of-magnitude larger than previous summarization datasets. Because the amount of text in input reference documents (Ci,Si) can be very large (see it is infeasible to train an end-to-end abstractive model given the memory constraints of current hardware. Hence, we first coarsely select a subset of the input using extractive summarization. The second stage involves training an abstractive model that generates the Wikipedia text while conditioning on this extraction. This two-stage process is inspired by by how humans might summarize multiple long documents: First highlight pertinent information, then conditionally generate the summary based on the highlights.