Tldr: Extreme Summarization of Scientific Documents
Table 3: Comparison of datasets. The lengths are in tokens. Target summary can be single (only one gold summary per document) or multi (multiple gold summaries for each document). Compression ratio shows the ratio of document length to summary length. {SciTldr}Full refers to SciTldr with full-text while {SciTldr}Abst only includes abstract and {SciTldr}AIC includes abstract, introduction and conclusion sections.
['Dataset', '# Docs', 'Doc length', 'Summ. length', 'Comp. ratio', 'Target']
There are a few key differences to note. First, SciTldr is a smaller dataset than existing datasets. Extreme summaries of scientific papers are significantly more difficult and expensive to collect because they require domain expertise to write and assess for quality. In contrast, other datasets such as XSUM, were collected automatically from the web. The next notable difference is the length of the documents and summaries. The compression ratio (ratio of document length to tldrs) is significantly higher compared to existing datasets, making it a more challenging summarization dataset. It is possible for models to only consider some sections of the paper as input (e.g., abstract or abstract + introduction + conclusion). This reduces the total document length and makes the task more manageable for models, at the cost of potentially missing some important information from other sections of the paper. We therefore define {SciTldr}Abst as a variant of the dataset that only has abstract as input while {SciTldr}AIC has abstract, introduction and conclusion sections, and in {SciTldr}Full, the entire paper is considered as input. Even when limiting the input to the abstract, introduction, and conclusion sections, the compression ratio of SciTldr is still significantly higher than other existing summarization datasets.