Multi-Document Summarization with Determinantal Point Processes and Contextualized Representations
Table 1: Position of summary-worthy sentences in a document for single-doc (CNN/DM) and multi-doc datasets (DUC-04, TAC11). ‘pos’ are summary-worthy document sentences; ‘neg’ are sentences that are randomly sampled from the same document.
['CNN/DM', 'mean', 'min', 'max']
The embedding layer, illustrated in Fig. These embeddings are element-wisely added up then fed to the model. The sentence position embeddings are incorporated in this work to capture the position of a sentence in the article. It is utilized only by BERT-imp, as position matters for sentence importance but not quite so for pairwise similarity. (see § Further, ground-truth summary sentences of the DUC and TAC datasets are likely to appear among the first five sentences of an article, indicating position embeddings are crucial for training the BERT-imp model.