A Pilot Study of Domain Adaptation Effect for Neural Abstractive Summarization
Table 1: Evaluation based on ROUGE-2 (R-2), ROUGE-L (R-L), and BLEU (multiplied by 100) for in-domain training.
['[EMPTY]', 'R-2', 'R-L', 'BLEU', 'Avg Len']
Effect of Pre-training with Extracts. We first evaluate whether pre-training can improve summarization performance for In-Domain setups, where we initialize model parameters by training on NYT-extract for about 20,000 iterations. Otherwise, parameters are randomly initialized. We also consider two baselines, Baseline1 outputs the first sentence, Basline2 selects the first 22 (news) and 15 (opinion) tokens (with similar lengths as human summaries). As can be seen, the pre-training step improves performance for news, whereas the performance on opinion remains roughly the same. This might be due to the fact that news abstracts reuse more words from input, which are closer to extractive summaries than opinion abstracts.