Searching for Effective Neural Extractive Summarization: What Works and What’s Next
Table 5: Results of different architectures with different pre-trained knowledge on CNN/DailyMail, where Enc. and Dec. represent document encoder and decoder respectively.
['[BOLD] Model  [BOLD] Dec.', '[BOLD] Model  [BOLD] Enc.', '[BOLD] R-1  [BOLD] Baseline', '[BOLD] R-2  [BOLD] Baseline', '[BOLD] R-L  [BOLD] Baseline', '[BOLD] R-1  [BOLD] + GloVe', '[BOLD] R-2  [BOLD] + GloVe', '[BOLD] R-L  [BOLD] + GloVe', '[BOLD] R-1  [BOLD] + BERT', '[BOLD] R-2  [BOLD] + BERT', '[BOLD] R-L  [BOLD] + BERT', '[BOLD] R-1  [BOLD] + Newsroom', '[BOLD] R-2  [BOLD] + Newsroom', '[BOLD] R-L  [BOLD] + Newsroom']
As shown in Tab. However, when the models are equipped with BERT, we are excited to observe that the performances of all types of architectures are improved by a large margin. Specifically, the model CNN-LSTM-Pointer has achieved a new state-of-the-art with 42.11 on R-1, surpassing existing models dramatically.