Syntactically Look-Ahead Attention Network for Sentence Compression
Table 1: F1 scores for Base with various features in the development data. The bold score represents the highest score.
['Glove ELMo', '✓', '✓', '✓', '✓', '✓', '✓', '[EMPTY]']
(cased_L-12_H-768_A-12) as our features. Following this result, we used the combination of all of GloVe, ELMo and BERT for all models. Recently, \citeauthorlin-etal-2019-unified \shortcitelin-etal-2019-unified reported that ELMo is better than BERT in sentence-level discourse parsing and \citeauthorakbik-etal-2019-pooled \shortciteakbik-etal-2019-pooled reported that LSTM with GloVe is better than BERT in named entity recognition. As \citeauthorclarke-lapata-2007-modelling \shortciteclarke-lapata-2007-modelling discussed, discourse and named entity information are both important in the sentence compression task. Therefore, our observation is consistent with the previous researches. These observations indicate that the best choice of word embedding types depends on a task.