The Neural Noisy Channel
Table 2: Overview of results on the abstractive sentence summarisation task. ABS+ (DBLP:conf/emnlp/RushCW15) is the attentive model with bag-of-words as the encoder. RAS-LSTM and RAS-Elman (chopra) are the sequence to sequence models with attention with the RNN cell implemented as LSTMs and an Elman architecture (elman1990finding), respectively. Pointing the unknown words (gulcehre2016pointing) uses pointer networks (vinyals2015pointer) to select the output token from the input sequence in order to avoid generating unknown tokens. ASC + FSC (miao2016) is the semi-supervised model based on a variational autoencoder.
['Model', '# Parallel data', '# Unpaired data', 'RG-1', 'RG-2', 'RG-L']
ABS+ (DBLP:conf/emnlp/RushCW15), RAS-LSTM and RAS-Elman (chopra) are different variations of the attentive models. Pointing the unkown words uses pointer networks (vinyals2015pointer) to select the output token from the input sequence in order to avoid generating unknown tokens. ASC + FSC (miao2016) is a semi-supervised model based on a variational autoencoder. Trained on 1m paired samples and 3.8m unpaired samples, the noisy channel achieves comparable or better results than (direct) models trained with 3.8m paired samples. Compared to miao2016, whose ASC + FSC models is an alternative strategy for using unpaired data, the noisy channel is significantly more effective — 33.21 versus 31.09 in ROUGE-1.