Keeping Notes: Conditional Natural Language Generationwith a Scratchpad Mechanism
Table 3: The middle third of the table contains the end-to-end models performing the best from Gehrmann et al. (2018), while the bottom section contains the current state-of-the-art which involves a 2-step training process and is not end-to-end. Scratchpad establishes a state of the art for end-to-end models on summarization without Reinforcement Learning on ROUGE, while remaining competitive with see2017get on METEOR. Additionally, Scratchpad does not use an auxiliary loss as in see2017get or the middle third of the table. Gehrmann2018BottomUpAS do not evaluate on METEOR.
['Model', 'Rouge 1', 'Rouge 2', 'Rouge L', 'Meteor exact match', 'Meteor +stem/syn/para']
We report F1 scores for ROUGE-1, ROUGE-2, and ROUGE-LCS (measuring word, bigram, and longest-common-subsequence overlap, respectively) and we report METEOR in exact and full mode. We compare to the pointer-generator baseline and the coverage variant introduced by \newcitesee2017get. \newcitesee2017get use a multi-step training procedure for the coverage component to improve performance where a pointer-generator model is first trained without the coverage component for a large number of iterations, then trained with the component and a tuned auxiliary coverage loss, finding that the auxiliary loss and pre-training the network without coverage are required to improve performance. As demonstrated in Tab. We reach this performance with half of the training iterations, no pretraining, and without the additional memory outlay and model complexity of including an auxiliary loss.