Closed-Book Training to Improve Summarization Encoder Memory
Table 7: ROUGE F1 and METEOR scores of sanity check ablations, evaluated on CNN/DM validation set.
['[EMPTY]', 'ROUGE 1', 'ROUGE 2', 'ROUGE L']
Model Capacity: To validate and sanity-check that the improvements are the result of the inclusion of our closed-book decoder and not due to some trivial effects of having two decoders or larger model capacity (more parameters), we train a variant of our model with two duplicated (initialized to be different) attention-pointer decoders. We also evaluate a pointer-generator baseline with 2-layer encoder and decoder (pg-2layer) and increase the LSTM hidden dimension and word embedding dimension of the pointer-generator baseline (pg-big) to exceed the total number of parameters of our 2-decoder model (34.5M versus 34.4M parameters). Moreover, as shown in Sec.