How Far is It From Samples to Test Time?
Table 5: Performance of three seq2seq models trained on different samples, evaluated on 2004 data.
['[EMPTY]', 'rouge-1', 'rouge-2', 'rouge-l']
Sampling training data randomly or taking the most recent period produces models with similar ROUGE scores, both much better than the identity baseline. However, about half of the gap to the identity baseline disappears when older training data is taken. In the §A.1, we give an example of temporal drift in the Headlines data: US largely replaces U.S. in the newer training set and the test set.