Citation Text Generation
Table 2: Automatic evaluation of generated texts. Differences between entries in bold are statistically significant.
['[EMPTY]', '[BOLD] BLEU', '[BOLD] Rouge-1', '[BOLD] Rouge-2', '[BOLD] Rouge-L', '[BOLD] SciBERTScore']
We find that context does make a difference for this category of model, and that models which have access to the intro of the documents outperform those which use abstracts or sampling. This table shows that the retrieval methods perform well on this task. We also observe that tuning the α and β parameters on the validation set results in overfitting for this method. Outputs are largely unchanged by this tuning; fewer than 400 test datapoints differ from the untuned outputs. A larger validation split may alleviate this problem.