Template-free Data-to-Text Generation of Finnish Sports News
Table 4: Automatic evaluation metrics on hockey corpus test set.
['BLEU', '0.1967']
We evaluate on event level using gold standard event selection, where each generated event description is compared to its existing reference text. As the model is trained to produce a tokenized sequence, we apply a detokenizer to be able to compare against the original untokenized reference text. On the test set, the model achieves a BLEU score of 19.67. To the extent that different datasets allow comparison, the best reported score on the Rotowire basketball news corpus is 16.50 Puduppully et al. Compared to our earlier E2E baseline experiment, we score lower than our closest comparable reference of 31.90 (with single references), which is understandable due to the much smaller train set size for the hockey corpus (about 13% in size).