DART: Open-Domain Structured Data Record to Text Generation
Table 4: Model results on the test set of DART v1.0.0b. ↑: Higher is better. ↓: Lower is better. BART performs the best due to its generalization ability from pretraining.
['[EMPTY]', 'BLEU ↑', 'METEOR ↑', 'TER ↓', 'MoverScore ↑', 'BERTScore(F1) ↑', 'BLEURT ↑']
Dart The BART model has the highest performance among three models with a BLEU score of 37.06. This is attributed to BART’s generalization ability due to pretraining. However, language models such as BART are pretrained by reconstructing text and, as a result, we found that their output on DART often contains hallucinated words Parikh et al. ; Harkous et al. In addition, while the pretrained language model shows better text generation quality due to its generalization ability from pretraining, it does not fully capture the hierarchical ontology nature of the triple sets in their linearized input. Furthermore, the end-to-end transformer has the lowest performance since the transformer model needs intermediate pipeline planning steps to have higher performance. Similar findings can be found in Castro Ferreira et al.