Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence
Table 3: Automatic and human evaluation results on WebNLG dataset. MELBOURNE and UPFUPF-FORGE are from previous works and the other models are our own implementations.
['[BOLD] Metrics  [BOLD] Models', '[BOLD] Word Overlap  [BOLD] BLEU', '[BOLD] Word Overlap  [BOLD] R-L', '[BOLD] Word Overlap  [BOLD] Meteor', '[BOLD] Word Overlap  [BOLD] CIDEr', '[BOLD] Human Evaluation  [BOLD] Fluent', '[BOLD] Human Evaluation  [BOLD] Wrong', '[BOLD] Human Evaluation  [BOLD] Repeat', '[BOLD] Human Evaluation  [BOLD] Miss', '[BOLD] Diversity  [BOLD] Dist-1', '[BOLD] Diversity  [BOLD] Dist-3']
We also include results from MELBOURNE, a seq2seq-based system achieving highest scores on automatic metrics in the WebNLG challenge and UPF-FORGE, a classic grammar-based system that wins in the human evaluation WebNLG contains significantly more distinct types of attributes than E2E, so the chance of making errors or repetitions increases greatly. Nevertheless, our model still performs on-par on automatic metrics with superior information adequacy and output diversity. The (+R) decoding constraint becomes important since the outputs in WebNLG are much longer than those in E2E, neural network models have problems tracking the history generation beyond certain range. Models might repeat facts that have been already generated long back before. The (+R) constraint effectively reduces the repetition cases from 19 to 2. These 2 cases are intra-segment repetitions and failed to be detected since our model can only track inter-segment constraints The (+RM) constraint brings down the information missing cases to 5 with slightly more wrong and repeated facts compared with (+R). Forcing models to keep generating until coveraging all records will inevitably increase the risk of making errors.