Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence
Table 2: Automatic and human evaluation results on E2E dataset. SLUG, DANGNT, TUDA and N_TEMP are from previous works and the other models are our own implementations.
['[BOLD] Metrics  [BOLD] Models', '[BOLD] Word Overlap  [BOLD] BLEU', '[BOLD] Word Overlap  [BOLD] R-L', '[BOLD] Word Overlap  [BOLD] Meteor', '[BOLD] Word Overlap  [BOLD] CIDEr', '[BOLD] Human Evaluation  [BOLD] Fluent', '[BOLD] Human Evaluation  [BOLD] Wrong', '[BOLD] Human Evaluation  [BOLD] Repeat', '[BOLD] Human Evaluation  [BOLD] Miss', '[BOLD] Diversity  [BOLD] Dist-1', '[BOLD] Diversity  [BOLD] Dist-3']
On the E2E dataset, apart from our implementations, we also compare agianst outputs from the SLUG Juraska et al. SLUG uses a heuristic slot aligner based on a set of handcrafted rules and combine a complex pipeline of data augmentation, selection, model ensemble and reranker, while our model has a simple end-to-end learning paradigm with no special delexicalizing, training or decoding tricks. Seq2seq-based models are more diverse than rule-based models at the cost of higher chances of making errors. As rule-based systems are by design always faithful to the input information, they made zero wrong facts in their outputs. Most models do not have the fact repetition issue because of the relatively simple patterns in the E2E dataset. therefore, adding the (+R) constraint only improves the performance minorly. The (+RM) constraint reduces the number of information missing to 3 without hurting the fluency. All the 3 missing cases are because of the wrong alignment between the period and one data record, which can be easily fixed by defining a simple rule. As also noted by the author, it trades-off the generation quality for interpretability and controllability. In contrast, our model, despite relying on no heuristics or complex pipelines, made zero wrong facts with the lowest information missing rate, even surpassing rule-based models. It also maintains interpretable and controllable without sacrificing the generation quality.