A Deep Ensemble Model with Slot Alignment for Sequence-to-Sequence Natural Language Generation
Table 6: Automatic metric scores of different models tested on the E2E dataset, both unmodified (¯¯¯s) and augmented (s) through the utterance splitting. The symbols † and ‡ indicate statistically significant improvement over the ¯¯¯s counterpart with p<0.05 and p<0.01, respectively, based on the paired t-test.
['[BOLD] LSTM', '¯¯¯ [BOLD] s', '[BOLD] BLEU 0.6664', '[BOLD] NIST 8.0150', '[BOLD] METEOR 0.4420', '[BOLD] ROUGE 0.7062']
In the first experiment, we assess what effect the augmenting of the training set via utterance splitting has on the performance of different models. This can likely be attributed to the model having access to more granular information about which parts of the utterance correspond to which slots in the MR. This may assist the model in sentence planning and building a stronger association between parts of the utterance and certain slots, such as that “it” is a substitute for the name. The scores here correspond to the model submitted to the E2E NLG Challenge.