Can Neural Generators for Dialogue Learn Sentence Planning and Discourse Structuring?
Table 4: Distribution of Training Data
['[EMPTY]', '[EMPTY]', '[BOLD] Number of Periods  [BOLD] 1', '[BOLD] Number of Periods  [BOLD] 2', '[BOLD] Number of Periods  [BOLD] 3', '[BOLD] Number of Periods  [BOLD] 4', '[BOLD] Number of Periods  [BOLD] 5', '[BOLD] Number of Periods  [BOLD] 6', '[BOLD] Number of Periods  [BOLD] 7']
For sentence scoping, we construct a training set of 64,442 output/ MR pairs and a test set of 398 output/ MR pairs where the reference utterances for the outputs are generated from personage. The right frontier of the table shows that there are low frequencies of training instances where each proposition in the MR is realized in its own sentence (Period = Number of MR attrs -1). The lower left hand side of the table shows that as the MRs get longer, there are lower frequencies of utterances with Period=1. This experiment results in an 84% accuracy (with correlation of 0.802 with the test refs, p≤0.01). When analyzing the mistakes, we observe that all of the scoping mistakes the model makes (31 in total) are the case of period=N-1. Thus while the period supervision improves the model, it still fails on cases where there were few instances in training.