A Good Sample is Hard to Find: Noise Injection Sampling and Self-Training for Neural Language Generation Models
Table 3: Attribute realization errors on the E2E test set. The Slug model and our delexicalized models delexicalize the NAME and NEAR slots, thus making 0 errors on these attributes. DANGNT and TUDA models perform complete delexicalization.
['Model', 'Model', 'Model', 'Model', 'Name', 'Near', 'Family Friendly', 'Area', 'Customer Rating', 'Food', 'Price Range', 'Eat Type', 'All']
The automatic quality evaluations are somewhat limited, however. To gain more insight into model performance we apply our rule based parser to estimate attribute realization error for all system outputs on the test set, similarly to Dušek et al. (e.g., if the MR specifies food[French] , we check to make sure the generated utterance says so). Immediately, it is revealed that p0 is far worse than the baseline methods making 115 and 83 errors using greedy and beam decoding respectively.