Data Augmentation for Copy-Mechanism in Dialogue State Tracking
Table 2: F1 scores on three datasets. The synthetic training/test sets are constructed using DC(D,1,1) with the same set of slot values.
['[EMPTY]', 'dataset', 'original test', 'synthetic test']
We guess that the attention based copy-mechanism is inclined to memorize values that appear in the train set rather than infer them from contexts. To verify our argument, we conduct four groups of experiments using original and synthetic datasets. The former corresponds to the case of seen slot values, and the latter corresponds to the case of unseen slot values. Since we do not change the contexts of slot values, we conclude that the model attempts to memorize the slot values rather than infer them from contexts. Interestingly, the model shows pretty good performances in the case that training/test set are both synthetic, which confirms that it is reasonable to use random strings to augment datasets.