A dataset for resolving referring expressions in spoken dialogue via contextual query rewrites (CQR)
Table 10: Accuracy metrics for contextual query rewrite task
['[EMPTY]', 'train/dev/test', 'F1', 'Bleu']
We train two separate models drawing from different distributions: one of only gold reformulation data and the other including crowd-sourced reformulations. To quantify task complexity, we note that over the entire corpus, 67% of slots are carried over from dialogue to reformulation, an indication of the non-triviality of the task. For reproducibility, we use the open source neural sequence modeling system OpenNMT Klein et al. The only hyper-parameter changed from initial settings is to remove copy_loss_by_seqlength, which improves overall accuracy. The entity F1 scores is higher, as the model has seen many more variations around the carryover entities.