Multimodal Dialogue State Tracking By QA Approach with Data Augmentation
Table 1: Model Performance. Models are ranked by an overall performance rather than any single metrics.
['[EMPTY]', 'Model', 'BLEU-1', 'BLEU-2', 'BLEU-3', 'BLEU-4', 'METEOR', 'ROUGE-L', 'CIDEr']
In order to further evaluate our model’s performance, we compare our results with the baseline model; as well as other participants in the DSTC7 challenge on the track of AVSD under two different settings – with and without video related modalities (i3d-flow, i3d-rgb and audio). Our best model fully utilizes the combination of teacher forcing, max pooling, BiGRU and data augmentation techniques. We hypothesize that as we regard the dialogue history as a group of discrete QA pairs, we miss certain inner temporal dependencies, impacting tasks like coreference resolution. While our data demonstrates that the ordering of the dialog history as a whole may not contribute as much information as previously assumed, it is important to consider that phenomena like coreference resolution most likely does depend on temporal location. Fig. Another example is in the case of an ambiguous pronoun. Both situations usually rely on the presence of a previous noun or a certain portion in an image or video. After such information has been deduced, our data suggests that the order of that processed information would likely have little effect on the performance of the system. However, without doing this before shuffling, information that can be obtained through techniques such as coreferencing may be untapped.