Dialog System Technology Challenge 7
Table 2: Performance comparison between the baseline and the best system.
['System', 'BLEU-4', 'METEOR', 'CIDEr', 'Human rating']
The AVSD Task received 31 system submission from 9 teams. We built a baseline end-to-end dialog system that can generate answers in response to user questions about events in a video sequence as described in Hori et al. Our architecture is similar to the Hierarchical Recurrent Encoder in Das et al. The question, visual features, and the dialog history are fed into corresponding LSTM-based encoders to build up a context embedding, and then the outputs of the encoders are fed into an LSTM-based decoder to generate an answer. The history consists of encodings of QA pairs. We feed multimodal attention-based video features into the LSTM encoder instead of single static image features. The systems submitted deployed LSTM, BLSTM, and GRU with cross entropy as the objective function. Under this evaluation, the human rating for the original answers was 3.938.