Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
Table 2: Comparison of Transformer decoder and GRU RNN decoder within the conversation model HERD (Serban et al., 2016) for response generation, on the Switchboard dataset (Zhao et al., 2017).
['[BOLD] Task: Conversation generation  [BOLD] Metrics', '[BOLD] Task: Conversation generation  [BOLD] HERD-GRU', '[BOLD] Task: Conversation generation  [BOLD] HERD-Transformer']
The second task is to generate a response given conversation history. Similar as above, we study the performance of a Transformer decoder in comparison with a more conventional GRU RNN decoder. Again, we see that the Transformer model generalizes well to the conversation generation setting, and consistently outperforms the GRU RNN counterpart.