Modelling Hierarchical Structure between Dialogue Policy and Natural Language Generator with Option Framework for Task-oriented Dialogue System
Table 3: The table shows the HRL results with beam search on MultiWoz 2.0 and MultiWoz 2.1.
['[BOLD] Reward Shapes', '[ITALIC] α', '[BOLD] Beam width = 1  [BOLD] Inform(%)', '[BOLD] Beam width = 1  [BOLD] Success(%)', '[BOLD] Beam width = 1  [BOLD] BLEU(%)', '[BOLD] Beam width = 1  [BOLD] Total', '[BOLD] Beam width = 2  [BOLD] Inform(%)', '[BOLD] Beam width = 2  [BOLD] Success(%)', '[BOLD] Beam width = 2  [BOLD] BLEU(%)', '[BOLD] Beam width = 2  [BOLD] Total', '[BOLD] Beam width = 5  [BOLD] Inform(%)', '[BOLD] Beam width = 5  [BOLD] Success(%)', '[BOLD] Beam width = 5  [BOLD] BLEU(%)', '[BOLD] Beam width = 5  [BOLD] Total']
Obviously, HRL gives an enormous improvement on the performance in comparison with the pretraining. Nevertheless, the pretraining is an essential part that cannot be replaced before training with reinforcement learning in task-oriented dialogue system. The possibility of training from scratch in HRL with a good performance for task-oriented dialogue system is left to be investigated.