A Corpus-free State2Seq User Simulator for Task-oriented Dialogue
Table 3: Evaluation of agent policy trained by different user model. Results above dash-line are from our model, which achieve best performance in most task.
['Model', 'Movie Booking Avg. Succ.', 'Movie Booking Avg. Rwd.', 'Movie Booking Avg. Turns', 'Restaurant Reservation Avg. Succ.', 'Restaurant Reservation Avg. Rwd.', 'Restaurant Reservation Avg. Turns']
The results show that the policy obtained by our model outperforms baselines. The results show our method could work consistently well in different domains. Most models score higher on data in the restaurant domain, because the field is relatively simpler and has fewer slots. The State2MLC model does not perform well. This is due to the fact that State2MLC model has a much simpler structure, so it is likely to overfit to generated data in a simple domain and limits the generalization ability of the policy.