User Evaluation of a Multi-dimensional Statistical Dialogue System
Table 3: Test results on simulated data (same error rates as in training): task success rate (SuccRate), average dialogue length (AvgLen), average reward (AvgRew).
['[BOLD] system', '[BOLD] SuccRate', '[BOLD] AvgLen', '[BOLD] AvgRew']
To get a better picture of what we might expect during the human evaluation, we first ran evaluations with simulated data. As we hypothesised, the scores are very similar, the one-dim system only slightly outperforming the multi-dimensional systems.