Learning to Ask Questions in Open-domain Conversational Systems with Typed Decoders
Table 2: Annotation results. Win for “A vs. B” means A is better than B. Significance tests with Z-test were conducted. Values marked with ∗ means p-value < 0.05, and ∗∗ for p-value < 0.01.
['Models', 'Appropriateness Win (%)', 'Appropriateness Lose (%)', 'Appropriateness Tie (%)', 'Richness Win (%)', 'Richness Lose (%)', 'Richness Tie (%)', 'Willingness Win (%)', 'Willingness Lose (%)', 'Willingness Tie (%)']
The label of each pair-wise comparison is decided by majority voting from five annotators. This demonstrates that our decoders produce more appropriate questions, with richer topics. Particularly, our decoders have substantially better willingness scores, indicating that questions generated by our models are more likely to elicit further interactions. Noticeably, HTD outperforms STD significantly, indicating that it is beneficial to specify word types explicitly and apply dynamic vocabularies in generation.