Tree-Structured Neural Machinefor Linguistics-Aware Sentence Generation
Table 1: The results from human judgment.
['[BOLD] Models', 'Level-1%', 'Level-2%', 'Level-3%', 'Agreement', 'BLEU']
For Seq2Seq, NRM and X2Tree, the agreement value is in a range from 0.6 to 0.8 which is interpreted as “substantial agreement”. Meanwhile, EncDec and ATT obtain a relatively higher kappa value between 0.8 to 1.0 which is “almost perfect agreement”. Hence, we believe the labeling standard is considered clear which leads to high agreement among labelers. It shows that responses generated by the tree-structured decoder are more grammatical than those from the chain-structured decoders and demonstrate the X2Tree’s robustness to parser errors. Additionally, X2Tree and EncDec achieve best grammatical ratio (99.56%), but EncDec fails in generating relevant responses. Hence, Tree Decoder can improve the response relevance in experiments. We conjecture the reason is that X2Tree firstly generate the core verb of the responses. The first generated may be more relevant to the post and makes the whole response more relevant to the post. We conjecture that the tree-structured decoder is easier to learn because its hidden states need to store less information than their counterparts in a chain-structured decoder.