Learning an Executable Neural Semantic Parser
Table 6: Fully supervised experimental results on the GeoQuery dataset. For Jia and Liang (2016), we include two of their results: one is a standard neural sequence to sequence model; and the other is the same model trained with a data augmentation algorithm on the labeled data (reported in parentheses).
['Models', 'Accuracy']
The first block contains conventional statistical semantic parsers, previously proposed neural models are presented in the second block, whereas variants of tnsp are shown in the third block. We report accuracy which is defined as the proportion of utterances which correctly parsed to their gold standard logical forms. Amongst tnsp models, a top-down system with structured (soft) attention performs best. Overall, we observe that differences between top-down and bottom-up systems are small; it is mostly the attention mechanism that affects performance, with hard attention performing worst and soft attention performing best for both top-down and bottom-up systems. tnsp outperforms previously proposed neural semantic parsers which treat semantic parsing as a sequence transduction problem and use LSTMs to map utterances to logical forms Dong and Lapata tnsp brings performance improvements over these systems when using comparable data sources for training.