Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision (Short Version) The long version with more details can be found at https://arxiv.org/abs/1611.00020.
Table 2: Comparison to previous state-of-the-art, average F1 is the main evaluation metric. Our model achieves better results without hand-crafted rules and feature engineering.
['Model', 'Avg. Prec.', 'Avg. Rec.', 'Avg. F1', 'Acc.']
We evaluate performance using the offical measures for WebQuestionsSP. Because the answer to a question can contain multiple entities or values, precision, recall and F1 are computed based on the output for each individual question. The average F1 score is reported as the main evaluation metric. The accuracy@1 measures the percentage of questions that are answered exactly. Besides the better performance, our model does not rely on domain-specific rules or feature engineering. More ablation studies and analysis are included in the long version.