A Probabilistic Generative Grammar for Semantic Parsing
Table 2: Precision and recall of the parsers evaluated on the out-of-knowledge base dataset. The uniform prior places equal probability mass on all semantic statements. The “type-correct” prior places higher mass (+4 in log probability) on semantic statements with subject and object types that agree with the domain and range of the relation predicate. The “knowledge base prior” is similar to the type-correct prior, except that it additionally places higher probability mass on semantic statements that correspond to true beliefs in the NELL knowledge base (additional +4 in log probability).
['[ITALIC] simple relation extraction', '[BOLD] precision', '[BOLD] recall', '[BOLD] F1']
Our parser is trained with the simple grammar in the simple relation extraction task, and with the grammar that models verb morphology in the temporally-scoped relation extraction task. As expected, the more informative priors do not uniformly improve parsing performance in this evaluation. Interestingly, the parser behaves more conservatively when incorporating the stronger priors, outputting a smaller set of the most confident responses, which results in higher precision and reduced recall. Our parser is indeed capable of extracting the correct semantic statements from sentences that refer to beliefs outside the knowledge base, and the use of the informative priors does not obviously hurt performance.