Multi-Field Structural Decomposition for Question Answering
Table 1: Results from our question-answering system on 8 types of questions in the bAbI tasks.
['Type', 'Lexical  [ITALIC] λ=1', 'Lexical  [ITALIC] λ=1', 'Lexical  [ITALIC] λ is learned', 'Lexical  [ITALIC] λ is learned', 'Lexical + Syntax  [ITALIC] λ=1', 'Lexical + Syntax  [ITALIC] λ=1', 'Lexical + Syntax  [ITALIC] λ is learned', 'Lexical + Syntax  [ITALIC] λ is learned', 'Lexical + Syntax + Semantics  [ITALIC] λ=1', 'Lexical + Syntax + Semantics  [ITALIC] λ=1', 'Lexical + Syntax + Semantics  [ITALIC] λ is learned', 'Lexical + Syntax + Semantics  [ITALIC] λ is learned']
The map and mrr show clear correlation with respect to the number of active fields. For the majority of tasks, using only the lexical fields does not perform well. The fictional stories included in this data often contain multiple occurrences of the same lexicons, and the lexical fields alone are not able to select the correct answer. Significantly lower accuracy for the last task is due to a fact that besides an answer is located within a single sentence, multiple passages for the single question are required to correctly locate the sentence with the answers. Lexical fields coupled with only syntactic fields do not perform much better. It may be due to a fact that the syntactic fields containing ordinary dependency labels do not provide sufficient context-wise information so that they do not generate enough features for statistical learning to capture specific characteristic of the context. The significant improvement, however, is reached when the semantics fields are added as they provide deeper understanding of the context.