An Attention-Based Word-Level Interaction Model: Relation Detection for Knowledge Base Question Answering
Table 3: Ablation experiment results
['Analysis content', 'Model', 'SimpleQA', 'WebQA']
First, both tasks benefit from both the bi-directional attention mechanism, especially for WebQA dataset (84.44% vs 78.41%). Attention mechanism enables the model to focus on important parts of the sequence and get a better representation. Second, comparing the sequences using word-level representations also outperforms the traditional encoding-comparing model, because multiple comparison operations are performed before information in the question is merged into a single vector. Finally, ABWIM, which combines the bi-directional attention and word-level interaction mechanisms, achieves state-of-the-art results on both tasks with accuracy 93.5% and 85.32% respectively. First, ABWIM outperforms several strong baselines, though there is no preprocessing operation. It indicates the effectiveness of learning attention-based word alignments between question and relation. Second, a context aware representation of words can further improve the result and the preprocessing layer is also important for the overall performance. The LSTM layer, which is capable of learning long range dependency, outperforms the CNN and gated liner transformation. Finally, CNN and gated linear transformation do not rely on the computations of previous timestep, so they can fully utilize the computational capability of GPU and are more faster to be trained and perform inference.