Query-based Attention CNN for Text Similarity Map
Table 2: Experiment result
['Models', 'dev set', 'test set']
In this experiment,we focused on the difference between one-stage QACNN and two-stage QACNN. For one-stage QACNN, we didn’t split an entire passage into sentences. That is, the shape of passage-query similarity map PQ and passage-choice similarity map PC are 2D rather than 3D. We convolved them directly on word-level and output passage feature without second-stage involved. The result shows that the modified one staged QACNN reaches 66.8% accuracy on validation set, which is ten percent lower than 78.1%, the original QACNN accuracy on validation set. However, this modified model would have a deficiency of query information. Therefore, we concatenated the final output representation of PQ and PC together before prediction layer. The result is almost ten percent less than the original one. 3) For the last one, instead of removing sentence-level attention, we removed word-level attention from QACNN Layer. We can see that QACNN(with only word-level attention) performs better than QACNN(without attention); QACNN(with only sentence-level attention) performs better than QACNN(with only word-level attention); And original QACNN which contains both word-level and sentence-level attention does the best job among all. Thus, not only word-level attention but also sentence-level attention can contribute to the performance of QACNN. However, sentence-level attention seems to play a more important role.