Gated Group Self-Attention for Answer Selection
Table 3. Comparison of group self-attention and local self-attention. All models use max-pooling for composition.
['Model (insuranceQA)', 'Test1', 'Test2', 'Seconds / Batch']
We have introduced that both group self-attention and local self-attention can be adopted in our GGSA model. Here, we compare these two self-attention methods in details. Local self-attention represents the model that a word in the sequence only attends to the words in the sliding window. The result of local self-attention is much lower than GGSA due to the lack of global information, which also proves the effectiveness of GGSA. We also incorporate the global information gate into local self-attention, which named local self-attention (+gate). The result of local self-attention (+gate) is comparable to GGSA, but with a much slower running speed. For efficiency consideration, we choose group self-attention as the key building block to design GGSA/iGGSA.