Hashing based Answer Selection
Table 6: Comparison of accuracy, time cost and memory cost on insuranceQA. Each question has 500 candidate answers. “Memory Cost ⋄ ” is the memory cost for storing representations of answers.
['Model', 'P@1 (Test1)', 'P@1 (Test2)', 'Time Cost per Question', 'Memory Cost ⋄']
All experiments are run on a Titan XP GPU. BERT-pooling can directly store the vector representations of answers with a low memory cost, which doesn’t have the time cost and memory cost problem. But the accuracy of BERT-pooling is much lower than BERT-attention and HAS. BERT-attention (recal.) denotes a BERT-attention variant with recalculation, and BERT-attention (store) denotes a BERT-attention variant which stores the matrix representations of answers in memory. BERT-attention (recal.) does not need to store the matrix representations of answers in memory, and BERT-attention (store) does not need recalculation. The time cost is 4.19 seconds per question for BERT-attention (recal.), which is 15 times slower than HAS. Although BERT-attention (store) has low time cost as that of HAS, the memory cost of it is 14.29 GB, which is 32 times larger than that of HAS.