Gated Group Self-Attention for Answer Selection
Table 7. The comparison of pairwise loss and pointwise loss on yahooQA.
['Model', 'P@1', 'MRR']
The performance of GGSA is also sensitive to loss functions. We have introduced two loss functions before, which named pairwise loss and pointwise loss respectively. Here, we design a comparison of these two loss functions on yahooQA. The outputs of models play different roles by using different loss functions. For pairwise loss, the models generate the matching score of question-answer pairs for ranking. While for pointwise loss, the models generate the probability of current answer corresponding to the question. We can find that pairwise loss is better than pointwise loss for P@1 metric, while pointwise loss is superior than pairwise loss for MRR metric. Our experiments are mainly designed for comparing GGSA with other encoders like LSTM, rather than comparing upper structures. So the loss function adopted in each dataset is selected for fair comparison. With this comparison of these two loss functions, the state-of-the-art P@1 result on yahooQA can be further improved by using pairwise loss.