SelQA: A New Benchmark for Selection-based Question Answering
TABLE XIV: Accuracies on the SelQA evaluation set for answer triggering w.r.t. question types.
['[BOLD] Type', '[BOLD] CNN0', '[BOLD] CNN2', '[BOLD] RNN1', '[BOLD] Q']
Interestingly, each model shows different strength on different types, which may suggest a possibility of an ensemble model. All the models tend to perform better as questions become longer. Similarly as in the answer sentence selection task, since longer questions are more informative, it is understandable. Interestingly, once the section becomes longer, the accuracy increases. We hypothesize that such a behavior might be caused by the fact that it is easier for the models to decide whether the context of the section is the same as the context of the question when there is more information (sentences) in the section. Thus, this phenomenon is related to the task of answer triggering, where the model not only choose the sentence with the answer, but must decide if the context matches first.