RankQA: Neural Question Answering with Answer Re-Ranking
Table 4: Feature importance (i. e., averaged performance of exact matches on a hold-out sample). We train the general model using the same data, but blind one group of features every time. We underline results that undershoot the baseline and mark results in bold that surpass the general model trained on all features.
['[EMPTY]', '[BOLD] SQuADOPEN', '[BOLD] CuratedTrec', '[BOLD] WebQuestions', '[BOLD] WikiMovies']
Feature importance: Tbl. This is measured by training the model with the same parameters and hyperparameters as before; however, we blind one (group of) feature(s) in every round. This was done as follows: when the information retrieval or machine comprehension features were blinded, we also removed the corresponding aggregated features. When omitting aggregation features, we keep the original un-aggregated feature. We show the performance of DrQA (i. e., system without answer re-ranking) and the full re-ranker for the sake of comparison. The original performance increase can only be achieved when all features are included. This has important implications for our approach to properly fusing information from information retrieval and machine comprehension. It suggests that aggregation features are especially informative and that it is not sufficient to use only a subset of those.