Towards Question Format Independent Numerical Reasoning: A Set of Prerequisite Tasks
Table 6: Table showing the performance of various baselines on our test set across all data types
['[EMPTY]', 'Type 1 EM', 'Type 1 F1', 'Type 2 EM', 'Type 2 F1', 'Type 3 EM', 'Type 3 F1', 'Type 4 EM', 'Type 4 F1', 'Type 5 EM', 'Type 5 F1', 'Type 6 EM', 'Type 6 F1', 'Type 7 EM', 'Type 7 F1', 'Type 8 EM', 'Type 8 F1']
Performance of all the baseline models is significantly less than human baseline. Answering questions in eight different setting using a single model is indeed a challenging task. We find that, in some of the cases, the model fails to distinguish among the question types. For example, it gave a span based answer where a number was expected and vice versa in some cases.