Answering Complicated Question Intents Expressed in Decomposed Question Sequences
Table 2: Dev accuracy of different table rewriting policies; the upper bound represents an almost 10% absolute improvement that the other policies do not come close to reaching due to the poor baseline performance of fp.
['Policy', 'Dev Acc', 'Dev Oracle']
The most important takeaway is that accuracy improvements are very small when we rewrite based on the previous predictions. Intuitively, this makes sense: if the parser only gets 30% accuracy, then 70% of the time it will be incorrect on the previous question, and rewriting the table based on a wrong answer could make it impossible for the parser to get the right answer (see the lower oracle scores for always rewrite and rewrite row/subset). Based on these results, table rewriting will only be useful if the base parser’s accuracy is high.