Answering Complex Open-domain Questions Through Iterative Query Generation
Table 4: Pipeline ablative analysis of GoldEn Retriever end-to-end QA performance by replacing each query generator with a query oracle.
['[BOLD] System', '[BOLD] Ans F1', '[BOLD] Sup F1', '[BOLD] Joint F1']
Lastly, we perform an ablation study in which we replace our query generator models with our query oracles and observe the effect on end-to-end performance. only slightly improves end-to-end performance, but further substituting G2 with the oracle yields a significant improvement. This illustrates that the performance loss is largely attributed to G2 rather than G1, because G2 solves a harder span selection problem from a longer retrieval context. In the next section, we examine the query generation models more closely by evaluating their performance without the QA component.