Graph-Structured Referring Expression Reasoning in The Wild
Table 3: Ablation study on Ref-Reasoning dataset. The best performing method is marked in bold.
['[EMPTY]', 'Number of Objects one', 'Number of Objects two', 'Number of Objects three', 'Number of Objects >= four', 'Split val', 'Split test']
To demonstrate the effectiveness of reasoning under the guidance of scene graphs inferred from referring expressions as well as the design of neural modules, we train four additional models for comparison. All the models have similar performance on the split of expressions directly describing the referents. For the other splits, SGMN without the Transfer module and SGMN without the Norm module have much lower performance than the original SGMN because the former treats the referent as an isolated node without performing relation reasoning while the latter unfairly treats different relational edges and the nodes connected by them. Next, we explore different options of the function (i.e., max, min and sum) used in the Merge module. Compared to SGMN with sum-merge, its performance with min-merge and max-merge drops because max-merge only captures the most significant relation for each intermediate node and min-merge is sensitive to parsing errors and recognition errors.