Object-and-Action Aware Model for Visual Language Navigation
Table 1: Ablation study of the proposed method on the R2R dataset (SPL is the main metric). The proposed object-and-action aware model (OAAM) and nearest point loss (NPL) mainly improve performance on the Unseen and Seen scenarios, respectively. Their combination leads to a better performance in unseen environments.
['[BOLD] Model Baseline', '[BOLD] Model OAAM', '[BOLD] Model NPL', '[BOLD] Val Seen SR↑', '[BOLD] Val Seen OSR↑', '[BOLD] Val Seen SDTW↑', '[BOLD] Val Seen SPL↑', '[BOLD] Val Seen TL↓', '[BOLD] Val Seen CLS↑', '[BOLD] Val Unseen SR↑', '[BOLD] Val Unseen OSR↑', '[BOLD] Val Unseen SDTW↑', '[BOLD] Val Unseen SPL↑', '[BOLD] Val Unseen TL↓', '[BOLD] Val Unseen CLS↑']
We conduct the ablation study to find out the effectiveness of the proposed object-and-action aware model (OAAM) and the nearest point loss (NPL). The results show that: (I) Both OAAM and NPL improve the performance on the Val Seen split with about 2% and 4% increase in SPL, respectively. (II) The phenomenon on Val Unseen is a bit complicated. When OAAM or NPL works alone with the baseline, OAAM brings about 1% improvement in SR and NPL harms the SR about 2%. When OAAM and NPL work together, the performance is significantly improved about 4% in SR and 3% in SPL. This can be attributed to that NPL is able to help OAAM to find viewpoints on the ground-truth path and therefore improves the SR as well as shortens the trajectory length compared to the case that they work separately. Overall, both OAAM and NPL facilitate our method to achieve better performance, especially in the unseen scenario.