BabyWalk: Going Farther in Vision-and-Language Navigationby Taking Baby Steps
Table 2: VLN agents trained on the r4r dataset and evaluated on the unseen portion of the r4r (in-domain) and the other 3 out-of-the-domain datasets: r2r, r6r and r8r with different distributions in instruction length. The Appendix has more comparisons. (+: pre-trained with data augmentation. ⋆: reimplemented or adapted from the original authors’ public codes).
['Setting', '[BOLD] In-domain r4r → r4r', '[BOLD] In-domain r4r → r4r', '[BOLD] In-domain r4r → r4r', '[BOLD] Generalization to other datasets r4r → r2r', '[BOLD] Generalization to other datasets r4r → r2r', '[BOLD] Generalization to other datasets r4r → r2r', '[BOLD] Generalization to other datasets r4r → r6r', '[BOLD] Generalization to other datasets r4r → r6r', '[BOLD] Generalization to other datasets r4r → r6r', '[BOLD] Generalization to other datasets r4r → r8r', '[BOLD] Generalization to other datasets r4r → r8r', '[BOLD] Generalization to other datasets r4r → r8r', '[BOLD] Generalization to other datasets Average', '[BOLD] Generalization to other datasets Average', '[BOLD] Generalization to other datasets Average']
In-domain Generalization This is the standard evaluation scenario where a trained agent is assessed on the unseen split from the same dataset as the training data. The BabyWalk agents outperform all other agents when evaluated on cls and sdtw.