Towards Learning a Generic Agent forVision-and-Language Navigation via Pre-training
Table 3: Results on test splits of HANNA. The agent with “perfect assistance” uses the teacher navigation policy to make decisions when executing a subtask from the assistant. Blue indicates the best value.
['[EMPTY]', 'Agent', 'SEEN-ENV SR ↑', 'SEEN-ENV SPL ↑', 'SEEN-ENV NE ↓', 'SEEN-ENV #R ↓', 'UNSEEN-ALL SR ↑', 'UNSEEN-ALL SPL ↑', 'UNSEEN-ALL NE ↓', 'UNSEEN-ALL #R ↓']
Our Prevalent outperforms the baseline agent Anna on the test unseen environments in terms of SR, SPL and NE, while requesting a slightly higher number of helps (#R). When measuring the performance gap between seen and unseen environments, we see that Prevalent shows a significantly smaller difference than Anna, e.g., (59.38-28.72=30.66) vs (63.92-25.50=38.42) for SPL. This means that the pre-trained joint representation by Prevalent can reduce over-fitting, and generalise better to unseen environments.