Towards Learning a Generic Agent forVision-and-Language Navigation via Pre-training
Table 5: Ablation study on R2R: feature-based vs fine-tuning. Blue indicates the better value.
['Methods', 'Validation Seen TL ↓', 'Validation Seen NE ↓', 'Validation Seen SR ↑', 'Validation Seen SPL ↑', 'Validation Unseen TL ↓', 'Validation Unseen NE ↓', 'Validation Unseen SR ↑', 'Validation Unseen SPL ↑', 'Test Unseen TL ↓', 'Test Unseen NE ↓', 'Test Unseen SR ↑', 'Test Unseen SPL ↑']
(ii) feature-based approach, where fixed features are extracted from the pre-trained model, and only the task-specific layer is updated. In this paper, all Prevalent presented results generally have used the feature-based approach, as there are major computational benefits to pre-computing an expensive representation of the training data once, and then running many experiments with cheaper models on top of this representation. In the R2R dataset, we consider a two-stage scheme, where we fine-tune the cross-attention layers of the agent, after training via the feature-based approach. We observe notable improvement with this two-stage scheme on nearly all metrics, expect the trajectory length.