Transferable Representation Learning in Vision-and-Language Navigation
Table 2: Results for Validation Seen and Validation Unseen, when trained with a small fraction of Fried-Augmented ordered by scores given by model trained on cma. SPL and SR are reported as percentages and NE and PL in meters.
['Dataset size', 'Strategy', '[BOLD] Validation Seen PL', '[BOLD] Validation Seen NE ↓', '[BOLD] Validation Seen SR ↑', '[BOLD] Validation Seen SPL ↑', '[BOLD] Validation Unseen PL', '[BOLD] Validation Unseen NE ↓', '[BOLD] Validation Unseen SR ↑', '[BOLD] Validation Unseen SPL ↑']
Scoring generated instructions. We use this trained model to rank all the paths in Fried-Augmented and train the RCM agent on different portions of the data. Using high-quality examples–as judged by the model–outperforms the ones trained using low-quality examples. Note that the performance is low in both cases because none of the original human-created instructions were used—what is important is the relative performance between examples judged higher or lower. This clearly indicates that the model scores instruction-path pairs effectively.