Transferable Representation Learning in Vision-and-Language Navigation
Table 3: AUC performance when the model is trained on different combinations of the two tasks and evaluated on the dataset containing only PR and RW negatives.
['Training', 'Val. Seen', 'Val. Unseen']
Improvements from Adding Coherence Loss. Finally we show that training a model on cma and nvs simultaneously improves the model’s performance when evaluated on cma alone. The model is trained using combined loss αLalignment+(1−α)Lcoherence with α=0.5 and is evaluated on its ability to differentiate incorrect instruction-path pairs from correct ones. As noted earlier, PS negatives are easier to discriminate, therefore, to keep the task challenging, the validation sets were limited to contain validation splits from PR and RW negative sampling strategies only. The area-under ROC curve (AUC) is used as the evaluation metric.