Tactical Rewind: Self-Correction via Backtrackingin Vision-and-Language Navigation
Table 3: Performance of different fusion methods for scoring partial trajectories. Tested on the validation unseen set.
['[ITALIC] Heur/step', '[ITALIC] Combine', '[ITALIC] SR', '[ITALIC] SPL', '[ITALIC] Len']
For example, the third row of the table shows that summing the logit scores of nodes along the partial trajectory as the L score for that trajectory achieves an SR score of 56.66. Note although all information originates with the same hidden vectors, the values computed and how they are aggregated substantially affect performance. Overall, we find that summing unnormalized logits (the 3rd row) performs the best considering its outstanding SR. This suggests that important activation information in the network outputs is being thrown away by normalization and therefore discarded by other techniques. These are motivated by beam-rescoring techniques in previous work (e.g., multiplying by the normalized progress monitor score). Correctly integrating signals is challenging, in part due to differences in scale. For example, the logit is unbounded (+/-), log probabilities are unbounded in the negative, and the progress monitor is normalized to a score between 0 and 1. Unfortunately, direct integration of the progress monitor did not yield promising results, but future signals may prove more powerful.