Speaker-Follower Models for Vision-and-Language Navigation
Table 1: Ablations showing the effect of each component in our model. Rows 2-4 show the effects of adding a single component to the baseline system (Row 1); Rows 5-7 show the effects of removing a single component from the full system (Row 8). NE is navigation error (in meters); lower is better. SR and OSR are success rate and oracle success rate (%); higher is better. See Sec. 4.2.1 for details.
['#', 'Data Augmentation', 'Pragmatic Inference', 'Panoramic Space', 'Validation-Seen NE ↓', 'Validation-Seen SR ↑', 'Validation-Seen OSR ↑', 'Validation-Unseen NE ↓', 'Validation-Unseen SR ↑', 'Validation-Unseen OSR ↑']
4.2.1 Component Contributions We first introduce the speaker at training time for data augmentation (Sec. , we see that adding the augmented data improves success rate (SR) from 40.3% to 46.8% on validation seen and from 19.9% to 24.6% on validation unseen, respectively. This higher relative gain on unseen environments shows that the follower can learn from the speaker-annotated routes to better generalize to new scenes.