Talk the Walk: Navigating Grids in New York City through Grounded Dialogue
Table 6: Full task performance of localization models trained on human and random trajectories. There are small benefits for training on random trajectories, but the most important hyper-parameter is to condition the tourist utterance on a single observation (i.e. trajectories of size T=0.) at evaluation time.
['Trajectories', 'T', 'Train', 'Valid', 'Test']
After training the supervised tourist model (conditioned on observations and action from human expert trajectories), there are two ways to train an accompanying guide model. We can optimize a location prediction model on either (i) extracted human trajectories (as in the localization setup from human utterances) or (ii) on all random paths of length T (as in the full task evaluation). Here, we investigate the impact of (1) using either human or random trajectories for training the guide model, and (2) the effect of varying the path length T during the full-task evaluation. For random trajectories, guide training uses the same path length T as is used during evaluation. We use a pre-trained tourist model with greedy decoding for generating the tourist utterances.