Listen, Attend, and Walk: Neural Mappingof Navigational Instructions to Action Sequences
Table 2: Model components ablations
['[EMPTY]', 'Full Model', 'High-level Aligner', 'No Aligner', 'Unidirectional', 'No Encoder']
Unlike most existing methods that align based only on the hidden annotations hj, we adopt a different approach by also including Our model utilizes alignment in the decoder as a means of focusing on word “regions” that are more salient to the current world state. Note that the “No Aligner” model still maintains all connections between the instruction and actions, but is just using non-learned, uniform weights. We train an alternative model that uses only a unidirectional (forward) encoder. We further evaluate the benefit of encoding the input sentence and consider an alternative model that directly feeds word vectors as randomly initialized embeddings into the decoder and relies on the alignment model to choose the salient words. We believe the difference is due to the RNN’s ability to incorporate sentence-level information into the word’s representation as it processes the sentence sequentially (in both directions). This advantage helps resolve ambiguities, such as “turn right before …” versus “turn right after …”.