Representation Learning for Grounded Spatial Reasoning
Table 2: Performance of models trained via reinforcement learning on a held-out set of environments and instructions. Policy quality is the true expected normalized reward and distance denotes the Manhattan distance from goal location prediction to true goal position. We show results from training on the local and global instructions both separately and jointly.
['[EMPTY]', 'Local Policy Quality', 'Local Distance', 'Global Policy Quality', 'Global Distance', 'Combined Policy Quality', 'Combined Distance']
Following Schaul et al. \shortciteschaul2015universal, we also evaluated our model using the metric of policy quality. This is defined as the expected discounted reward achieved by following a softmax policy of the value predictions. Policy quality is normalized such that an optimal policy has a score of 1 and a uniform random policy has a score of 0. Intuitively, policy quality is the true normalized expectation of score over all maps in the dataset, instructions per map, and start states per map-instruction pair. While the UVFA variant learned a better policy on the train set, it did not generalize to new environments as well as the CNN + LSTM.