Guided Feature Transformation (GFT): A Neural Language Grounding Module for Embodied Agents
Table 2: The evaluation results for 10k test sessions. The average navigation success rates are reported as percentages. Numbers in bold represent the best ones. The second number in each cell represents the standard deviation over twelve test runs, each run corresponding to one pass (out of three final passes) of one trained model (out of four random initializations).
['[EMPTY]', 'nav 2D', 'nav 3D', 'nav_avoid 2D', 'nav_avoid 3D', 'nav_bw 2D', 'nav_bw 3D', 'nav_dir 2D', 'nav_dir 3D', 'nav_near 2D', 'nav_near 3D']
Results. We observe that GFT-1 has some marginal improvement on the success rates of the best-performing comparison methods such as FiLM and Gated. As we perform a second feature transformation, GFT-2 produces a performance jump. Our explanation is that the visual recognition challenge, with random object yaws and scales in each session, requires an expressive language grounding function that can be better modeled by multiple steps of Eq. GFT-2 produces the best numbers in all the five columns. Unsurprisingly, nav_avoid has the highest rate because the agent only has to go to an arbitrary target which is not the specified one. nav has the lowest rate because the agent cannot exploit any object arrangement pattern like in nav_bw. See Results. We observe that GFT-1 already has a huge advantage over the best-performing comparison method FiLM. On top of this, GFT-2 shows a faster performance increase during the training time. This suggests that the original feature space cannot easily comply with various language commands. A rotation of the feature space depending on the input command, an operation which FiLM lacks but GFT-1 owns, is important for producing better grounding results. Unlike in the 2D case, now for GFT-2, nav has the second-best success rate while nav_dir has the lowest rate. Visualization of the nav_dir test cases reveals that due to severe perspective distortion in 3D, the agent has some difficulty of grounding the spatial-relation words, especially when multiple objects are located nearby. But still, GFT agents obtain much better nav_dir results than the comparison methods (over a 15% increase on average).