Talk the Walk: Navigating Grids in New York City through Grounded Dialogue
Table 10: Results for landmark classification.
['Features', 'Train loss', 'Valid Loss', 'Train F1', 'Valid F1', 'Valid prec.', 'Valid recall']
We compare to an “all positive” baseline that always predicts that the landmark class is visible and observe that all presented models struggle to outperform this baseline. Although 256-dimensional ResNet features achieve slightly better precision on the validation set, it results in much worse recall and a lower F1 score. Our results indicate that perceptual grounding is a difficult task, which easily merits a paper of its own right, and so we leave further improvements (e.g. better text recognizers) for future work.