Learning with Latent Language
Table 1: Evaluation on image classification. Val (old) and Val (new) denote subsets of the validation set that contain only previously-used and novel visual concepts respectively. L3 consistently outperforms alternative learning methods based on multitask learning, meta-learning, and meta-learning jointly trained to predict descriptions (Meta+Joint). The last row of the table shows results when the model is given a ground-truth concept description rather than having to infer it from examples.
['Model', 'Val (old)', 'Val (new)', 'Val', 'Test']
It can be seen that L3 provides consistent improvements over the baselines, and that these improvements are present both when identifying new instances of previously-learned concepts and when discovering new ones. The model often succeeds in making correct predictions, even though its inferred descriptions rarely match the ground truth. Sometimes this is because of inherent ambiguity in the description language