Visually Grounded Continual Learning of Compositional Semantics
Table 2: Overall performance of methods in MS COCO dataset and Flickr30k dataset.
['[BOLD] Dataset  [BOLD] Method/Metrics', '[BOLD] COCO-shift  [BOLD] PPL', '[BOLD] COCO-shift  [BOLD] Noun acc.', '[BOLD] COCO-shift  [BOLD] Verb acc.', '[BOLD] COCO-shift  [BOLD] Adj. acc.', '[BOLD] Flickr-shift  [BOLD] PPL', '[BOLD] Flickr-shift  [BOLD] Noun acc.', '[BOLD] Flickr-shift  [BOLD] Adj. acc.']
We see a clear performance gap from the comparison between vanilla online training and the offline methods. We see the largest gap in the prediction accuracy of nouns, which are bound with task identities according to our stream construction. We also see ER could alleviate forgetting, but the performance is close to offline training only when the replay buffer is very large (|M|=100,000, about 20% of the training examples). It contradicts the performance of ER in popular benchmark datasets, where storing only a few examples is believed to be sufficient to achieve a good performance Chaudhry et al. We see MIR, which is a state-of-the-art continual learning algorithm, could improve perplexity and prediction accuracy on nouns on two datasets at the same memory cost compared to ER. However, there is still a huge space where the performance can be improved.