Look, Read and Enrich Learning from Scientific Figures and their Captions
Table 5: Caption and figure classification (%accuracy)
['Model', 'Caption Non-trainable', 'Caption Trainable', 'Figure Non-trainable', 'Figure Trainable']
The delta is particularly noticeable in the non trainable case for both caption and figure classification and is considerably increased in ”Ours FCC7”, which uses pre-trained semantic embeddings. This includes both the random and VGG baselines and illustrates again the additional complexity of analyzing scientific figures compared to natural images, even if the latter is trained on a considerably larger corpus like ImageNet. Fine tuning the whole networks on SciGraph further improves accuracies. In this case, ”Ours FCC6”, which uses FCC features without additional pre-trained embeddings, slightly outperforms ”Ours FCC7”, suggesting a larger margin to learn from the task-specific corpus. Note that both FCC6 and FCC7 were trained on SemScholar.