Self-supervised Knowledge Triplet Learning for Zero-shot Question Answering
Table 1: Accuracy comparison with our baseline models on the SocialIQA dataset. We compare the models on the Zero-shot task. We compare them on both Train-Val split (35k) and the Validation split (2k), to enable measuring better statistical significance.
['[BOLD] Type', '[BOLD] Model', '[BOLD] Train-Val', '[BOLD] Val', '[BOLD] Deviation']
We can observe our KTL trained models perform significantly well compared to the baselines. When comparing the different KRL models, the NCELoss with Cosine similarity performs the best. This might be due to the additional supervision provided by the negative samples as the L2Loss model only tries to minimize the two projections. When comparing different KTL instantiations we can see the SMLM model performs the best overall but has a slightly higher deviation. We are analyzing our model to understand this phenomenon better. Our KRL model does perform equal to the current state-of-the-art model, Self-Talk Shwartz et al. Our models have half the parameters compared to Self-Talk. We also compare our model on the entire Train-Val set of 35,364 questions in the zero-shot setting to better gauge the statistical significance of the different model accuracies.