MultiFC: A Real-World Multi-Domain Dataset forEvidence-Based Fact Checking of Claims
Table 7: Ablation results with base model crawled_ranked for different types of metadata
['[BOLD] Metadata', '[BOLD] Micro F1', '[BOLD] Macro F1']
S5SS2SSS0Px2 Metadata. Out of the different types of metadata, topic tags on their own contribute the most. This is likely because they offer highly complementary information to the claim text of evidence pages. Only using all metadata together achieves a higher Macro F1 at similar Micro F1 than using no metadata at all. To further investigate this, we split the test set into those instances for which no metadata is available vs. those for which metadata is available. We find that encoding metadata within the model hurts performance for domains where no metadata is available, but improves performance where it is. In practice, an ensemble of both types of models would be sensible, as well as exploring more involved methods of encoding metadata.