Multimodal Analytics for Real-world News using Measures of Cross-modal Entity Consistency
Table 1. Number of test documents |D|, unique entities T∗ in all articles, and mean amount of unique entities ¯¯¯¯T in articles containing a given entity type (for context this is the mean amount of nouns as explained in Section 3.1.2) for TamperedNews (left) and News400 (right). Valid image-text relations for News400 were first manually verified according to Section 4.1.3.
['[BOLD] News400 dataset  [BOLD] Documents', '[BOLD] News400 dataset | [ITALIC] D|', '[BOLD] News400 dataset  [ITALIC] T∗', '[BOLD] News400 dataset ¯¯¯¯ [ITALIC] T']
They were both manipulated to perform experiments for cross-modal consistency verification. Experiments and comparisons to related work (jaiswal2017multimedia; sabir2018deep) on datasets such as MEIR (sabir2018deep) are not reasonable since 1) they do not contain public persons or events, and 2) rely on pre-defined reference or training data for given entities. These restrictions severely limit the application in practice. We propose an automated solution for real-world scenarios that works for public personalities and entities represented in a knowledge base.