Multimodal Analytics for Real-world News using Measures of Cross-modal Entity Consistency
Table 5. Results for document verification (DV) and collection retrieval for the the News400 dataset. Results are reported for all available and verified documents |D|.
['[BOLD] Test set (| [ITALIC] D|)', '[BOLD] DV  [BOLD] VA', '[BOLD] Collection Retrieval  [BOLD] AUC', '[BOLD] Collection Retrieval  [BOLD] AP-clean', '[BOLD] Collection Retrieval  [BOLD] AP-clean', '[BOLD] Collection Retrieval  [BOLD] AP-clean', '[BOLD] Collection Retrieval  [BOLD] AP-tampered', '[BOLD] Collection Retrieval  [BOLD] AP-tampered', '[BOLD] Collection Retrieval  [BOLD] AP-tampered']
Since the number of documents is rather limited and cross-modal mutual presence of entities was manually verified, results for News400 are reported for all documents with verified relations. However, results while retrieving tampered documents are noticeable worse. This is mainly caused by the fact that some untampered entities in the documents that are depicted in both image and text, can be either unspecific (e.g. mentioning of a country) or the retrieved images for visual verification do not fit the document’s image content. this problem was bypassed. We have verified the same behavior for News400 when experimenting on these subsets. In addition, performance for context verification is worse compared to TamperedNews. We assume that this is due to the less powerful word embedding for the German language.