Cross-media Structured Common Space for Multimedia Event Extraction
Table 3: Event and argument extraction results (%). We compare three categories of baselines in three evaluation settings. The main contribution of the paper is joint training and joint inference on multimedia data (bottom right).
['[BOLD] Training', '[BOLD] Model', '[BOLD] Text-Only Evaluation  [BOLD] Event Mention', '[BOLD] Text-Only Evaluation  [BOLD] Event Mention', '[BOLD] Text-Only Evaluation  [BOLD] Event Mention', '[BOLD] Text-Only Evaluation  [BOLD] Argument Role', '[BOLD] Text-Only Evaluation  [BOLD] Argument Role', '[BOLD] Text-Only Evaluation  [BOLD] Argument Role', '[BOLD] Image-Only Evaluation  [BOLD] Event Mention', '[BOLD] Image-Only Evaluation  [BOLD] Event Mention', '[BOLD] Image-Only Evaluation  [BOLD] Event Mention', '[BOLD] Image-Only Evaluation  [BOLD] Argument Role', '[BOLD] Image-Only Evaluation  [BOLD] Argument Role', '[BOLD] Image-Only Evaluation  [BOLD] Argument Role', '[BOLD] Multimedia Evaluation  [BOLD] Event Mention', '[BOLD] Multimedia Evaluation  [BOLD] Event Mention', '[BOLD] Multimedia Evaluation  [BOLD] Event Mention', '[BOLD] Multimedia Evaluation  [BOLD] Argument Role', '[BOLD] Multimedia Evaluation  [BOLD] Argument Role', '[BOLD] Multimedia Evaluation  [BOLD] Argument Role']
The comparison with other multimedia models demonstrates the effectiveness of our model architecture and training strategy. The advantage of structured embedding is shown by the better performance over the flat baseline. Our model outperforms its text-only and image-only variants on multimedia events, showing the inadequacy of single-modal information for complex news understanding. Furthermore, our model achieves better performance on text-only and image-only events, which demonstrates the effectiveness of multimedia training framework in knowledge transfer between modalities.